Microsoft's LASER Sharpens Large Language Models

Microsoft researchers have developed a new method called LASER that can improve the accuracy of large language models by removing some data correlations.


  • Microsoft announced LASER (Layer-Selective Rank Reduction), a new technique to improve accuracy of large language models (LLMs).
  • LASER works by replacing weight matrices in the model with smaller approximate ones, removing some correlations.
  • Counterintuitively, this makes the models smaller but also more accurate.
  • LASER was tested on models like RoBERTa, Llama 2 and GPT-J, improving accuracy by 20-30 percentage points.
  • For example, GPT-J's accuracy on gender prediction from biographies increased from 70.9% to 97.5% using LASER.
  • LASER helps address the issue of factual mistakes made by LLMs which can be harmful.
  • Improving LLM accuracy remains an important area of research to make AI language generation more reliable.


Related post


BharatGPT Aims to Become India's Meta for Indic Language Models

BharatGPT is an Indian initiative aimed at developing open source Indic language models from scratch to address the linguistic and cultural context of India, with the goal of becoming the leading provider of foundational models for the Indian subcontinent. BharatGPT wants to position itself as the "Meta" of Indic language…