Microsoft's LASER Sharpens Large Language Models

Microsoft researchers have developed a new method called LASER that can improve the accuracy of large language models by removing some data correlations.

Summary

Microsoft announced LASER (Layer-Selective Rank Reduction), a new technique to improve accuracy of large language models (LLMs).
LASER works by replacing weight matrices in the model with smaller approximate ones, removing some correlations.
Counterintuitively, this makes the models smaller but also more accurate.
LASER was tested on models like RoBERTa, Llama 2 and GPT-J, improving accuracy by 20-30 percentage points.
For example, GPT-J's accuracy on gender prediction from biographies increased from 70.9% to 97.5% using LASER.
LASER helps address the issue of factual mistakes made by LLMs which can be harmful.
Improving LLM accuracy remains an important area of research to make AI language generation more reliable.