abbreviation LoRA
Low-Rank Adaptation
Low-Rank Adaptation (LoRA) is a method that adjusts large AI models by modifying only a small subset of their parameters, rather than retraining the entire model. It works by adding low-rank matrices to the model’s existing layers, allowing it to learn new tasks with minimal computational cost.
LoRA is particularly useful for adapting pre-trained models to new tasks, such as generating specialized text or images, without the need for extensive training. This reduces the time and computational power required, making it accessible to users with limited resources.
For users, LoRA changes the way they fine-tune AI models by making the process faster and cheaper. It allows for quick customization of models for specific tasks, such as fine-tuning a language model to generate better responses in a niche domain. For example, a company could use LoRA to adapt a general AI model to understand and respond to customer service queries in a specific industry, reducing training time from days to hours.
A developer uses LoRA to fine-tune a pre-trained language model for medical terminology, cutting training costs by 90%.
See also: fine-tuning · Transfer Learning