abbreviation LoRA
low-rank adaptation
LoRA (low-rank adaptation) is a method that modifies large AI models, like those used for text or image generation, by adjusting only a small portion of their parameters. This avoids the need to retrain the entire model, saving time and resources.
It works by adding small, trainable matrices to the model’s existing layers, which capture new knowledge without altering the original weights. This makes it much cheaper to adapt models for specific tasks, like customizing an image generator for a particular art style.
For users, LoRA changes the game by making model customization accessible to individuals or small teams. Instead of needing expensive hardware or massive datasets, you can fine-tune a model on a single GPU or even a powerful laptop. For example, a digital artist could train a LoRA variant of Stable Diffusion to mimic their unique style using just 50-100 example images, costing only a few dollars in cloud computing fees.
A photographer uses LoRA to fine-tune a text-to-image model so it better understands their lighting techniques, reducing the need for manual edits.
In other languages
See also: fine-tuning