PEFT

A family of fine-tuning methods that adapt large pre-trained models by updating only a small number of parameters instead of retraining the full model. Parameter-Efficient Fine-Tuning (PEFT) reduces the cost of adapting large AI models by freezing most of the base model and training lightweight added parameters, adapters, prompts, or low-rank matrices.

PEFT methods such as LoRA, adapters, prefix tuning, and prompt tuning make it practical to customise LLMs and diffusion models with far less GPU memory, storage, and training time than full fine-tuning.

Parameter-Efficient Fine-Tuning is a foundational concept in Training Technique. A family of fine-tuning methods that adapt large pre-trained models by updating only a small number of parameters instead of retraining the full model.