Large language models typically remain static after deployment: their learned weights do not persistently change when a user provides new information, and in-context learning is ephemeral across conversations. Researchers at MIT propose a different approach that leverages a model’s in-context strengths to teach it how to update its own weights. Their framework, called seal for self-adapting LLMs, prompts a model to create multiple rewrites of an input passage as synthetic training examples, analogous to students making study sheets from a lecture.
The model generates several candidate self-edits, quizzes itself on each one, and uses a trial-and-error method based on reinforcement learning to select the self-edit that most improves downstream performance. It then applies that chosen self-edit to update its internal weights so the information is permanently internalized. The framework also lets the model choose aspects of the learning process, including which synthetic data to use, the learning rate, and how many training iterations to run, effectively enabling the model to configure its own optimization strategy.
In evaluations the approach improved accuracy on question-answering tasks by nearly 15 percent and increased success rates on some skill-learning tasks by more than 50 percent, and a small model using the technique outperformed much larger large language models on several benchmarks. The researchers note an outstanding limitation: catastrophic forgetting, where performance on earlier tasks degrades as the model continually adapts. The paper’s authors include jyothish pari, adam zweiger, han guo, ekin akyürek, yoon kim, and pulkit agrawal, and the work will be presented at the conference on neural information processing systems. The team plans to address catastrophic forgetting and explore multi-agent settings, with support from the U.S. Army Research Office, the U.S. Air Force Artificial Intelligence Accelerator, the Stevens Fund for MIT UROP, and the MIT-IBM Watson Artificial Intelligence Lab.
