Sleeping LLM: Language Model Learns Through Sleep

This research introduces a novel approach to language model learning, drawing inspiration from the Complementary Learning Systems theory in neuroscience. The model utilizes a 'sleep' cycle to consolidate memories, transferring knowledge from a fast, brittle short-term memory (MEMIT) to a slow, stable long-term memory (LoRA). During wakefulness, facts are injected directly into the model weights via MEMIT. During sleep, a maintenance cycle audits and refreshes degraded memories, then LoRA consolidation progressively transfers knowledge from MEMIT to fused LoRA.

The results show that LoRA consolidation achieves a 100% fact advancement rate at various fact loads. The model also demonstrates sleep convergence, recovering to 100% recall within a few sleep cycles. However, the researchers observed an 'alignment tax' during wakefulness, where RLHF actively suppresses LoRA-injected knowledge. This led them to abandon direct LoRA training during wake and focus on consolidation during sleep.

This approach offers a promising way to improve LLM memory and learning. By incorporating sleep-like mechanisms, the model can consolidate knowledge and prevent the decay of information. However, further research is needed to address the challenges related to alignment and ensure the safety and reliability of the model. Transparency is crucial. Disclose the model's architecture and training process. This promotes reproducibility and facilitates further research.

*Transparency Disclosure: The analysis above was partially assisted by an AI model.*