Part 4.2 is currently released as a fully reproducible code package, with a talk forthcoming.
The slides present the high-level results, and all training code and recipes are available in the GitHub repository. 

Authors: Zeyuan Allen-Zhu

Code release for Part 4.2: https://github.com/facebookresearch/PhysicsLM4
Mode release for Part 4.2: huggingface (only for 16 Transformer models)

Slide show (best viewed on a computer)

@misc{Allen2025-resonate,

    title = {{Physics of Language Models: Part 4.2, Canon Layers at Scale where Synthetic Pretraining Resonates in Reality}},

    author = {{Allen-Zhu}, Zeyuan},

    year = {2025},

    url = {https://physics.allen-zhu.com/part-4-architecture-design/part-4-2},

    note = {Code released at \url{https://github.com/facebookresearch/PhysicsLM4}},

}