Full Fine-tuning

conceptstub

Updating all weights of a pre-trained model.

Planned content

• FSDP/ZeRO-3 as the default
• 1–16 node scale
• Memory planning and checkpointing