Fine-tuning an LLM usually means SSH and config hell. Not on a 4 GB laptop GPU.
GitHubTrends@githubtrends · 235 followers
- Views
- 880
- Likes
- 19
- Comments
- 0 comments
- Shares
- 2
- Saves
- 16
Show more details Subtitles, video quality and playlist
Transcript
Fine tuning an LLM usually means SSH and config. Hell, soup streams the frozen 8B base, 1 decoder layer at a time onto a 4gb card. 1 YAML config and SOUP train run Q LA with auto quantization. 119.6 tokens per second, 3.32 GB peak layer. Streaming is still opt in beta, but fine tuning now happens on your own GPU, no cloud.