#GGUF (3)

Fine-tune, Convert and Deploy Language Models on NVIDIA Jetson with Unsloth and llama.cpp

Official Jetson AI Lab tutorial: fine-tune models directly on Jetson with Unsloth and JetPack 7.2 using memory-efficient QLoRA, export to GGUF, and run locally with llama.cpp. Two hands-on examples: Qwen3.5-4B vision-language model on Jetson Orin Nano (LaTeX OCR fine-tuning), and NVIDIA Nemotron 3.5 Lightning 30B-A3B on Jetson AGX Thor (3 training steps in 44.8s, 66.1 tokens/sec with Q4_K_M quantization). No cloud required.

BLOG