Jeff: Fast, Local AI Decision Models for Code Integration
Jeff is a set of small, fast decision models designed for easy integration into code. Fine-tuned versions of Qwen3.5 and Gemma 4, these models provide calibrated probabilities for options in zero-shot scenarios, meaning they can handle any category without prior training data. They are optimized for speed, with decisions made in about 22-28 milliseconds on high-end hardware. Jeff is built entirely on local hardware, using synthetic training data generated by open models, ensuring no reliance on cloud GPUs or closed-model output. The project is independent and not affiliated with Jev, though it uses the same request format. Jeff’s performance is benchmarked against Jev, showing competitive results in classification tasks but lagging in reasoning-heavy scenarios due to its smaller size. Fine-tuning on specific datasets can significantly improve accuracy, as demonstrated in a voice-navigation example where accuracy jumped from 31.7% to 95.8%.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.