NVIDIA´s NemoSkills team clinched first place in the latest Kaggle AI Mathematical Olympiad competition, showcasing a powerful new reasoning model adept at solving challenging math problems. The transcontinental team engaged in a day-night relay, leveraging expertise from members across the U.S., Armenia, Finland, Germany, and Northern Ireland. Each day’s iterative submissions resulted in a model that could tackle 34 out of 50 Olympiad-level problems within a five-hour window, powered by a cluster of four NVIDIA L4 GPUs.
To win the inference challenge, the team fine-tuned Qwen2.5-14B-Base—a foundation large language model with robust chain-of-thought capabilities—on millions of synthetically generated solutions. These training solutions were distilled from even larger models, such as DeepSeek-R1 and QwQ-32B, enabling the NVIDIA model to think through complex math questions using a blend of natural language reasoning and Python code execution. The team boosted efficiency through parallel reasoning chains and innovative early-stopping techniques with the NeMo-Skills pipeline and NVIDIA TensorRT-LLM. FP8 quantization and Apple’s ReDrafter speculative decoding contributed major speed improvements, helping the team meet rigorous competition time constraints.
The model’s strong performance on the unseen final dataset, surpassing public leaderboard results, demonstrated exceptional generalization and minimized overfitting—a key goal in Artificial Intelligence model development. Following their victory, the team is set to publish a technical report and release datasets and models on Hugging Face. These advancements also contributed to the just-released NVIDIA Llama Nemotron Ultra model. Notably, the Kaggle Grandmasters of NVIDIA (KGMON) initiative played a pivotal role, with team member Christof Henkel regaining the world champion title among 23 million Kaggle users and Ivan Sorokin achieving Grandmaster status. The team pledged their prize to the NVIDIA Foundation for charitable causes, solidifying their collaborative and impactful approach to furthering Artificial Intelligence reasoning research.
