Author: Dan Mount - Senior Product Manager
Read Time: 5 mins
The release of Grok 4 marks yet another chapter in what's been an extraordinary year for AI. Following models like Claude Sonnet 4.0, GPT-4o, and Gemini 2.5, Grok 4 joins a crowded field of large language models making bold claims, especially around reasoning and intelligence.
At Monolith, we welcome this progress. Our mission has always been to tackle the world’s toughest engineering problems (it’s why we started working with NASA). We thrive on complexity, and that’s why we’re well placed to assess whether these models are truly up to the challenge.
What exactly is Grok 4? In short, it’s not just a scaled-up transformer. Grok 4 – Elon Musk’s latest model from xAI – uses a novel hybrid architecture. It embraces massive reinforcement learning at pre-training scale and even a multi-agent “Heavy” mode (multiple AI reasoning units working in parallel) (Source).
In other words, rather than simply making the neural network larger, Grok 4 was designed for deep reasoning using native tool use and collaborative problem solving among its sub-models. It’s also huge (rumoured ~1.7 trillion parameters) and multimodal, but its reasoning-first design is what really sets it apart.
Source: Initial benchmarks from Axion Launchpad
The results are telling. Grok 4 has achieved 15.9% on the ARC-AGI v2 reasoning benchmark – nearly double the previous best (8.6% by Anthropic’s Claude Opus 4). In a field where most models struggled to get beyond a few percent on this test of fluid intelligence, Grok 4 now stands clearly ahead of every other known AI system.
Its creators even report top-tier performance on other challenging evaluations (e.g. 100% on the AIME math exam, major leaps on “Humanity’s Last Exam”). Benchmarks like these suggest we’re inching closer to models that can approach human-level problem solving.
Grok 4 ranks among top LLMs in hard prompts, coding, and math on the LMArena leaderboard (July 2025)
The hype is real – perhaps too real. Elon Musk has even boldly predicted that Grok 4 may “discover new technologies as soon as later this year”, saying he’d be shocked if it hasn’t by next year. He’s hinted that AI-driven scientific or engineering breakthroughs (even new physics) could be on the horizon within a year or two. Such promises highlight the excitement around these advances.
But even with better scores, there’s still a gap between AI capabilities in theory and usefulness in practice. For companies like ours—and the teams we serve—that distinction matters.
Even Musk acknowledges that today’s AI models remain “primitive tools, not the kind of tools that serious commercial companies use” (Source). In other words, high scores alone don’t automatically translate to real-world engineering value.
With so many new models appearing, we need a faster, more reliable way to evaluate them. That’s why we’re building an internal framework to assess each model’s strengths and weaknesses against realistic, mock customer workflows.
We’re testing how well these models:
This rigorous vetting helps us determine when a model is ready, not just for impressive demos, but to deliver real value to our customers.
We’re not just experimenting for fun. As models mature, we want to be prepared to integrate them meaningfully into our platform and service packages, enabling teams to work more efficiently and uncover new insights.
While we’re still in the early stages of trials, we believe models like Grok 4 could soon assist in everything from automating support to accelerating scientific discovery. But we’ll only deploy them once they meet our standards in practical settings.
At Monolith, we’re not here to ride the hype—we’re here to make AI work for engineering.
By setting our own high benchmarks and vetting each new model against real-world challenges, we ensure that AI’s rapid progress truly translates into tools that engineers can trust and benefit from.
Stay tuned for our next blog where we reveal our benchmarking outcomes.