Models Still Answer Nonsense. BullshitBench Shows Which Ones Don't
BullshitBench v2 tests whether AI models reject plausible nonsense instead of confidently answering it. The results are uncomfortable.
NESTFRONTIER / ARCHIVE
All entries
Technical analysis, model releases, research, and deployment notes.
PAGE_07
291 PUBLISHED ENTRIESAgents Still Fail Half Their Real World Jobs
A new 400-task benchmark finds leading agents pass less than half of realistic jobs, with memory, vision, and cross-app coordination still breaking most systems.
Four Billion Parameters Beat the Big Image Models on Speed
Microsoft's 4B Mage-Flow pairs a cheap tokenizer with native-resolution diffusion, hitting 0.59 seconds at 1024² on one A100 while exposing the tradeoffs.
Tutorials are becoming an agent's missing memory
Microsoft's Resource2Skill turns tutorials, code, and visual references into executable agent skills, scoring 11.9 points above agents without the wiki.
Training Agents Still Breaks at the Harness Layer
OpenForgeRL trains agents inside the real harnesses they use in deployment, posting strong tool-use and GUI results while exposing why error recovery remains hard.
Frontier Coding Just Got Cheaper Than The Hype
Grok 4.5 pairs frontier coding claims with $2 input pricing, 80 TPS, and an open agent harness. The cost story is more convincing than the leaderboard story.
Russian prompts expose a hidden coding agent gap
RuBench tests coding agents on fresh repository fixes written in native Russian, then audits what each product actually read and ran.
Half the price is easy. Token efficiency is the story.
Claude Opus 5 keeps Opus 4.8 pricing, adds adaptive effort controls, and targets cheaper long-running agent work.
Open weights just reached 2.8 trillion parameters
Moonshot AI's 2.8T-parameter Kimi K3 is close to the closed frontier, but the real test starts when its promised weights arrive.
Deep Research Got Better When AREX Learned to Doubt Itself
AREX improves deep research by verifying partial answers, preserving useful evidence, and sending unresolved claims into a targeted second pass.