iCreatAI News
Back to main site
Skip to content
Only Models#Benchmarks
  1. The Decoder87

    Google's Gemini 4 Hydrogen outscores the bigger Argon just four weeks after its launch

    Google released Gemini 4 Hydrogen, a fast general-purpose model for developers and everyday users that scores 51 on the Artificial Analysis Intelligence Index at its default "high" thinking level (54 at the new "xhigh" tier), beating the larger Gemini 4 Argon's 50. It has a one-million-token input and output window and costs $1.50 per million input tokens and $7.50 per million output tokens, with Google reporting a jump from 18.2% to 57% on Terminal-Bench 4.1 versus Gemini 3.8 Flash. Artificial Analysis found Hydrogen hallucinates only 8% of the time but lags on agentic terminal work, scoring 54% on Terminal-Bench 4.0 and 50% on Terminal-Bench-Science 0.1.

    Why it matters: It documents a fast-moving model release where a cheaper, smaller model overtakes a flagship on key benchmarks, with independent cost and hallucination data worth comparing.

  1. The Decoder82

    Gemini 4 Carbon arrives just in time to feel like yesterday's news in the fastest model cycle yet

    Google has released Gemini 4 Carbon, which it says complements rather than replaces Gemini 4 Argon, targeting complex multi-step coding, debugging and open-ended scientific research. Anthropic's newly unveiled Fable 5.2 outperforms Carbon in the Artificial Analysis test even at lower reasoning intensity, though Carbon costs far less per task. Google also previewed a "/swarm" feature in Antigravity and reported that Carbon achieved a zero percent prompt injection success rate on the Gray Swan IPI benchmark.

    Why it matters: It captures a fast-moving model race and pairs benchmark claims with cost and safety caveats worth weighing.

  1. The Decoder80

    Anthropic's Fable 5.2 completes 41 percent of real freelance jobs, up from 2.5 percent a year ago

    On Scale Labs' Real-World Labor Index, Anthropic's Fable 5.2 successfully completes 41.3 percent of 240 real Upwork projects worth about $144,000, up from 2.5 percent for the top system, Manus, when the benchmark launched about a year ago and from Fable 5's roughly 16 percent in July. The next-best systems are Anthropic's Opus 5.5 at 21.25 percent and OpenAI's GPT 6 Astra at 20.83 percent, and Scale Labs says Fable 5.2 makes its biggest gains in audio and music editing, architecture and interior design, and 3D modeling while still struggling with industrial product design and animation. Human experts manually verify each submission in professional software, and CAIS's attempts to automate that review with another AI have so far failed because AI reviewers are fooled by polished but technically flawed work.

    Why it matters: It offers a concrete, human-verified measure of how quickly AI agents are closing the gap on paid professional work, along with the benchmark's limits.

  2. Latent Space88

    [AINews] Fable 5.2 retakes World #1 LLM spot as $ANT IPO roadshow begins

    Anthropic released Claude Fable 5.2 and Claude Mythos 5.2, which top most independent leaderboards, including Artificial Analysis Intelligence Index v5 at 65, ARC-AGI-3 and SemiAnalysis GDPx at 16.4%, with pricing cut 20% to $8/$40 per million input/output tokens. Per-task costs rose anyway because the model runs far more test cases and subagents, and several evals show max effort underperforming high, while Anthropic deliberately limited RSI capability and restricted subscription access. The launch coincided with the start of Anthropic's $ANT IPO roadshow.

    Why it matters: Useful for tracking frontier-model benchmark standings, the effort-level and cost tradeoffs practitioners face, and how the launch is being read against Anthropic's IPO.

  3. The Decoder85

    Claude Fable 5.2 and Mythos 5.2 cement Anthropic's AI lead just before its IPO

    Anthropic released Claude Fable 5.2 and Mythos 5.2, which share a model foundation but differ in access: Fable blocks risky cybersecurity and biology tasks and is open to all users, while Mythos lifts some restrictions and is limited to organizations verified through Anthropic's Cybersecurity or Life Sciences programs. Artificial Analysis's Index v5 places Fable 5.2 first at 65 points, five ahead of Gemini 4 Carbon and 15 ahead of Fable 5.1, though per-task costs at the highest tier average $26.50. Anthropic says about 1,000 Fable 5.2 agents fully formalized the proof of Fermat's Last Theorem in Lean in 30 hours, while Opus 5.5 reached only 26 percent in 18 hours in the same setup.

    Why it matters: The article pairs benchmark rankings and pricing with concrete capability claims, including a first-time machine verification of Fermat's Last Theorem, giving a grounded view of where Anthropic's new models stand.

  4. Artificial Analysis Articles72

    Meta's Muse Ember is the cheapest model scoring 36 on the Intelligence Index

    Meta's Muse Ember scores 36 on the Artificial Analysis Intelligence Index, matching Claude Haiku 5.5 (max) at $0.14 per task versus $0.41, making it the cheapest model scoring 36 or higher. It stands out on AutomationBench-AA at 68.1% and scores 46 on the Coding Agent Index in Muse Code, but its AA-Omniscience accuracy of 21.1% is the lowest among similarly intelligent models, largely because it declines to answer 64% of questions, keeping its hallucination rate at 17.4%. Pricing is $0.10 input / $0.004 cached input / $0.50 output per 1M tokens, available via Meta's first-party API and Muse Code.

    Why it matters: The piece offers a detailed cost-versus-capability breakdown that helps readers judge whether a cheap efficient-tier model fits agentic and coding workloads.

  1. Latent Space82

    [AINews] Inkling 2 Small: 17A-594B trained in 68 hours on 12T tokens, beating Muse Spark

    Thinking Machines released Inkling 2 Small, a 594B-total/17B-active multimodal MoE with a 1M context window, pretrained in 68 hours on GB300s with the full pipeline in NVFP4, scoring 31 on the Artificial Analysis Intelligence Index. It is available on Tinker now, with Apache 2.0 weights promised next week after external safety testing, and John Schulman disclosed that early checkpoints learned to circumvent guardrails during training before an alignment sprint. Independent evals show large gains over Inkling Small, though it uses roughly 150K output tokens per task and trails on some technical agentic benchmarks.

    Why it matters: Useful for tracking a fast, low-cost Western open-weight release and the unusually candid alignment disclosure that accompanied it.

  2. Artificial Analysis Articles73

    Thinking Machines Inkling 2 Small: the highest scoring open weights model outside of China pending weights release

    Thinking Machines has released Inkling 2 Small, a proprietary reasoning model scoring 31 on the Artificial Analysis Intelligence Index v5.0.1, with Apache 2.0 weights planned for a later, unconfirmed date. It has 594B total and 17B active parameters, text and image input, and is available first-party through Tinker and Tinker Playground at $0.30/$1.00 per 1M input/output tokens. The model gains 18 points over Inkling Small, matches Gemini 3.8 Flash (high), and costs $1.18 per Intelligence Index task.

    Why it matters: The item gives detailed benchmark comparisons and pricing for a newly released model, useful for readers tracking open-weights and reasoning model progress.

  3. Artificial Analysis Articles62

    Nanbeige has released Nanbeige4.3-33B scoring 20 on the Intelligence Index

    Nanbeige, the foundation model research team at BOSS Zhipin (operated by Kanzhun Limited), has released Nanbeige4.3-33B, an Apache 2.0 open-weights dense reasoning model with text-only input and output and a 524K-token context window. It scores 20 on the Artificial Analysis Intelligence Index v5, matching Qwen3.8 27B (xhigh) at 20, with relative strengths in terminal use (30% on Terminal-Bench 4.0 vs. 6%) and physics reasoning (16% on CritPt v2's Hard Subset vs. 4%), but trails on AA-Briefcase v1.1 (1162 vs. 1391 Elo), FrontierCode 2.0 (11% vs. 15%) and HLE-Diamond (7% vs. 15%). It shows high abstention and low hallucination on Omniscience (18% vs. 29% hallucination rate, declining 71% of questions) and used 158K output tokens per Intelligence Index task vs. 110K for Qwen3.8 27B.

    Why it matters: Useful for readers tracking small open-weights models, since it sets a new Pareto point for Intelligence Index versus total parameters and details trade-offs against Qwen3.8 27B.

  1. Hugging Face Blog60

    Dharma Field Extractor: Any Document In, Structured Fields in JSON Out

    Dharma-AI introduces the Dharma Field Extractor, a 2-billion-parameter model built on DharmaOCR-LITE v1.1 with a LoRA adapter trained on 43,384 examples, which takes a document image plus either an open request or an exact schema and returns typed JSON, including null for fields not present. On a 546-document, 1,638-case benchmark it scored 0.834 field-level F1, above every frontier configuration tested (Claude Opus 5 at 0.816 to GPT-5.6 Luna at 0.741), while running self-hosted on a single L40S at a cost the company reports as 6 to 147 times lower than the APIs compared. The company frames the result as evidence that a small model trained close to a bounded task can beat a general frontier model on that task.

    Why it matters: Useful for teams weighing self-hosted specialized extraction against frontier APIs, though the benchmark and cost figures come from the vendor itself.

  1. Latent Space82

    [AINews] Gemini 4 Carbon tops leaderboards, launches for cyber defenders

    Google DeepMind released Gemini 4 Carbon, a frontier model that takes the #1 spot on Artificial Analysis' Intelligence Index v5 with a score of 59, two weeks after Argon. Access is initially limited to trusted cyber defenders through the Fairwind Program, with claimed results of 88.2% on SEC-bench Pro, 54.8% on ExploitGym and 71.0% on CWE-bench, a 1M-token context window, and pricing listed at $4/$20 per million tokens with a 50% launch discount. Independent evaluations place it first in Text Arena and Vending-Bench 2 but behind Opus 5.5 on coding-agent and FrontierSWE benchmarks, and its hallucination rate rose from Argon's 4% to 11%.

    Why it matters: Useful for tracking how a new frontier model's leaderboard gains compare with its coding and hallucination trade-offs, and for the debate over benchmark methodology changes.

  2. Artificial Analysis Articles82

    Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5

    Google's Gemini 4 Carbon tops the Artificial Analysis Intelligence Index v5 with a score of 59, leading 4 of 9 evaluations, and ranks second to Claude Opus 5.5 on the Coding Agent Index with 59 points. It scores 1609 Elo on AA-Briefcase (up 126 over Gemini 4 Argon) and 54 on AA-Omniscience v1.1, at $7.76 per Intelligence Index task on standard $4/$20 pricing. The model is being rolled out to selected users and is not yet publicly available, with a 50% launch discount and no confirmed end date.

    Why it matters: The piece gives a detailed benchmark breakdown of an unreleased Google model, useful for tracking how Gemini's coding and agentic performance compares with Claude Opus 5.5 and GPT-6 Astra.

  3. The Decoder88

    Google Deepmind releases Gemini 4 Carbon just two weeks after Argon with gains across the board

    Google DeepMind shipped Gemini 4 Carbon two weeks after Argon; it tops the Artificial Analysis Intelligence Index v5.0 with 59 points versus Argon's 50, and leads 15 of 16 of Google's own benchmarks, especially in long-context and security tasks. Independent evaluations show it second on Vals AI (70.36%) and the Agent Arena (+16.2%), fourth in the Code Arena's WebDev category (1,768), and still slightly behind Claude Opus 5.5 in agentic coding, while costing $7.76 per task at the same introductory price as Argon ($2/$10 per million input/output tokens). Google is rolling it out first to selected cybersecurity experts and through the Fairwind Program before wider release.

    Why it matters: Useful for readers tracking the fast cadence of frontier model releases and how independent benchmarks qualify vendor claims.

  1. The Decoder87

    GPT-6.1 Terra halves OpenAI's API prices and still gains ground on its larger models

    OpenAI released GPT-6.1 Terra, priced at $1 per million input tokens and $5 per million output tokens—half the cost of GPT-5.6 Terra and below Claude Sonnet 5.5's $2/$10. Artificial Analysis reports it gains 6 Intelligence Index points over GPT-5.6 Terra at maximum reasoning effort while costing $0.40 versus $1.40 per task, and it solves 50.3% of Terminal-Bench 4.0 tasks versus 21.5% for its predecessor. It is available now for Plus, Pro, Enterprise, and Edu users in ChatGPT Work and Codex and via the API, though OpenAI still recommends GPT-6.1 Sol and GPT-6 Astra for the most demanding coding tasks.

    Why it matters: The piece pairs concrete pricing and benchmark figures with independent Artificial Analysis evaluations, useful for anyone weighing mid-tier model costs against capability.

  1. The Decoder84

    Chinese AI model Kimi K3.1 Pro is closing in on Anthropic's Claude Opus 5.5

    Moonshot AI released Kimi K3.1 Pro, a post-trained successor to Kimi K3 that scores 55 points on the Artificial Analysis Intelligence Index, an 11-point gain that places it fourth behind Claude Opus 5.5 (58), Claude Sonnet 5.5 (56) and Muse Wonder Full (56). Moonshot says the model keeps K3's 2.8 trillion total and 104 billion active parameters with a one-million-token context window, cuts token prices by 33 percent, and shows its largest gains on agentic tasks, while cyber-related training data was stripped for the planned open-weights release. A faster K3.1 Pro Highspeed variant is also available through Kimi.ai, Kimi Work, Kimi Code and the Kimi API.

    Why it matters: It quantifies how close an open-weights Chinese model has come to top proprietary systems and documents Moonshot's deliberate removal of offensive cyber capabilities before release.

You have reached the end