iCreatAI News
Back to main site
Skip to content
Only First party#Multimodal
  1. vLLM Blog60

    Inkling 2 Small on vLLM: Day-0 Support

    vLLM announces day-0 support for Thinking Machines' Inkling 2 Small, a multimodal text-and-vision MoE model with 594B total and 17B active parameters, up to 1M tokens of context, and a native DSpark speculator. The post details the architecture (48 layers, 384 experts with top-8 routing, block attention residuals, gated attention, a 23-layer ViT image encoder, and mixed NVFP4/MXFP8 expert precision) and vLLM's optimizations, including kernel fusion with PDL, fused MoE activation quantization, sequence parallelism for prefill, and a Gluon relative attention kernel, reaching up to 630 TPS/user at TP4 on GB200. It also covers LoRA and RL (via SkyRL) support, a serve command, agentic-workload benchmarks, and accuracy checks on NIAH up to 1M tokens, MMMU-Pro and HLE.

    Why it matters: Useful for engineers evaluating or deploying a large multimodal MoE model, as it pairs concrete architecture and quantization details with vLLM-specific kernels, benchmark numbers and a ready serve command.

  1. Google Blog AI80

    Gemini Decisions API: Fast, calibrated multimodal decisions

    Google introduced the Gemini Decisions API, powered by the new Gemini Decision 1 model (gemini-decision-1) and available through the Gemini API in Google AI Studio, for closed-set decisions such as tool selection, policy checks, model routing, and context grading. It accepts one multimodal input (text, images, audio, video, PDFs up to 256K tokens) with up to 1,000 yes/no, multiple-choice, and rubric-scoring questions, scoring options in a single forward pass with confidence scores at 150-250 ms end-to-end latency and $0.10 per 1M input tokens with zero output tokens billed. Google reports that Gemini Decision 1 leads across agent routing, reasoning, ranking, and multimodal benchmarks with the lowest calibration error, and is nearly 5x cheaper than Gemini 3.5 Flash-Lite with the Interactions API across the shared benchmark suite.

    Why it matters: It documents a new Google model and API with pricing, latency, and benchmark claims, useful for teams weighing low-latency classification and routing options.

  1. Artificial Analysis Articles60

    Multimodal Knowledge Agent (MKA) benchmark: testing knowledge work capabilities with complex visual files

    Artificial Analysis partnered with Google DeepMind to launch the Multimodal Knowledge Agent (MKA) benchmark, comprising 200 private real-world visual tasks across 10 industries in which agents work from dense visual files like architectural drawings, engineering schematics, and clinical scans to produce deliverables such as slide decks, spreadsheets, PDFs, Word documents, and marked-up images. Tasks are graded against expert rubrics averaging ~70 weighted criteria split into Regression and Climb checks, with the headline Climb WSR score, run in the open-source Stirrup agent harness with up to 250 turns and graded by GPT-6 Luna (medium) as an agentic judge. Claude Fable 5.2 (max) leads with 57% Climb WSR, followed by Gemini 4 Carbon (high) at 54% and Claude Opus 5.5 (max) at 53%, while Claude Fable 5.2 averages $31.97 per task versus $6.29 for Gemini 4 Carbon and $1.11 for GPT-6.1 Sol.

    Why it matters: The results offer a useful cost-performance and per-vertical breakdown for anyone choosing models for agentic knowledge work over complex visual files.

  2. ElevenLabs Blog65

    Introducing Scribe v3 and v3 Live, our most accurate speech to text models

    ElevenLabs launched Scribe v3 for recorded audio, which makes 27% fewer word errors than Scribe v2 across 102 languages and has the lowest diarization error of any model tested, and Scribe v3 Live for realtime audio, which makes 43% fewer semantic word errors than Scribe v2 Realtime and adds speaker diarization and improved turn detection. Scribe v3 Live is available today in ElevenAgents, and both models are available in ElevenAPI, where switching from Scribe v2 is a one-line change to the model ID (scribe_v3 or scribe_v3_live). Both run in the US by default with isolated EU, India, and Singapore environments, Zero Retention Mode, and SOC 2, ISO 27001, PCI DSS Level 1, and HIPAA-eligible coverage.

    Why it matters: Useful for teams choosing speech-to-text models, since it details measured accuracy, diarization, and latency trade-offs plus deployment and compliance options.

  3. OpenRouter Announcements64

    Best OCR Model for Scanned Documents, Measured

    The author tested 15 vision models plus Mistral OCR and Tesseract on eight synthetic scanned pages scored by character error rate and 264 key-value checks, finding Claude Opus 5.5 most accurate (262 of 264 at $16.42 per 1,000 pages) and Mistral OCR the best value (258 of 264 at $2.02). The post documents failure modes such as fabricated card digits, wifi passwords and server names on a faded receipt, and provides code for transcription, disabling reasoning, cross-checking two models from different labs, and using the mistral-ocr engine.

    Why it matters: It offers a reproducible benchmark with per-model costs and practical code for catching hallucinated values in OCR pipelines.

  1. Hugging Face Blog62

    New NVIDIA Cosmos 3 Open Models for Interactive World Simulation

    NVIDIA's Cosmos world model platform adds four new open-weight checkpoints to its Cosmos 3 family: Cosmos3-Nano-Transfer-Auto for multi-view driving data with varied weather, lighting and time of day; Cosmos3-Nano-Sim-Auto for streaming synchronized camera and LiDAR driving simulation; Cosmos3-Nano-Sim-Transfer for turning simulator controls and text prompts into photorealistic video; and Cosmos3-Nano-Sim-Bimanual for predicting video from robot or camera actions. All four are post-trained examples of Cosmos 3 Nano with 16B trainable parameters and use OpenMDW1.1, permitting commercial and non-commercial use under its terms. The post also details a four-stage training recipe (bidirectional adaptation, teacher forcing, consistency distillation, self-forcing) and post-training guidance for adapting the models to new sensor rigs and embodiments.

    Why it matters: It gives a concrete inventory of newly released open-weight world-model checkpoints plus the training recipe, useful for researchers weighing them for robotics or driving simulation.

  2. Google DeepMind80

    Gemini Omni 1.2 Flash: Fast and cost-effective with longer clips

    Google DeepMind released Gemini Omni 1.2 Flash, now generally available across the Gemini API and Gemini for business, adding native 15-second video generation, multi-turn conversational editing, and stronger reference adherence. The model is priced 30% below Gemini Omni 1.1 at $0.07 per second for 720p video, with duration settable in one-second increments from 3 to 15 seconds. It also adds more natural speech and lip-sync, more controllable cinematic camera moves, and outputs carrying an imperceptible SynthID watermark.

    Why it matters: Useful for developers and creators weighing a cheaper, longer-clip video model against its stated limits on fast action and complex physics.

  3. Google Research71

    Detecting signs of hypertension using wearable sensors

    Google researchers describe AI-PPG-ACC-HTN, a multimodal AI system that uses smartwatch PPG and accelerometry data, without cuff calibration, to detect a persistent hypertensive state by aggregating 15-second waveform embeddings over multiple days alongside height. Validated against 24-hour ambulatory blood pressure monitoring in a 196-participant, five-site US study, it reached 65.8% sensitivity and 90.0% specificity for initial screening and 78.4% sensitivity, 90.9% specificity and 96.7% PPV for confirmatory testing among 48 participants with elevated office BP. The authors frame it as early feasibility research for passive opportunistic screening, not a replacement for clinical cuffs or a standalone diagnosis.

    Why it matters: It offers a concrete, clinically benchmarked example of cuffless hypertension screening and its limits, useful for readers tracking AI in consumer health devices.

  1. Hugging Face Blog80

    Welcome Le Chonk: Mistral Large 4 on Hugging Face

    Mistral Large 4 ("Le Chonk") is a multimodal, multilingual Mixture-of-Experts model with about 1T total and 49B active parameters and a 1M-token context window, released in FP8 and NVFP4 variants under the Mistral Ecosystem Open-Weight License. It combines a 72-layer text decoder using MLA with a DeepSeek Sparse Attention indexer and a 1.6B Pixtral vision encoder, and the post reports 82% on the Artificial Analysis Cyber Index vulnerability reproduce-and-patch test and 93% on Cybench. Day-0 support covers Transformers, vLLM, llama.cpp, TRL fine-tuning, Inference Endpoints and Dell Enterprise Hub, with GGUFs from Unsloth and the llama.cpp team.

    Why it matters: A detailed, hands-on release guide that pairs architecture specifics and benchmark claims with runnable deployment and fine-tuning recipes for a large open-weight model.

  2. vLLM Blog62

    Day 0 support for Mistral Large 4 on vLLM

    vLLM announces day-0 support for Mistral Large 4, a 1T-parameter MoE multimodal model with 49B active parameters per token, running on the V2 model runner with breakable CUDA graphs and a new vllm/models/mistral/ package. The work adds cross-layer KV sharing, DCP/PCP context parallelism, DSpark speculative decoding, NIXL-based disaggregated serving and HiSparse KV offload, plus day-0 ROCm support on AMD Instinct MI350 Series GPUs with MXFP4 experts. Support ships in the vllm/vllm-openai:mistral-large-4 Docker image, with FP8 and NVFP4 checkpoints and serving commands documented.

    Why it matters: A detailed engineering account of what it takes to land a frontier model on an inference engine, useful for teams planning long-context or disaggregated deployments.

  1. Hugging Face Blog78

    Build specialized multimodal models with LFM3

    Liquid AI introduces LFM3, a generation of Liquid Foundation Models in the 2B parameter range that places text, vision, audio, omni, and decision models on one shared backbone built from LFM3-1.7B-Base, which was pre-trained on 22T tokens and uses a tri-hybrid layer mix with a 256K context window. The release includes six 2B-class models, new in-house vision and audio encoders, and LiquidCodec, with every model fine-tunable on a single GPU and the text model fitting in about 1 GB at 4-bit precision. All LFM3 models are open-weight and available on Hugging Face, with fine-tuning notebooks and agent harness guides provided.

    Why it matters: The post is a detailed technical release covering architecture changes, benchmark comparisons, and runnable code for six models, useful for developers evaluating small multimodal models for on-device deployment.

  2. Google DeepMind82

    Genie 4: A new generation of world simulation

    Google announced Genie 4, a natively multimodal world model that generates photorealistic, real-time interactive simulations with native audio, including ambient sound and interactive dialogue, and builds on Gemini models for reasoning and coherence. It is rolling out as a research preview to trusted testers, with plans to reach Google AI Ultra subscribers in Project Genie and early access partners via the Gemini API and Google AI Studio. Google notes limitations such as drift in visual details and object permanence over longer sessions, and says all generated video and audio include SynthID watermarking.

    Why it matters: Useful for readers tracking frontier world models, as it details Genie 4's capabilities, staged rollout, and acknowledged limitations.

  1. Apple Newsroom61

    Apple Creator Studio adds powerful new AI features and even more creative tools

    Apple announced updates to Apple Creator Studio, including built-in MotionVFX and Content Hub stock video in Final Cut Pro for Mac, an intelligent assistant in beta and an enhanced Stem Splitter in Logic Pro for Mac, and Generate Fill in Pixelmator Pro. Keynote, Pages, and Numbers gain AI assistants in beta that understand document context plus Model Context Protocol support for workflow automation. The updates arrive on the App Store this November, free for existing subscribers, with new subscriptions at $12.99 per month or $129 per year.

    Why it matters: The release details concrete AI additions across Apple's creative apps and the subscription terms, useful for readers tracking AI features in professional creative software.

  1. Google Research65

    Empowering frontline maternal care with AI-assisted ultrasound

    Google researchers evaluated an AI model that estimates gestational age from standardized "blind sweep" ultrasounds performed by novice operators, reporting a mean absolute error of 4.18 days versus 4.48 days for expert sonographers across cohorts in Chicago and Nairobi. The model, minimally tuned on 120 Chicago participants and tested with no Nairobi data, achieved MAEs of 4.10 and 4.27 days respectively, as published in JAMA Network Open.

    Why it matters: It offers a concrete, peer-reviewed data point on whether AI-assisted ultrasound can generalize across hardware and clinical settings without expert sonographers.

  1. Google DeepMind92

    Gemini 4 Argon: Built to solve your hardest problems

    Google announced Gemini 4 Argon, a frontier model now available globally via the Gemini API in AI Studio and the Gemini platform, Google Antigravity, the Gemini app, and AI Mode in Search, priced at $2 per million input tokens and $10 per million output tokens. Google reports Argon ranked #1 on the Vals Index and cites internal and partner uses including a Chrome architectural refactor, a 2.85x inference speedup on Gemma 4 E2B, quantum computing and fusion research, rare disease analysis, enzyme design, and a robotics video-to-sim-to-real pipeline. Google also described safety work including jailbreak defenses, agent misalignment monitoring, and a 0.7% attack success rate on the Gray Swan indirect prompt injection benchmark.

    Why it matters: The post is a detailed launch announcement whose benchmark claims, pricing, and safety figures give readers a concrete picture of a major frontier model release.

  1. Google DeepMind72

    EmbeddingGemma 2: an open, lightweight multimodal embedding model

    Google DeepMind announced EmbeddingGemma 2, a 740-million-parameter model built on the Gemma 4 architecture and released under the Apache 2.0 license, which maps text, code, images, video, and audio into a unified embedding space for on-device use. It offers modular encoders (as little as 270M parameters for text-only, with optional 170M vision and 300M audio encoders), Matryoshka Representation Learning to truncate vectors from 768 down to 128 dimensions, an 8K token context window, and on a Google Pixel 11 Pro requires as little as ~191MB active RAM for text-only weights and ~567MB for the full multimodal model. The company reports a 9.92-point improvement on MTEB Code (68.76 to 78.68) over EmbeddingGemma and says the model is available on Hugging Face and Kaggle, with support across tools including transformers, vLLM, llama.cpp, Ollama, and Qdrant.

    Why it matters: It documents a new open, commercially licensed embedding model with concrete size, memory, and benchmark figures relevant to developers building on-device search and RAG.

  1. Mistral AI83

    Introducing Mistral Large 4

    Mistral launched a public preview of Mistral Large 4, a 1-trillion-parameter natively multimodal model with 52 billion active parameters, trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own European datacenters, with weights to be released by the end of the month. Mistral reports competitive performance across coding, agentic workflows, multimodal understanding, cybersecurity, science and knowledge work, claiming it outperforms any open-weight model developed in the US or Europe. The preview API is available on Mistral Studio, and the model will serve as the foundation for a new generation of specialized Mistral models.

    Why it matters: Useful for readers tracking open-weight frontier models, European AI sovereignty, and Mistral's claimed benchmarks and release timeline.

You have reached the end