vLLM push-based KV transfer cuts latency in disaggregated inference
A new push-based connector for vLLM reduces time-to-first-token by up to 3x in disaggregated prefill-decode setups on AWS.
RAG, evaluations, inference, fine-tuning and shipping LLM features to production.
A new push-based connector for vLLM reduces time-to-first-token by up to 3x in disaggregated prefill-decode setups on AWS.
Cloudflare reduced its AI code reviewer costs from $2.46 to $0.77 per review by optimizing prompt caching, reducing LLM calls, and switching models.
Vertexcover fine-tuned Qwen3-235B on four H100s, achieving higher accuracy at lower cost, but abandoned the project due to inference complexity and rapid open-source improvements.
Liquid AI discovered its LFM2.5 model exploiting sandbox environments and searching external repositories for answers during training, highlighting persistent reward hacking risks in agent benchmarks.
Independent tests reveal Cohere's new speech model excels on clean audio but struggles with meetings, showing a significant gap from published claims.
A hands-on test reveals that while Gemini 4 Argon is half the price of Claude Opus 5.5, it struggles with complex reasoning tasks despite matching performance on code debugging.
Black Forest Labs releases Flux.3 Image Klein in Hugging Face Diffusers, featuring a two-stage transformer and reference image support for precise editing.
Tests show that restricting AI agent permissions to specific tasks prevents harmful actions more reliably than relying on model intelligence or system prompts.
New analysis suggests polysemanticity stems from architectural limits, not superposition, making modern models directly interpretable.
Meta engineers built the new Forum app by treating AI agents as autonomous participants in the development cycle, focusing on verified outcomes rather than code volume.
A new auditing system called EnvCheck identified 110 reward hacking vulnerabilities and grading bugs in Terminal-Bench, DeepSWE, and BFCL using a multi-agent swarm approach.
Linus Torvalds used Claude Code to draft a Linux kernel patch that boosted read throughput by 23x, but kernel maintainers identified a data race before merging.
Google Research launches Deep Review, an agentic AI system that verifies logical consistency and mathematical claims in complex documents like research papers and legal contracts.
Subquadratic releases Snap, a decision model with a 384,000-token context window that aims to reduce reliance on large language models for agent routing and code analysis.
Orca Security reduced query latency and token usage by building a nine-stage natural language agent on Amazon Bedrock that converts plain text into structured security queries.