AI news

Gemini Decision 1 fails to detect AI voices generated after 2024

Google's new multimodal model can distinguish older synthetic speech but performs at chance level on voices from 2025 and 2026.

A glass sphere split into clear and blurred sound wave halves.
Illustration generated for this article

Google released Gemini Decision 1, a multimodal decision model, on November 11, 2026. Recent empirical analysis reveals that while the model can separate AI-generated voices from human ones if the generator was published before 2025, it fails to do so for newer systems. For audio created in 2025 and 2026, the model’s performance drops to near-random chance.

What happened

An independent analysis tested Gemini Decision 1 against a dataset of 34 different AI voice generators. The test used 30 audio clips from each generator, comparing them against 200 human audiobook clips from the LibriTTS dataset. The goal was to determine if the model could reliably identify synthetic speech. The results showed a sharp divide based on the release date of the voice generator technology.

Generators released between 2017 and 2024 were separated from human voices with an Area Under the Curve (AUC) score of 0.67. This indicates a moderate ability to distinguish synthetic from real audio. However, for generators released in 2025 and 2026, the AUC dropped to 0.52. An AUC of 0.5 represents random guessing, meaning the model effectively cannot tell these newer AI voices apart from human recordings.

The study also noted a strong bias in the model toward labeling audio as human-made. Out of 1,020 AI-generated clips tested, only 20 were assigned a probability of 50% or higher of being AI-generated. Despite this bias, the relative ranking of probabilities still allowed for the calculation of separation performance. The drop in performance was not gradual; it occurred sharply at the start of 2025, suggesting a significant leap in the realism of newer text-to-speech models.

How it works

The evaluation used three different prompting formats to query Gemini Decision 1: a simple binary question, a criteria-based question, and a five-point rubric ranging from "Definitely authentic" to "Definitely AI-generated." The Criteria Noul format yielded the highest separation scores for older models. To ensure the results were based on audio content rather than file artifacts, every clip was processed before testing. Metadata was stripped, and the audio was re-encoded using ffmpeg into mono MP3s at 128 kbps. The clips were also band-limited to 8 kHz and loudness-normalized to remove technical variations that could serve as unintended cues.

The performance of Gemini Decision 1 was compared to human perception data from a 2026 listening study involving 1,768 people. There was a moderate positive correlation (r = +0.57) between the generators that fooled humans and those that fooled the model. However, there were notable discrepancies. For example, Griffin-Lim, a signal-processing method from 1984, was easily identified by humans (72% correct) but nearly confused the model (AUC 0.53). Conversely, Cartesia Sonic-3 was the most difficult for humans (32% correct) and also sat at chance level for the model (AUC 0.51).

Key details

  • Gemini Decision 1 was released by Google on November 11, 2026.
  • The model achieves an AUC of 0.67 for AI voices generated by tools released before 2025.
  • For AI voices from 2025 and 2026, the model’s AUC drops to 0.52, which is effectively random chance.
  • Only one 2025-2026 generator, Inworld TTS-2, showed a confidence interval slightly above chance (0.51-0.75).
  • The model has a strong bias toward labeling audio as human, identifying only 20 out of 1,020 AI clips as potentially synthetic.
  • Audio clips were stripped of metadata and normalized to prevent technical artifacts from influencing the detection.

Why it matters

For developers building content moderation or authentication systems, this finding signals that relying on current-generation multimodal models for deepfake detection is becoming increasingly unreliable. The sharp decline in detection capability for post-2024 models suggests that synthetic voice technology has crossed a threshold where acoustic artifacts are no longer sufficient for identification. Systems that depend on detecting subtle digital fingerprints in audio waveforms may need to be re-architected or supplemented with other verification methods, such as watermarking or cryptographic provenance.

The alignment between human and machine failure modes also highlights a growing challenge in user trust. If neither humans nor advanced AI models can reliably distinguish recent synthetic voices from real ones, the potential for fraud and misinformation increases significantly. Engineers must assume that any audio input could be synthetic and design workflows that do not rely solely on perceptual authenticity. This shift requires a move toward zero-trust architectures for media inputs in sensitive applications.

What you can do

  • Do not rely solely on Gemini Decision 1 or similar models to detect AI voices generated in 2025 or later.
  • Implement audio watermarking or cryptographic signing for all legitimate voice content to establish provenance.
  • Use multi-factor authentication for voice-based systems, combining audio analysis with behavioral biometrics or device checks.
  • Monitor updates from voice generator developers, as detection capabilities may lag behind generation quality.
  • Test your detection pipelines against the latest open-source TTS models to benchmark current vulnerability levels.
  • Educate users about the limitations of AI detection tools to manage expectations regarding security and authenticity.
  • #ai detection
  • #deepfakes
  • #gemini
  • #audio synthesis
  • #machine learning

Tools from the Bytechap store

$89

DocBento

Self-hosted document management that reads every scan and answers with page citations.

Live demo

Keep reading

All stories