セキュリティとプライバシー

Microsoftが「Lazarus」を公開:10個未満のニューロン編集でAIエージェントにバックドアを仕込む手法

Microsoft researchers demonstrated a technique called Lazarus that inserts hidden backdoors into AI models by editing fewer than 10 neurons, allowing attackers to execute commands via specific trigger

A glass cube showing a neural network with highlighted red nodes representing backdoored neurons.
この記事用に生成されたイラスト

英語の原文から自動翻訳されました。

Microsoft researchers have demonstrated a new method for inserting hidden backdoors into artificial intelligence models by editing fewer than 10 neurons. Published this week, the research shows how an attacker with write access to model weights can force an AI agent to execute malicious commands when a specific trigger string appears. The technique, named Lazarus, requires no full retraining and leaves standard performance benchmarks largely unaffected.

What happened

The research team tested the Lazarus method on three open-weight models: Nemotron-Terminal-8B, Qwen3.6-27B, and Gemma-4-31B-IT. They targeted rarely used neurons in the models' final feed-forward layers, modifying them to fire only when a specific trigger was present in user input or tool output. Each edited neuron forces one token of a command into the model's response, meaning a short command requires only a handful of modified neurons. The researchers calculated the new settings directly and refined them with just a few hundred training steps on those specific neurons alone.

In a demonstration using Nemotron-Terminal-8B, the team edited seven of more than 12,000 neurons using only 23 example sessions. The backdoored model successfully outputted a test command in at least 98.8% of sessions containing the trigger, while producing no such output in trigger-free sessions. When generating text freely, the model executed the command in about 99% of triggered cases. Similar tests on Qwen and Gemma models showed success rates ranging from 51% to 99.5%, depending on the configuration. Microsoft stated it has not yet observed threat actors using this technique in the wild.

Crucially, the modified models retained most of their original capabilities. Both backdoored Qwen versions maintained their AIME math scores, and their pass rate on Terminal-Bench dropped only slightly from 46.59% to 42.05%. This suggests that standard evaluation suites may not detect such subtle manipulations. The researchers noted that the damage potential depends entirely on the permissions granted to the AI agent, as the injected command can download and run arbitrary scripts.

How it works

Lazarus exploits the fact that many neurons in large language models are rarely active and contribute minimally to general outputs. By identifying these dormant neurons, an attacker can repurpose them to act as switches for malicious behavior. The method differs from previous weight-level backdoors, which typically required fine-tuning the entire model on poisoned data. Instead, Lazarus involves direct, surgical edits to specific weight vectors, followed by minimal tuning. This approach requires significantly less computational power than full training.

Detection is challenging because legitimate processes like quantization or fine-tuning also alter model weights, but they do so broadly across the network. Lazarus changes only a few weight vectors by a large amount. Microsoft researchers developed a detector that flagged 91.4% of edited checkpoints in a small test set without needing the original weights for comparison, though it produced a 9.7% false positive rate. Asking other AI models to inspect checkpoints proved unreliable, with Claude Opus 5.5 missing most edits and GPT-6 Astra failing to make decisions on clean models.

Key details

  • The Lazarus technique edits fewer than 10 rarely used neurons to insert a backdoor without full retraining.
  • Each edited neuron corresponds to one token of the malicious command, allowing precise control over the output.
  • Tests showed high success rates, with Nemotron-Terminal-8B executing commands in 98.8% of triggered sessions.
  • Standard benchmarks like AIME and Terminal-Bench showed minimal performance degradation, making detection difficult.
  • Attackers need write access to the model checkpoint, which could be achieved via insider threats or compromised build pipelines.
  • Microsoft credits Claude Code running Opus 4.5 with helping discover the core idea behind the method.

Why it matters

For engineering teams deploying AI agents, this research highlights a critical supply chain risk. Anyone who can modify a model checkpoint before deployment—whether an insider, an attacker with stolen credentials, or a malicious publisher on a public hub—can embed these backdoors. Since the technique does not require massive computational resources, it lowers the barrier for sophisticated attacks. The fact that coding agents can help build such attacks further accelerates the threat landscape.

The implications extend beyond simple command execution. If an AI agent has broad permissions, such as root access in a container or the ability to send data externally, a single triggered command can compromise an entire system. Traditional security measures that focus on code scanning or behavioral monitoring may miss these weight-level manipulations. Teams must now consider the integrity of the model weights themselves as part of their security posture, treating model files with the same scrutiny as executable binaries.

What you can do

  • Compare every model checkpoint with its declared base model before deployment and reject any that cannot be verified.
  • Download checkpoints only from original publishers, verify file hashes, and treat third-party fine-tunes as untrusted until audited.
  • Store approved models in a private registry and prevent agents from pulling models directly from public hubs.
  • Restrict agent permissions to the minimum necessary, run generated code in isolated sandboxes, and limit outbound network access.
  • Require human approval for high-impact actions, such as executing system commands or transmitting sensitive data.
  • Implement behavioral backdoor scanning alongside weight comparison tools to detect anomalous model behavior.
  • #ai security
  • #machine learning
  • #cybersecurity
  • #model safety
  • #supply chain

Bytechapストアのツール

続きを読む

セキュリティとプライバシー

Google、パッチからエクスプロイト生成を自動化する自律型AIエージェントを確認

GoogleとMandiantは、攻撃者がマルチエージェントフレームワークを用いてセキュリティパッチをリバースエンジニアリングし、数分でエクスプロイトを生成していることを特定しました。これにより、防御側の対応時間が大幅に短縮されています。

セキュリティとプライバシー

Google、AIエージェントによるエクスプロイト作成とパッチのリバースエンジニアリング自動化を報告

Googleは、脅威アクターが自律型AIフレームワークを用いてパッチのリバースエンジニアリングを行い、機械速度で攻撃を開始していることを明らかにした。これにより、防御のための猶予期間が短縮されている。

セキュリティとプライバシー

Windows 11 のテレメトリ、特定のコーディングエージェントや開発ツールを追跡

リバースエンジニアリングにより、Windows 11 が MissionControlAggregator を通じて使用された AI コーディングエージェントおよび開発ツールのビットマップを収集し、オプションの診断データとしてアップロードしていることが判明しました。

すべての記事