Analysis of 1,866 GitHub projects reveals gaps in AI agent instructions

One in three popular GitHub projects now includes specific instruction files for AI coding agents, with AGENTS.md emerging as the standard format. A recent analysis of 1,866 active repositories reveals that while adoption is widespread, particularly in TypeScript projects, the quality and enforceability of these instructions vary significantly.

What happened

Researchers analyzed the most-starred repositories on GitHub to understand how teams are guiding AI assistants. They examined 1,866 projects that had received commits in the current year, looking for files like AGENTS.md, CLAUDE.md, and others designed to steer agent behavior. The study found that 627 of these projects, or 34%, ship at least one such instruction file.

The data shows a clear preference for AGENTS.md, which appears in 78% of the projects that have any agent instructions. CLAUDE.md is also common, present in 52% of those projects, but it usually coexists with AGENTS.md rather than replacing it. Adoption rates differ sharply by programming language. TypeScript leads with 73% of projects including agent files, followed by Go at 58% and Rust at 52%. In contrast, Perl projects rarely include them, with only 4% adoption.

The study also highlighted issues with file length and redundancy. Half of the main instruction files exceed 100 lines, with a median length of 106 lines. About 30 projects maintain two separate rulebooks, such as an AGENTS.md and a CLAUDE.md with different content, creating potential conflicts for agents using different tools.

How it works

The analysis combined automated scripting with human-like judgment from an AI reviewer. A script first collected data on file existence, length, and headings. Then, a separate AI reviewer classified random samples of rules to determine if they were genuine instructions and whether they could be enforced by code.

The researchers found that three in five rules could be checked by a script, linter, or test, at least in part. However, fewer than one in twenty rules explicitly name the check that enforces them. Most rules rely on the agent simply remembering the instruction, which is less reliable than automated enforcement. For example, a rule might state "Do not use empty destructuring patterns," which a linter could easily catch, but the file does not specify which linter rule applies.

The study also looked at where these files live. While many are at the repository root, 30% of projects keep instruction files in subfolders. This allows rules to be specific to certain parts of the codebase, preventing agents from loading irrelevant context when working in other areas.

Key details

• Adoption rate: 34% of popular active projects have agent instruction files, rising to 73% for TypeScript.
• File length: The median main file is 106 lines, with 52% exceeding 100 lines.
• Enforceability: 61% of rules could be partially or fully checked by a script, but only 3.7% name the specific check.
• Redundancy: About 30 projects maintain separate, non-identical AGENTS.md and CLAUDE.md files.
• Staleness: Only 2% of files reference paths or scripts that no longer exist.
• Content focus: Code style, agent behavior, and project-specific correctness are the most common topics, while security and performance are rare.

Why it matters

For engineering teams, this data highlights a gap between writing instructions and ensuring they are followed. Long instruction files increase token costs and can dilute the agent's focus. The ETH Zurich study cited in the article notes that while developer-written files help marginally, they can add up to 19% to agent costs. Keeping files short and pointing to external documentation is more efficient.

More importantly, relying on an agent's memory for critical rules is risky. When a rule names its enforcing mechanism, such as a specific linter or CI check, it creates a safety net. If the agent forgets, the tool catches the error. Without this link, teams depend on the agent's consistency, which degrades as context windows fill up. The prevalence of unenforced rules suggests many teams are treating these files as wish lists rather than operational constraints.

What you can do

• Audit file length: Keep your main AGENTS.md under 100 lines by moving detailed explanations to linked documentation.
• Name your checks: For every rule, specify the linter, test, or script that enforces it, such as "Enforced by Sorbet/ForbidTMust."
• Verify paths: Regularly check that all commands and file paths referenced in your instructions still exist.
• Consolidate files: If you use multiple instruction files, ensure secondary files point to the primary one to avoid conflicting rules.
• Prioritize enforceable rules: Focus on rules that can be automated, and remove those that rely solely on agent memory unless they are critical.
• Use subfolder rules: Place specific instructions in subdirectories to keep context relevant to the code being edited.