--- title: "Reviewing AI-Written Code: A 15-Point Checklist for People Who Read the Diff" url: https://www.thinnd.site/en/engineering/reviewing-ai-written-code/ language: en author: "Nhu Duy Thin (Thinnd)" published: 2026-10-29 updated: 2026-10-29 section: "Engineering" tags: ["code review", "AI", "verification", "security", "checklist"] summary: "When AI writes code, the bottleneck moves to review, so reviewers need a fixed checklist instead of reading by feel. The 15 points fall into four groups: understand the task, correct and safe, tests that matter, and easy to maintain. The points most often missed are whether the library is real, whether inputs are trusted too much, and whether the tests fail when the code is wrong." license: "Cite with the author name and the canonical URL above." --- # Reviewing AI-Written Code: A 15-Point Checklist for People Who Read the Diff > For people who review code, including beginners. Every technical term gets a short explanation. > > Figures come from public reports and studies listed at the end. Many were published by technology companies, so read them with care. Examples marked "illustration" are rebuilt to be easy to picture and do not belong to any specific incident. --- ## Quick summary - When AI writes code fast, **the bottleneck moves to review**. Teams using AI heavily merge more changes, but each change takes longer to review. - AI-written code **looks tidy and runs**, which makes reviewers complacent. Use a fixed checklist instead of reading by feel. - This list has **15 points in four groups**: understand the task (3), correct and safe (5), tests that matter (3), easy to maintain (4). - The three most often missed: **is the library real**, **are inputs trusted too much**, **do the tests fail when the code is wrong**. - AI can do a first pass, but **a person stays accountable**. Ask AI to "find mistakes", not "is this right". - A good PR description states **what the AI wrote, what the human checked, and how**. --- ## Hard words, plain meaning | Term | Plain meaning | |---|---| | **Diff** | A comparison of two versions of code: green lines are added, red lines are removed. Reviewers read the diff. | | **PR (pull request)** | A proposal to put a change into the project so others can look before it is accepted. | | **XSS, injection** | Attacks that sneak malicious code into input fields. Good code treats every input as untrusted. | | **Dependency** | An outside library that the code calls. | | **Always-green test** | A test that reports a pass even when the code is wrong, because it only checks itself. | | **Mutation testing** | Deliberately breaking the code to see whether the tests complain. | --- ## 1. Why review harder Three kinds of evidence, all to be read with care. - **The bottleneck moved.** The "AI Productivity Paradox" report by Faros AI (July 2025, over 10,000 developers in 1,255 teams) found teams using AI heavily merged about 98% more changes, but review time per change rose about 91% and change size grew about 154%. This is data from a vendor that sells measurement tools, it is a correlation not necessarily causation, and I have only read summaries. - **Security flaws.** Veracode's 2025 GenAI Code Security Report (80 tasks, over 100 models) found AI-generated code had a vulnerability in about 45% of tasks. For cross-site scripting and unsafe logging, models failed to protect in about 86% and 88% of cases. Earlier, Pearce et al. (2021) found about 40% of 1,689 programs written by Copilot had a vulnerability. These figures were measured on test tasks, not on code running in production. - **Overconfidence.** Perry et al. (Stanford, 2022) found people using an AI assistant wrote less secure code on four of five tasks yet believed more strongly that it was secure. Small group, older model, so read it as a sign. This links to [why you should not trust AI blindly](https://www.thinnd.site/en/engineering/why-not-to-trust-ai-blindly/) and [IT teamwork in the AI era](https://www.thinnd.site/en/experiences/team-work-in-the-ai-era/): results look complete so flaws are hard to see, and the bottleneck now sits at reading. Google's public code review guide (eng-practices) has three base questions worth remembering: does the design fit the system, does the behavior match the intent and serve users, and could the code be simpler. The list below makes those three concrete for AI-written code. --- ## 2. The 15-point checklist Severity: **High** can cause an incident or data loss, **Medium** causes technical debt or small bugs. ### Group A. Understand the task | # | Question | Typical warning sign in AI code | How to check | Severity | |---|---|---|---|---| | 1 | Does the code do what was asked, no more, no less? | AI adds things nobody asked for, or misses half the request because it read it differently | Read the task description first, say the expectation in your own words, then read the diff | High | | 2 | Is the change scope tight? | The diff touches unrelated files, "while at it" renames or reformatting | List changed files and ask of each "why is it here" | Medium | | 3 | Are assumptions stated? | AI quietly picks one reading of an ambiguous request and runs with it | Ask: "What did you assume?" and compare with the task description | High | ### Group B. Correct and safe | # | Question | Typical warning sign in AI code | How to check | Severity | |---|---|---|---|---| | 4 | Are the libraries, functions and parameters real and the right version? | Calls a function that does not exist or was removed, or installs a plausible but invented package | Open the package's official page, check maintainers, creation date, downloads. Run it to see import errors | High | | 5 | Is every outside input validated? | Forgets protection against XSS, injection, odd file paths | Trace each input from where it arrives to where it is used. Try it with a bad string | High | | 6 | Are any secrets or sensitive data exposed? | Keys or tokens pasted into code, logs that print personal data | Search for key-like strings, read the logging lines | High | | 7 | Are edge cases and errors handled? | Written only for the happy path: empty, null, time zones, huge numbers, two requests at once | List three edge cases yourself and find where each is handled | High | | 8 | Do destructive actions have guardrails? | Deletes or overwrites without confirmation, statements missing a condition | Search for delete, overwrite, bulk update. Ask "how do we undo this" | High | ### Group C. Tests that matter | # | Question | Typical warning sign in AI code | How to check | Severity | |---|---|---|---|---| | 9 | Does the test check real behavior? | A test whose mock returns exactly the value the test checks, so it is always green | Read the tests before the code. Ask: "What does this test check besides itself?" | High | | 10 | Does the test fail when the code is wrong? | The test passes even when you deliberately break the code | Break one important line and run the tests (manual mutation testing) | High | | 11 | Does it run on a clean machine? | Works on the author's machine because of something not recorded in the project | Fetch a fresh copy, follow the instructions or let CI run it | Medium | ### Group D. Easy to maintain | # | Question | Typical warning sign in AI code | How to check | Severity | |---|---|---|---|---| | 12 | Does it duplicate or ignore something already in the project? | Rewrites an existing function, pastes near-identical blocks | Search the project for similar function names and blocks before accepting | Medium | | 13 | Is the code simple and readable? | Needless layers of abstraction, deep nesting for a simple job | Try explaining the code to someone in one minute | Medium | | 14 | Does it follow the project's conventions? | Naming, folder structure and error handling differ from the rest | Compare with the [handbook for AI](https://www.thinnd.site/en/engineering/writing-agents-md/) and neighboring files | Medium | | 15 | Is the PR small, single-purpose and clearly described? | One PR mixing several jobs, a vague description | Ask for a split. Use the description template in section 4 | Medium | Two notes on evidence: - **Point 4** links to research on invented package names (Virginia Tech and others, 2025): about one in five package suggestions did not exist, and about 43% of invented names repeated in every run, so attackers can register those names in advance (slopsquatting). - **Point 12** links to GitClear's analysis (2025, over 211 million changed lines): summaries report a sharp rise in duplicated code blocks and a fall in the share of refactored lines in 2024. This is data from a vendor of analytics tools, it is a correlation, and I have only read summaries. --- ## 3. The order for reading a PR in 15 minutes 1. **Read the description and the original task (2 minutes).** If the description is unclear, stop and ask. Points 1, 2, 3. 2. **Read the tests before the code (3 minutes).** Tests show how the author understood the job. Points 9, 10. 3. **Scan high-risk spots in the diff (6 minutes).** Inputs, secrets, deletes and overwrites, new libraries. Points 4, 5, 6, 7, 8. 4. **Run it (3 minutes).** Run the tests, run the feature, try one bad case. Points 10, 11. 5. **Read the rest for maintenance (1 minute or more).** Duplication, complexity, conventions. Points 12 to 15. If step 3 finds a problem, do not keep reading just to "collect every bug". Send it back early so the author can fix it, because every round costs time. --- ## 4. A PR description template for AI-assisted code A good description tells the reviewer where to look. ```text Task: add a cancel-order button on the order page Done means: tests, refunds handled, a screenshot AI wrote: the button UI, the cancel API call, tests for the normal case Human checked: read the whole diff, ran the tests, tried cancelling a delivered order Human changed: removed an unneeded library, added handling for API errors How to re-check: run npm test, open the order page, cancel a test order Not sure about: refunds for installment orders, needs someone who knows the business ``` When the reviewer sees the "Not sure about" line, they know where to spend effort. The description also forces the author to reread the diff before sending. --- ## 5. What about using AI to review AI can help with a **first pass**: finding small errors, reminding of forgotten points, skimming long parts. But do not treat it as the final reviewer, for three reasons: - **The tendency to please.** Anthropic's research on sycophancy (Sharma et al., 2024) found many AI assistants often admit a mistake that does not exist when challenged and tend to agree with the asker. An AI asked "is this code right" is likely to say "yes". - **The same source of errors.** A reviewing AI may make exactly the kind of mistakes the writing AI made. - **No accountability.** The person who signs is the one responsible. A lower-risk use is a specific request: ```text Read this diff like a strict reviewer. Find three places that could be wrong and say why. List every new library, function and parameter, and say how sure you are. Point out any input that is not validated. If you find no errors, say what you looked at. ``` Then a person rereads the places the AI pointed to, and also the places it did not mention. --- ## 6. Three illustrations Rebuilt from common mistakes, not tied to any specific incident. **A library that "sounds very real".** The AI adds `pip install fastparse-utils` to read a file format. Everything looks reasonable, but the reviewer checks the package page (point 4): the package was registered last week and has one maintainer. A popular alternative was already in the project. **An always-green test.** The AI writes a test for a fee function, but the test uses a mock that returns exactly the number the test expects (point 9). The reviewer breaks the formula on one line (point 10) and the test still passes. They ask for the test to be rewritten with real data. **A diff that is too big.** One PR mixes renaming, refactoring and a new feature, over a thousand lines. The reviewer sends it back and asks for three (point 15). Three small PRs are reviewed faster than one large PR waiting in line. --- ## 7. A balanced view This list does not mean AI-written code is worse than human-written code. People also forget input checks and write weak tests. The difference is **speed and appearance**: AI produces more code faster, and tidy-looking code hides its flaws. So the list is useful for human code too. Security figures were measured on test tasks by a security vendor and may differ for newer models. The Faros and GitClear figures are correlations. What is more certain is the structure: **the cost of writing a draft is falling, the cost of verifying is not falling with it**. Investing in review process, automated tests and a shared handbook usually pays off more than adding reviewers. --- ## 8. The 15 points on one page A summary to print. | Group | Points | |---|---| | A. Understand the task | 1 Matches the request. 2 Tight scope. 3 Assumptions stated | | B. Correct and safe | 4 Real libraries and functions. 5 Inputs validated. 6 No exposed secrets. 7 Edge cases and errors. 8 Guardrails on destructive actions | | C. Tests | 9 Tests check real behavior. 10 Tests fail when code is wrong. 11 Runs on a clean machine | | D. Maintenance | 12 No duplication. 13 Simple and readable. 14 Follows conventions. 15 Small PR, clear description | --- ## 9. Conclusion When AI makes writing code cheaper, review becomes the stage that decides quality. A fixed checklist spares reviewers from relying on feel, and an honest PR description tells them where to look. AI can read first, but the person who signs is the one who is responsible. Start small: take the three most often missed points (4, 5, 10), apply them to the next PR and note what you catch. A week later, add the rest according to the bugs your team actually meets. --- ## References Figures come from public sources as of October 2026. Many sources are vendor reports or summaries, and some numbers are reported slightly differently, so open the originals to check before quoting. - Faros AI, AI Productivity Paradox (July 2025): [Faros summary](https://www.faros.ai/blog/lab-vs-reality-ai-productivity-study-findings) and [Software Seni summary](https://softwareseni.com/the-ai-productivity-paradox-in-software-development-and-what-the-research-actually-shows) (original not read) - Veracode, GenAI Code Security Report 2025: [Veracode press release](https://www.veracode.com/press-release/ai-generated-code-poses-major-security-risks-in-nearly-half-of-all-development-tasks-veracode-research-reveals/) and [Help Net Security](https://www.helpnetsecurity.com/2025/08/07/create-ai-code-security-risks/) - Pearce et al., [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://arxiv.org/pdf/2108.09293) (2021) - Perry et al., [Do Users Write More Insecure Code with AI Assistants?](https://arxiv.org/pdf/2211.03622) (Stanford, 2022) - Research on slopsquatting: [Dark Reading](https://darkreading.com/application-security/ai-code-tools-widely-hallucinate-packages) and [Infosecurity Magazine](https://www.infosecurity-magazine.com/news/ai-hallucinations-slopsquatting/) - GitClear, AI Copilot Code Quality 2025: [LeadDev analysis](https://leaddev.com/software-quality/how-ai-generated-code-accelerates-technical-debt) (a summary, original not read) - Sharma et al., [Towards Understanding Sycophancy in Language Models](https://arxiv.org/pdf/2310.13548) (ICLR 2024) - Google, [Engineering Practices: The Code Reviewer's Guide](https://google.github.io/eng-practices/review/)