--- title: "Three Hours with an AI Agent on a Real Project: A Log and the Numbers" url: https://www.thinnd.site/en/field-notes/agent-field-notes-real-project/ language: en author: "Nhu Duy Thin (Thinnd)" published: 2026-10-29 updated: 2026-10-29 section: "Field notes" tags: ["AI agent", "field notes", "measurement", "verification", "workflow"] summary: "In a three-hour session with an AI agent on this project, 7 of 12 commits were fixes after human feedback, even though they made up only about 5% of changed lines. Nearly half of the human's messages were bug reports or redo requests, and most bugs were things the eye sees, not logic errors." license: "Cite with the author name and the canonical URL above." --- # Three Hours with an AI Agent on a Real Project: A Log and the Numbers > These field notes are measured from two sources you can check: the git history of this blog project and the chat log between the author and the AI agent (Claude Code). No figure here was reconstructed. > > The limit, said upfront: this is **one session, one project, one pair of human and agent**, with no control group. It shows one concrete picture and does not prove anything about every project. --- ## Quick summary - Between the first and the last commit, **3 hours 18 minutes** passed, and the agent and the human together made **12 commits** with **6,922 lines** added or deleted in total. - The human sent **33 messages**: 11 commit commands, 7 requests for new work and **15 bug reports or redo requests** (about 45%). - **7 of the 12 commits** were fixes after feedback, yet they were only **323 lines**, about **4.7%** of the changed lines. Small bugs, many rounds. - The bugs a human found were mostly things **the eye sees**: a box nested in a box, a clipped dot, text in the wrong language, stiff wording. At least four **slipped past the automated checks** because no check of the right kind existed yet. - Once the agent **fixed the wrong place**: it moved a label while the real cause was an image frame. Measuring before fixing found the cause. - Practical lesson: every bug should leave behind **one new check** and **one line in the project handbook**. --- ## 1. Context The project is this personal static blog: Markdown content, a hand-written page generator, three languages (Vietnamese, English, Japanese) and slide decks. In the session the author handed work to the agent by message; the agent read and edited files, ran the build and check scripts, then the human looked at the result in the browser and replied. Commits were made only when the human asked. This differs from "an agent running alone overnight". It is the **human sitting beside it** style, with constant feedback. Keep that in mind when reading the numbers. --- ## 2. The commit log Times come from git history (local time, one day). "Type" is the commit prefix. | Time | Type | What | Why it exists | |---|---|---|---| | 11:08 | feat | Three-language translations and UI polish | The human sent UI bug screenshots and asked for a review | | 11:26 | feat | The article "Why You Should Not Trust AI Blindly" with a deck | New work | | 11:45 | feat | Made the article easier to follow, added dev examples | Feedback: hard to understand, dry terms | | 12:26 | fix | Rewrote vague slides, fixed the keyboard demo layout | Feedback with a bug screenshot | | 12:30 | docs | Aligned the article with the standard used for slides | Feedback: not up to standard | | 12:35 | fix | Translated leftover Vietnamese labels in the English and Japanese decks | Feedback with a screenshot | | 12:49 | feat | The article "IT Teamwork in the AI Era" with 21 slides | New work | | 12:55 | feat | Rewrote slide copy to be catchier | Feedback: wording was clumsy | | 12:57 | fix | Added the topic name to the cover | Feedback with a screenshot | | 14:15 | feat | SEO, GEO and AEO rollout (33 files) | New work | | 14:22 | fix | Kept the mascot antenna dot visible | Feedback with a screenshot | | 14:26 | fix | Fixed the real cause: the SVG frame clipped the dot | Feedback: still broken | Every big delivery was followed by a fix. After the teamwork deck, the first fix came **6 minutes** later. After the SEO package, the first fix came **7 minutes** later and the fix for the real cause **11 minutes** later. --- ## 3. The numbers | Measure | Value | Measured from | |---|---|---| | Span between first and last commit | 3 hours 18 minutes | git log | | Commits | 12 (6 features, 1 docs, 5 fixes) | git log | | Total lines added or deleted across commits | 6,922 (a line rewritten twice counts twice) | git log, per-commit stats | | Net difference against before the session | 40 files, 5,983 added, 311 deleted | git diff | | Human messages | 33 | Chat log | | Of which: commit commands, new work, bug reports or redo requests | 11, 7, 15 | Manual classification | | Commits that fixed things after feedback | 7 of 12 | Manual classification | | Lines in those fix commits | 323 of 6,922 (4.7%) | git log | How messages were classified: a "bug report or redo" is a message saying the result just delivered had an error, was unclear or missed the intent. "New work" asks for more content or a feature. The classification is by hand and a few messages could be argued. The most striking point is the **7 of 12 commits against 4.7% of lines**. Most of the effort sits in the number of returns, not in the volume of text. --- ## 4. Eleven bugs a human found These are representative bugs the human saw before the agent did. The last column gives the root. | # | What the human saw | Root | Group | |---|---|---|---| | 1 | The search box "does not work" | The filter logic was right, but results sat below the first screen so nothing seemed to happen | Experience | | 2 | The mascot's "hair" was black and vanished in dark mode | A fixed color inside the SVG | Visible | | 3 | The pale beige outline looked ugly | The agent picked a color to be visible, not to look good | Taste | | 4 | A Japanese title broke its lines and lost the meaning | The browser broke in the middle of a phrase | Language | | 5 | A slide said "temperature" and nobody understood it | A mechanical translation of a term | Wording | | 6 | A slide's opening line was vague, only announcing content | A sentence like "there is a way to picture how AI picks words" says nothing | Wording | | 7 | Vietnamese text left on a Japanese slide | Two cover labels typed in directly, outside the translation mechanism | Language | | 8 | A box nested in a box on the keyboard slide | A CSS rule hit the wrapper around translated text | Visible | | 9 | Slide titles copied the request word for word and sounded clumsy | The agent follows the words of the request | Wording | | 10 | The cover lacked the topic name | A catchy title replaced the topic name | Wording | | 11 | The mascot dot was still clipped after the first fix | The first fix targeted the wrong place: the label was not the culprit, the SVG frame was | Cause | Grouping: 4 about wording, 2 about language, 2 about what is visible, 1 about experience, 1 about taste, 1 about a wrong diagnosis. **Worth noting:** none of these is a logic error in the program. They sit in **how the result looks to an eye**, where the agent has no eyes unless the human gives it a picture, and where automated checks had not been taught to look. --- ## 5. Did automated checks catch them? The agent wrote check scripts: slide layout (overflow, overlap, titles wrapping), and later leaked-language text. These caught many bugs before the human saw them. But at least four bugs in the table above **slipped through because no check of the right kind existed at the time**: - Bugs 7, 8, 10 and 11: the layout check said "fine" or did not exist, yet the human still saw something wrong. A layout check does not ask "did text leak into the wrong language", "does the page name the right topic" or "is the dot clipped while it moves". Each time, the right response was to **add a new check**: the leaked-language scan, a measurement of the gap between the dot and the label at four widths, a rule that "the title must name the topic". Automated checks grow with the bugs a person meets. Bug 11 deserves a careful read. The gap measurement said no overlap, but the culprit was the SVG frame clipping the shape when the dot jumped up. Only reproducing the bug at the **peak of the motion** and comparing the shape with its frame revealed the cause. The lesson: **measure before fixing, and reproduce the exact situation**. It is the same pattern as in [why you should not trust AI blindly](https://www.thinnd.site/en/engineering/why-not-to-trust-ai-blindly/): the result looks complete, the internal check says fine, and a person looking finds what is wrong. --- ## 6. Six takeaways 1. **The human's feedback is mostly about what is seen and felt.** Pictures, words, names, layout. Prepare a way for the agent to see its own result (screenshots, script measurements) instead of waiting for a person to point. 2. **The cost is in rounds, not lines.** Seven fix commits were only 323 lines. To cut cost, cut the number of returns by stating "what done means" from the start. 3. **Automated checks only catch what they were taught.** After each bug, add a check. Otherwise the same kind returns. 4. **Measure before fixing.** The first dot fix missed because it followed the first plausible guess. Reproduce the bug in its worst state. 5. **The agent follows the words.** When the human said "instead of A, heading toward B", the agent copied it word for word into a title. Ask for rewording rather than copying the brief. 6. **Write the lesson into the project handbook.** Each repeated bug becomes a line in the guidance file (see [a handbook for AI](https://www.thinnd.site/en/engineering/writing-agents-md/)). This project now has four specialist guides and one shared notes file, and nearly every entry was born from a bug above. --- ## 7. Limits of these notes - **One session, one project, one pair.** The general level cannot be inferred. This is a content and interface project where visible bugs weigh heavily. A backend project would give a different picture. - **No control.** We do not know how long the human would have taken alone, so we cannot say the agent was faster or slower. The METR study mentioned in an earlier post warned: feeling fast is not being fast. - **Manual classification.** Messages and bugs were sorted by hand, and another person could file a few differently. - **Time comes from commits.** The real session was longer than the span between two commits, because commits are made only when the human asks. - **Not measured:** the time the human spent looking and typing feedback, quality as seen by readers, and the cost of using the agent. --- ## 8. Measure it in your own project A small log is enough to get your own numbers. Each task given to an agent is one row: | Column | Example | |---|---| | Task | "Add a search page" | | Start, finish | 10:05, 10:40 | | Rounds of fixes | 3 | | Bug type | visible, logic, wording, language, wrong diagnosis, taste | | Who found it | person, automated check, the agent itself | | Added a check or handbook line | yes, no | With two commands: ```bash git log --format='%h %ad %s' --date=format:%H:%M --shortstat git diff --shortstat HEAD ``` After a week, read three numbers: how many tasks needed more than two rounds, which bug type repeats most, and who finds most bugs. Those tell you whether to invest in automated checks, the handbook or how you hand over work. --- ## 9. Conclusion In three hours the agent produced a large volume quickly, and the human had to come back many times over small details that only a human eye sees. Both are true. The value of sitting beside an agent is in **turning every fix into a check or a handbook line**, so next time that bug does not need a person to point at it. These notes are only one sample, but the way of measuring can be reused. If you log one week of your own, those numbers will say what no one else's report can say for you. --- ## References Figures come from the git history of this project's repository (commits `3791938` to `ec1bc85`) and the chat log of the session. Related posts in this repository: - [Why you should not trust AI blindly](https://www.thinnd.site/en/engineering/why-not-to-trust-ai-blindly/) (with sources on the METR study, the Stack Overflow Developer Survey 2025 and studies of AI-written code) - [IT teamwork in the AI era](https://www.thinnd.site/en/experiences/team-work-in-the-ai-era/) (with sources on Faros AI and DORA 2025) - [Reviewing AI-written code](https://www.thinnd.site/en/engineering/reviewing-ai-written-code/) - [A handbook for AI](https://www.thinnd.site/en/engineering/writing-agents-md/) External references: - [git log](https://git-scm.com/docs/git-log) and [git diff](https://git-scm.com/docs/git-diff) (official documentation, the source of the commands used for measuring in "Measure it yourself") - [Faros AI: AI productivity, lab versus reality](https://www.faros.ai/blog/lab-vs-reality-ai-productivity-study-findings) and [DORA 2025](https://research.google/pubs/dora-2025-state-of-ai-assisted-software-development-report/) are two large-scale sources to compare against. This post is only a small sample and cannot replace them.