Three Hours with an AI Agent on a Real Project: A Log and the Numbers

These field notes are measured from two sources you can check: the git history of this blog project and the chat log between the author and the AI agent (Claude Code). No figure here was reconstructed.

The limit, said upfront: this is one session, one project, one pair of human and agent, with no control group. It shows one concrete picture and does not prove anything about every project.

---

Quick summary#

• Between the first and the last commit, 3 hours 18 minutes passed, and the agent and the human together made 12 commits with 6,922 lines added or deleted in total.
• The human sent 33 messages: 11 commit commands, 7 requests for new work and 15 bug reports or redo requests (about 45%).
• 7 of the 12 commits were fixes after feedback, yet they were only 323 lines, about 4.7% of the changed lines. Small bugs, many rounds.
• The bugs a human found were mostly things the eye sees: a box nested in a box, a clipped dot, text in the wrong language, stiff wording. At least four slipped past the automated checks because no check of the right kind existed yet.
• Once the agent fixed the wrong place: it moved a label while the real cause was an image frame. Measuring before fixing found the cause.
• Practical lesson: every bug should leave behind one new check and one line in the project handbook.

---

1. Context#

The project is this personal static blog: Markdown content, a hand-written page generator, three languages (Vietnamese, English, Japanese) and slide decks. In the session the author handed work to the agent by message; the agent read and edited files, ran the build and check scripts, then the human looked at the result in the browser and replied. Commits were made only when the human asked.

This differs from "an agent running alone overnight". It is the human sitting beside it style, with constant feedback. Keep that in mind when reading the numbers.

---

2. The commit log#

Times come from git history (local time, one day). "Type" is the commit prefix.

Every big delivery was followed by a fix. After the teamwork deck, the first fix came 6 minutes later. After the SEO package, the first fix came 7 minutes later and the fix for the real cause 11 minutes later.

---

3. The numbers#

How messages were classified: a "bug report or redo" is a message saying the result just delivered had an error, was unclear or missed the intent. "New work" asks for more content or a feature. The classification is by hand and a few messages could be argued.

The most striking point is the 7 of 12 commits against 4.7% of lines. Most of the effort sits in the number of returns, not in the volume of text.

---

4. Eleven bugs a human found#

These are representative bugs the human saw before the agent did. The last column gives the root.

Grouping: 4 about wording, 2 about language, 2 about what is visible, 1 about experience, 1 about taste, 1 about a wrong diagnosis.

Worth noting: none of these is a logic error in the program. They sit in how the result looks to an eye, where the agent has no eyes unless the human gives it a picture, and where automated checks had not been taught to look.

---

5. Did automated checks catch them?#

The agent wrote check scripts: slide layout (overflow, overlap, titles wrapping), and later leaked-language text. These caught many bugs before the human saw them. But at least four bugs in the table above slipped through because no check of the right kind existed at the time:

• Bugs 7, 8, 10 and 11: the layout check said "fine" or did not exist, yet the human still saw something wrong. A layout check does not ask "did text leak into the wrong language", "does the page name the right topic" or "is the dot clipped while it moves".

Each time, the right response was to add a new check: the leaked-language scan, a measurement of the gap between the dot and the label at four widths, a rule that "the title must name the topic". Automated checks grow with the bugs a person meets.

Bug 11 deserves a careful read. The gap measurement said no overlap, but the culprit was the SVG frame clipping the shape when the dot jumped up. Only reproducing the bug at the peak of the motion and comparing the shape with its frame revealed the cause. The lesson: measure before fixing, and reproduce the exact situation.

It is the same pattern as in why you should not trust AI blindly: the result looks complete, the internal check says fine, and a person looking finds what is wrong.

---

6. Six takeaways#

1. The human's feedback is mostly about what is seen and felt. Pictures, words, names, layout. Prepare a way for the agent to see its own result (screenshots, script measurements) instead of waiting for a person to point.
2. The cost is in rounds, not lines. Seven fix commits were only 323 lines. To cut cost, cut the number of returns by stating "what done means" from the start.
3. Automated checks only catch what they were taught. After each bug, add a check. Otherwise the same kind returns.
4. Measure before fixing. The first dot fix missed because it followed the first plausible guess. Reproduce the bug in its worst state.
5. The agent follows the words. When the human said "instead of A, heading toward B", the agent copied it word for word into a title. Ask for rewording rather than copying the brief.
6. Write the lesson into the project handbook. Each repeated bug becomes a line in the guidance file (see a handbook for AI). This project now has four specialist guides and one shared notes file, and nearly every entry was born from a bug above.

---

7. Limits of these notes#

• One session, one project, one pair. The general level cannot be inferred. This is a content and interface project where visible bugs weigh heavily. A backend project would give a different picture.
• No control. We do not know how long the human would have taken alone, so we cannot say the agent was faster or slower. The METR study mentioned in an earlier post warned: feeling fast is not being fast.
• Manual classification. Messages and bugs were sorted by hand, and another person could file a few differently.
• Time comes from commits. The real session was longer than the span between two commits, because commits are made only when the human asks.
• Not measured: the time the human spent looking and typing feedback, quality as seen by readers, and the cost of using the agent.

---

8. Measure it in your own project#

A small log is enough to get your own numbers. Each task given to an agent is one row:

With two commands:

git log --format='%h %ad %s' --date=format:%H:%M --shortstat
git diff --shortstat <commit-before-the-session> HEAD

After a week, read three numbers: how many tasks needed more than two rounds, which bug type repeats most, and who finds most bugs. Those tell you whether to invest in automated checks, the handbook or how you hand over work.

---

9. Conclusion#

In three hours the agent produced a large volume quickly, and the human had to come back many times over small details that only a human eye sees. Both are true. The value of sitting beside an agent is in turning every fix into a check or a handbook line, so next time that bug does not need a person to point at it.

These notes are only one sample, but the way of measuring can be reused. If you log one week of your own, those numbers will say what no one else's report can say for you.

---

References#

Figures come from the git history of this project's repository (commits 3791938 to ec1bc85) and the chat log of the session. Related posts in this repository:

• Why you should not trust AI blindly (with sources on the METR study, the Stack Overflow Developer Survey 2025 and studies of AI-written code)
• IT teamwork in the AI era (with sources on Faros AI and DORA 2025)
• Reviewing AI-written code
• A handbook for AI

External references:

• git log and git diff (official documentation, the source of the commands used for measuring in "Measure it yourself")
• Faros AI: AI productivity, lab versus reality and DORA 2025 are two large-scale sources to compare against. This post is only a small sample and cannot replace them.