---
title: "Learning an Unfamiliar Library with AI Without Losing the Skill: A Four-Week Plan"
url: https://www.thinnd.site/en/journeys/learning-with-ai-keep-your-skills/
language: en
author: "Nhu Duy Thin (Thinnd)"
published: 2026-10-30
updated: 2026-10-30
section: "Journeys"
tags: ["learning with AI", "learning to code", "skills", "learning path", "dev skills"]
summary: "You can learn with AI and keep your skills if you do not hand everything to it. In Anthropic's experiment with 52 developers learning the Trio library, the AI group scored about 50% against 67% for the hand-coding group, but three ways of using AI that involved asking and explaining kept learning intact. Suggested rules: predict first, ask for explanation, rewrite from memory, keep AI-off sessions."
license: "Cite with the author name and the canonical URL above."
---

# Learning an Unfamiliar Library with AI Without Losing the Skill: A Four-Week Plan

> For anyone learning a new library or language, especially students and freshers. No background in research or statistics is needed, and every technical term has a short explanation.
>
> The evidence on this topic is **small and recent**, mostly small, short studies. The four-week plan is **my suggestion and nobody has tested it**. Treat it as an experiment you run on yourself.

---

## Quick summary

- **How you use AI may matter more than whether you use it.** In a small Anthropic experiment, the group that handed the work to the AI understood less, while the group that used AI to ask and understand kept its score.
- **Faster does not always mean learning more.** The AI group was only a few minutes faster, and that difference was not enough to conclude anything.
- **A feeling is not a measurement.** In another study (METR), people using AI took 19% longer yet believed they were faster.
- **A four-week plan:** AI-on sessions, AI-off sessions and a 10-point self-test every Friday.
- **Four rules:** predict before you ask, ask for explanation and not only code, rewrite from memory, keep AI-off sessions.
- **Sometimes letting AI go first is fine.** The article says when.

---

## Hard words, plain meaning

| Term | Plain meaning |
|---|---|
| **Library** | Ready-made code you call on, like a toolbox. |
| **Predict before running** | Write down the result you expect before you run the code or ask the AI. |
| **Recall without looking (retrieval practice)** | Write or say what you just learned without opening the notes. |
| **Delegation** | Ask the AI to write the whole piece and paste it in without reading or trying it. |
| **AI-off session** | A study block with the AI assistant off, using only official docs. |
| **Self-test** | A short quiz with no notes, to see what you remember and can do. |
| **Control group** | The group that does not get the thing being tested, used for comparison. |

---

## 1. Picture it: watching someone cook and following a GPS

Picture two ways to learn cooking. In the first, you stand and watch a chef finish a whole dish. You get to eat it, but the next day you may not be able to cook it again. In the second, you cook it yourself while a helper stands next to you, points out the mistake and explains why. The dish may come out slower, but your hands remember.

Directions work the same way. Following a GPS gets you there fast, but switch it off and you are lost. Learning the map is slower at first, and then you can find your way even with no signal.

To picture it, think of AI as both the chef and the GPS: it can do things for you, which is both its strength and its risk. The question of this article is: **when should you let it do the work, and when should you switch it off so your own hands remember?**

One general note: nobody fully understands the inside of AI, and research on learning with AI is only a few years old. The pictures in this article are only aids.

---

## 2. What the evidence says

| Study | Who, what was measured | Main result | How to read it |
|---|---|---|---|
| **Shen and Tamkin, Anthropic (arXiv 2601.20245, January 2026)** | 52 developers learned Python's Trio library for about an hour, one group with an AI chat and one writing by hand, then took a 27-point quiz | AI group about **50%**, hand-coding group about **67%**, a gap of about **17%** (about 4 of 27 points in the paper), largest on debugging. A few minutes faster, not statistically significant | A strong signal but a small sample, measured right after the session, run by an AI vendor |
| **Bastani and colleagues, PNAS 2025** | Nearly 1,000 high-school students in Turkey, math, GPT-4 | During practice: GPT Base **48%** higher, GPT Tutor **127%** higher. On an exam with AI removed: GPT Base **17%** lower than students who never had AI; GPT Tutor about the same as the control group | School students, math. The version with prompts that protect learning avoided most of the harm |
| **METR, July 2025** | 16 experienced developers, 246 real tasks in their own open-source projects | With AI they took **19%** longer, while believing they were about **20%** faster | Measures productivity, not learning. METR says the figures are now out of date (update in February 2026) |
| **Lee and colleagues, Microsoft and CMU, CHI 2025** | A survey of 319 knowledge workers | More trust in AI went with less critical thinking, more self-confidence went with more | Self-reported and only a correlation, not proof that AI reduces thinking |

The figures come from Anthropic's research page and the arXiv paper. The two round slightly differently (one gives 50% and 67%, the other a gap of 4.15 points out of 27), so I write "about". The AI group finished a few minutes faster, but the summaries give slightly different numbers and none is statistically significant.

### Six ways of using AI in Anthropic's experiment

The researchers reviewed screen recordings and sorted the AI users into six patterns:

| Pattern | People | Quiz score |
|---|---|---|
| Handed everything to AI | 4 | under 40% |
| Asked a few questions, then handed it all over | 4 | under 40% |
| Used AI only to fix errors | 4 | under 40% |
| Generated code, then asked to understand it | 2 | about 86% |
| Asked for code with explanations | 3 | about 68% |
| Asked concepts only, fixed errors alone | 7 | about 65% |

The first three learned less, the last three kept learning. What the last three share is that you still **do your own thinking**: asking to understand, or handling errors yourself. But each group has only 2 to 7 people and this is a descriptive analysis, so read it as a hint about usage, not a law.

### Why "predict first" and "rewrite from memory" make sense

Teaching guidance from MIT's Teaching and Learning Lab says that actively retrieving information improves long-term retention compared with re-reading, citing Roediger and Karpicke (2006). The same page says that guessing wrong and then seeing an explained answer improves learning, and suggests telling learners that such effort is a "desirable difficulty" (Bjork 1994). A University of Arizona guide adds that questions asking learners to predict or compare help more than questions that only ask for definitions.

This is evidence from classrooms in general. **Nobody has tested this exact plan with AI.** I use it as a reasonable basis for designing the rules, not as proof that the plan works.

---

## 3. Illustration: the same task, two ways to learn

*This is an illustration with made-up numbers, not real people.*

A learner is writing a data-loading step with a new library.

**Way A, hand it all over:**

| Time | What happens |
|---|---|
| 10:00 | Asks the assistant to write the whole piece. |
| 10:10 | It runs. Done after 10 minutes. |
| 11:30 | The reviewer asks for a one-line change. The learner does not understand this part and has to ask again from scratch. |

**Way B, predict first, then ask why:**

| Time | What happens |
|---|---|
| 10:00 | Writes a prediction: running it will print 3 lines. |
| 10:05 | Runs it and gets 2 lines. Asks the assistant why, then closes the window and rewrites it from memory. |
| 10:25 | Done after 25 minutes. |
| 11:30 | The reviewer asks for the same one-line change. The learner makes it in 3 minutes. |

Way B is slower by 25 minus 10, which is **15 minutes**. In return, at 11:30 the learner can make the change alone. The times here are made up for illustration, not measurements.

What supports this example is the evidence in section 2, not the example itself: the delegation groups scored lower in Anthropic's experiment.

---

## 4. Four rules

1. **Predict before you ask.** Write one line: "I think the result will be X". Then run it or ask, and compare. A wrong guess is fine, since the gap is exactly what you are about to learn.
2. **Ask for explanation, not just code.** Ask why this piece is written this way, when it would fail, and whether there is another way. In Anthropic's study this is close to the group that asked for code with explanations.
3. **Rewrite it from memory.** Once you understand, close the AI tab and the docs and retype it. Mark where you got stuck, reopen, then type it again.
4. **Keep AI-off sessions.** Each week has a block with only official docs and the machine's error messages. That is when you check whether your hands still remember.

Two side rules: when you receive code from AI, [review it like a stranger's code](https://www.thinnd.site/en/engineering/reviewing-ai-written-code/), and remember that AI [can be wrong in a very confident voice](https://www.thinnd.site/en/engineering/why-not-to-trust-ai-blindly/).

---

## 5. The four-week plan

I assume **4 sessions of 60 minutes** a week (Monday to Thursday) and a **30-minute self-test on Friday** (15 minutes for the test, 15 minutes to review). Pick **one** library or language so you can measure.

| Week | Focus | AI-off / AI-on sessions | Output |
|---|---|---|---|
| **1. Map first** | Read the official tutorial, run examples, predict then run | 3 off, 1 on (to ask for explanations) | A prediction notebook and a first test as your baseline |
| **2. AI explains** | Turn AI on to ask why and when not to use it, not for long code | 2 off, 2 on | Ten good questions, each with an answer in your own words |
| **3. Build with AI** | A small project: AI drafts, you read, then rewrite from memory | 2 off, 2 on | A small working project where you can explain every part |
| **4. Solo first** | Build version two without AI, then ask AI to review | 3 off, 1 on | Version two and a list of what the AI found that you missed |

### The arithmetic

- 4 sessions a week, 60 minutes each, over 4 weeks: 4 times 4 times 60 equals **960 minutes** of study.
- Fridays: 4 times 30 equals **120 minutes**. Whole plan: 960 plus 120 equals **1,080 minutes**, which is 18 hours.
- AI-off minutes: weeks 1 and 4 have 180 each, weeks 2 and 3 have 120 each, total 600. Add 120 minutes of Fridays (no AI) to get **720 minutes**.
- AI-off share: 720 divided by 1,080 equals **66.7%**. AI-on minutes are 360.

This ratio is my choice, and no study says what the right number is. If you are under deadline pressure, keep at least the week 1 sessions and the Friday test.

---

## 6. The Friday self-test

No AI, no notes, **15 minutes, 10 points**:

| Part | Points |
|---|---|
| Explain 3 concepts from the week aloud or in writing, as if to a classmate | 3 |
| Predict the output of 2 short snippets | 2 |
| Rewrite 1 function you learned, from memory | 3 |
| Find 1 bug in a 15-line snippet | 2 |

Then open the docs, mark it again and write the misses in your notebook. The misses are next week's homework.

**A made-up example of one learner:** week 1 scores 4 out of 10 (40%), week 2 scores 5 (50%), week 3 scores 7 (70%), week 4 scores 8 (80%). From week 1 to week 4 that is 4 more points, or 40 percentage points. These numbers are invented, and a self-marked test has error. Watch **your own trend**, not other people.

One way to compare yourself: each week pick two concepts you learned with AI on and two you learned with AI off, and see which you remember better on Friday. Four items is a tiny sample, so use it only to learn about yourself.

---

## 7. When letting AI go first is fine

| Usually fine | Better with AI off |
|---|---|
| A throwaway script | A core skill of your job |
| Repetitive work you already know | A new concept with no base yet |
| Something you do not need to learn | Debugging and reading code |
| A test already checks the result | Interview preparation |

Anthropic notes that an earlier study of theirs found AI raised productivity on tasks people had already mastered, so the two results may not conflict.

---

## 8. A balanced view

- **The experiment lasted only about an hour.** It is unknown whether the gap narrows or stays after months.
- **Small sample, one library, one quiz.** The authors state these limits too, and another experiment might give a different result.
- **Run by an AI vendor.** The results are somewhat unfavorable to using AI, but you should still read the original and compare it with independent sources.
- **The tool in the experiment was a chat.** Agentic tools that write whole projects may behave differently, and the authors say so.
- **No comparison with learning from a person.** It is unclear whether AI or a good mentor is better.
- **The PNAS math study used school students.** The results may not carry over to adults learning to code.
- **The Microsoft and CMU survey is self-reported and only a correlation.**
- **This plan has not been tried on many people.** If it feels too slow or too strict, adjust the AI-off ratio, as long as the Friday test stays so you can measure.

---

## 9. Conclusion

The question is not "should I use AI to learn", but **how to use it so my own hands still remember**. The evidence so far, though thin, points one way: people who keep thinking, asking why and retyping keep their learning. Predict first, ask for explanation, rewrite from memory, keep AI-off sessions, and measure with the Friday test.

Start today: pick one library, write your first prediction in a notebook, and book your first Friday test.

If you lack the basics, see the [12-month plan for freshers](https://www.thinnd.site/en/journeys/fresher-ai-era-12-month-plan/). If an interview is coming, see the [two-week plan](https://www.thinnd.site/en/journeys/ai-era-interview-two-week-plan/). To hear two people talk about this, see the [dialogue between a fresher and a senior](https://www.thinnd.site/en/conversations/fresher-senior-dialogue-learning-in-ai-era/).

---

## References

Open the originals before quoting, because figures may be updated.

- [Shen and Tamkin, "How AI Impacts Skill Formation", arXiv 2601.20245 (January 2026)](https://arxiv.org/abs/2601.20245)
- [Anthropic, research page "How AI assistance impacts the formation of coding skills"](https://www.anthropic.com/research/AI-assistance-coding-skills)
- [Bastani and colleagues, "Generative AI without guardrails can harm learning", PNAS 2025](https://www.pnas.org/doi/10.1073/pnas.2422633122)
- [Wharton, summary of the Bastani and colleagues study](https://knowledge.wharton.upenn.edu/article/without-guardrails-generative-ai-can-harm-education)
- [METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" (July 2025)](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/)
- [Lee and colleagues, "The Impact of Generative AI on Critical Thinking", CHI 2025 (authors' PDF)](https://advait.org/files/lee_2025_ai_critical_thinking_survey.pdf)
- [MIT Teaching + Learning Lab, "Help Students Retain, Organize and Integrate Knowledge"](https://tll.mit.edu/teaching-resources/how-to-teach/help-students-retain-organize-and-integrate-knowledge/)
- [Roediger and Karpicke, "Test-enhanced learning", Psychological Science (2006)](https://doi.org/10.1111/j.1467-9280.2006.01693.x)
- [University of Arizona, "L2L Strategy: Retrieval Practice"](https://academicaffairs.arizona.edu/l2l-strategy-retrieval-practice)
- Related posts on this blog: [A Fresher in the AI Era: a 12-month plan](https://www.thinnd.site/en/journeys/fresher-ai-era-12-month-plan/), [Interviewing in the AI Era: a two-week plan](https://www.thinnd.site/en/journeys/ai-era-interview-two-week-plan/), [A fresher and a senior in dialogue](https://www.thinnd.site/en/conversations/fresher-senior-dialogue-learning-in-ai-era/), [Reviewing AI-written code](https://www.thinnd.site/en/engineering/reviewing-ai-written-code/), [Why you should not trust AI blindly](https://www.thinnd.site/en/engineering/why-not-to-trust-ai-blindly/)