2. Training Data: Where the Model Learns From, and What It Learns¶

The result of fine-tuning is determined almost entirely by the data: however the model ends up talking is exactly what the demonstrations you show it look like. So before you spend money renting a GPU, get a clear picture of the data first.

Chapter at a glance

What this chapter does: read one fact card and one conversation field by field; see how a conversation is split into training samples; see exactly what the model "sees and learns"
What you need: just a browser or editor; if you want to run the checks yourself, section 2.8 explains how to install a lightweight environment on your own computer (no GPU needed)
What you get: you can explain where the number 1150 comes from; you can point out which tokens in a sample contribute to the loss; you know how to make data changes take effect

You can remember the whole data flow in one sentence: one JSON file → split into samples of "context + one reference answer" → turned into tokens by the chat template → loss computed only at the reference-answer positions. The sections below follow that order.

2.1 Maintain just one dataset.json¶

Open data/dataset.json. All the reference material, the 500 training conversations and the evaluation conversations live in this one file. At the top level there are four sibling arrays: facts holds the reference fact cards, and train, val and test are three sets of complete conversations. Only train actually takes part in training; fact cards serve only as context, not as a learning target.

The three sets are split by science topic (group_id): any one topic appears in only one set, so the model never meets a topic on val/test that it already saw during training.

Set	Science topics	Full conversations	Answer targets	Purpose
train	50	500	1150	Update the LoRA parameters
val	5	50	115	Compare configurations, pick a candidate
test	5	50	115	Evaluate after the candidate is frozen

Why do we need a val set at all? Isn't train plus test enough?

If you train once and just take the result, two sets are enough. The trouble starts when you make choices based on test results: train with r=16, look at test; not happy, switch to r=8, look at test again; then tweak the learning rate, change the prompt... and finally pick whichever version scores highest on test. At that point test no longer counts as "unseen data": the model parameters never touched it, but every one of your choices was guided by it, which is the same as using it to pick the answer, so the score you end up with is inflated.

So keep "data used for making choices" separate from "data used for the final grade". A backend release analogy: train is the development environment, where the model updates its parameters again and again; val is staging, which you can look at as often as you like to compare and choose; test is real production traffic, which you look at only once after the candidate is settled, to produce the final report.

In this project val supports these choices: computing validation loss every 12 steps during training to judge whether it is overfitting (section 4.5); comparing r16 and r8 (section 6.4); comparing group B, "base model + a good prompt", with group C, LoRA (sections 6.5 and 6.6); and finally choosing the candidate (section 6.7). Once chosen, you freeze it with evaluate freeze; only then does the code allow inference on test, and it does not let you use --limit to cherry-pick a subset of questions. If you look at test and then go back to change the model or the data, the next round needs a fresh test set.

When you see "just a training set and a test set" elsewhere, it usually means they train only once and make no comparisons; or they use cross-validation, rotating a validation slice out of train (the val set is hidden inside train); or they simply use test as val, in which case the test score they report is optimistic.

Training grade-band distribution: 167 conversations for grades 1–2, 167 for grades 3–4, 166 for grades 5–6. Each topic has 10 conversation paths, each with 2–3 turns; they include explanations of the same knowledge at different grade bands as well as follow-ups that change depending on the child's different reactions.

data/dataset.json
├── schema_version / description   文件版本与说明
├── facts[60]                      参考事实卡，只存一份，被对话按 id 引用
├── train[500]                     训练对话：唯一参与参数更新的部分
├── val[50]                        验证对话：比较配置、选候选、算 eval loss
└── test[50]                       测试对话：候选冻结后才使用

train / val / test 里的每段对话结构相同：
├── id / group_id                  对话编号 / 科学主题分组
├── grade_band                     1-2 / 3-4 / 5-6
├── fact_ids                       引用 facts 中的 id
└── turns[]                        user / assistant / behavior

(In English: schema_version / description are the file version and description; facts[60] are the reference fact cards, stored once and referenced by conversations via id; train[500] are the training conversations, the only part that updates parameters; val[50] are validation conversations for comparing configurations, picking a candidate and computing eval loss; test[50] are test conversations used only after the candidate is frozen. Every conversation in train / val / test has the same structure: id / group_id are the conversation id and science-topic group; fact_ids references ids in facts.)

Which set a conversation belongs to is determined solely by which array it sits in. Conversations no longer carry a split field, which avoids self-contradictions like "sits in val but says train". When the program reads the file (lab.data.read_dataset; in the notebook, tag_splits), it adds split to each conversation in memory, and all later filtering and cross-set checks use it; the source file stays unchanged. To move a conversation to another set, cut the whole conversation into the other array, and move the other conversations with the same group_id and the fact cards it references along with it.

JSON does not support // comments. The fields are explained here, and the notebook marks the places where they are actually used with D1–D7. Do not stuff the whole JSON file into the prompt, and do not pass the evaluation conversations to the trainer.

2.2 The first fact card¶

{
  "id": "day-night",
  "category": "space",
  "topic": "昼夜",
  "text": "自转使地球表面各处交替朝向和背向太阳，形成主要的昼夜变化。太阳没有按昼夜关闭发光。地球还绕太阳公转；假定取消自转但保留公转，光照位置仍会随公转缓慢变化。",
  "source_url": "https://spaceplace.nasa.gov/seasons/en/",
  "checked_at": "2026-09-29",
  "source_review": "reference_read_claim_review_pending",
  "scope": "昼夜的入门解释；模型和比喻不能替代真实测量。",
  "kind": "fact",
  "reuse": "科学事实摘要和教学对话为原创中文表达；来源仅供事实核对，不复制来源的教材段落、题目或图像，不把网页正文纳入训练。链接不表示来源机构认可本数据或授予网页内容的训练许可。"
}

(Roughly: the topic is "day and night". text: "Rotation makes each place on Earth's surface alternately face toward and away from the Sun, producing the main day-night cycle. The Sun does not switch off its light at night. Earth also orbits the Sun; if rotation stopped but the orbit continued, the lit area would still shift slowly with the orbit." scope: "An introductory explanation of day and night; models and analogies cannot replace real measurement." reuse: the summaries and dialogues are original Chinese writing, sources are used only for fact-checking, and a link does not mean the source endorses this data or grants a training license.)

Field	Meaning and use
`id`	Primary key, referenced by a conversation's `fact_ids`
`category/topic`	Category and topic, for lookup, statistics and review
`text`	Scientific material an answer can be based on; enters the context
`kind/scope`	Material type and scope of applicability; enters the context
`source_url`, `checked_at`, `source_review`, `reuse`	Source and review records, for humans only; does not enter the model

Only the four fields id, text, kind and scope are put into the prompt. The other fields are for reviewers; you can skip them on a first read.

Why fact cards are needed: three common questions¶

Fine-tuning teaches "how to say it" (section 0.1). Beginners who get this far usually come up with three questions in a row.

Q1: Primary-school science is common knowledge, and the base model already knows it. Why have fact cards at all?

True, the base model knows most of this common knowledge. If the only question were "where does the knowledge come from", fact cards would not be necessary. We still want them, and the reason is not to add knowledge:

So training learns only "how to say it". The loss is computed on the reference answer, and the reference answer contains both phrasing and factual content. When the input has no material, the model can only memorize content like "turning toward the Sun" and "the side facing away from the light" into its weights, and training ends up learning "phrasing" and "knowledge" mixed together. When the material is right there in the input, the model can predict the content correctly just by following the material, so LoRA's limited capacity goes mainly to "short sentences, correct the misconception first, offer one safe small observation". The mechanism is explained in Q3.
You can't inspect or edit the base model's knowledge, but you can inspect and edit fact cards. You can't tell in advance which question a 4B model will get wrong (for example, attributing the seasons to distance from the Sun), and when it is wrong you can't open the weights and fix it. A fact card is a visible piece of text with a source and a review status; if it is wrong you fix one card, and the change takes effect on the next request. A backend analogy: business rules live in configuration or a database, not scattered through a compiled binary.
Cards carry boundaries the base model won't respect on its own. For example, the text of day-night states "if rotation stopped but the orbit continued, the lit area would still shift slowly with the orbit", and the follow-up question in the second turn of the reference conversation relies on it; scope states "models and analogies cannot replace real measurement"; kind distinguishes fact, legend and current, so a legend must not be told as fact.
This is what real products look like, and it is the basis for evaluation. In real business, what a model has to explain is often something it doesn't know at all: product specs, a particular school's textbook, today's data. The common approach is for the caller to look up the material and put it in the prompt while the model explains it clearly, which is what is usually called RAG (retrieval-augmented generation). This project uses fact cards to practice that pattern end to end. In blind review, graders also check each claim against the fact card to judge whether it is supported (section 6.6).

Q2: Fact cards are in the training samples too, so does the model remember this knowledge after training?

That's not the right way to think about it. Fact cards are the input of a training sample, not the learning target. In the 408-token sample in section 2.5, the fact card is among the 386 tokens with mask=0: the model can see it, but is not penalized for "not remembering it". The loss tests one thing only: given this material, how closely does the answer match the reference answer.

	Learned after training?	Why
How to read the material and use it to structure an answer	Yes	This is what the loss pushes it to learn
The content and boundaries of the material itself	Not reliably	During training the material was always right in front of it; no need to memorize

To be honest about one thing: the reference answers themselves contain factual content, which sits in the mask=1 region, so the model will incidentally pick up some common phrasings. But with only 1 epoch, seeing each sample once, that memory is shallow; it can't remember the full content and boundaries of the cards, and you have no way of knowing what it did remember or got wrong. That is why at deployment you must pass the fact cards every time, and why the call example in section 5.5 first uses read_dataset to load the facts.

To verify this yourself: after training, ask the same question twice, once with the fact card and once with the CURRENT REFERENCE FACTS section removed, and compare the accuracy and boundaries of the answers. The bigger the difference, the more the model relies on the material given on the spot rather than on memorized knowledge.

Q3: If we don't rely on it to memorize knowledge, why include it during training? Can't we just pass it at deployment?

No. Training is not about making the model remember the fact cards; it is about teaching it to use them, and to learn to use them it must see them during training. An analogy with training a new employee:

	Method A: Q&A only	Method B: Q&A with the product manual alongside (this project)
Training material	Customer asks X → standard reply Y	Customer asks X + relevant manual page → standard reply Y
Habit formed	Hear the question, recite the answer from memory	Check the manual first, then explain it in the standard tone
Handed the manual on the job	Never practiced reading the manual, probably still recites old answers	Exactly the process they practiced
One entry in the manual is updated	Keeps giving the old answer	Follows the new manual
Faces a question never covered in training	Bluffs from vague memory, prone to making things up	Explains from the manual, doesn't make up what the manual doesn't say

Here is the same sample from this project; the target answer is identical, only the input differs:

不带事实卡：
  输入：为什么白天亮、晚上黑？
  目标：地球在不停转，我们住的地方转向太阳就是白天……

带事实卡：
  输入：[事实卡：自转使地球表面交替朝向和背向太阳……] 为什么白天亮、晚上黑？
  目标：地球在不停转，我们住的地方转向太阳就是白天……

(In English: "Without fact card — input: Why is it bright in the day and dark at night? Target: The Earth keeps turning; when the place we live turns toward the Sun, it is daytime…" and "With fact card — input: [Fact card: Rotation makes Earth's surface alternately face toward and away from the Sun…] Why is it bright in the day and dark at night? Target: the same.")

Without the card, the input contains no basis at all, yet the target is a definite scientific conclusion, so what the 1150 samples keep reinforcing is "hear a question, produce a definite conclusion out of thin air". With the card, everything in the target can be found in the input, so what gets reinforced is "take content from the given material and rewrite it in words a child can understand".

If you leave the cards out during training and add them only at deployment, four problems appear:

The model never practiced reading material. At deployment a big extra CURRENT REFERENCE FACTS block shows up; to the model it is unfamiliar input that may be ignored, and it answers from memory as before.
Editing the material has no effect. If you find a card is wrong, or want to add an applicability boundary to some card's scope, a Method A model won't change accordingly; you can only edit the training data and retrain.
New topics are more likely to be made up. Method A builds the habit of "producing conclusions out of thin air", so on topics not in the training data it still draws conclusions confidently.
The input format doesn't match. During training the prompt has no fact cards, at deployment it does; the way the model has learned to talk depends on what its training inputs looked like. This is also why section 5.5 stresses that "the prompt must match the one used in training".

In code terms, training is learning a function answer = f(question, facts, grade_band, history). If training only provides question, what it learns is f(question); passing an extra parameter facts at deployment that it has never used won't make it start using it.

You can train and answer without fact cards; the cost is everything above: mistakes you can't fix, a tendency toward overconfidence, and starting over whenever you move to a domain the base model doesn't know. Separating "facts" from "phrasing" is this project's design trade-off, not the only way to do it; its benefit ultimately has to be verified with the comparisons in chapter 6.

Where fact cards are used¶

Why they are stored separately. The 60 cards are referenced by the 600 conversations via fact_ids (one conversation can reference several cards), much like a foreign key in a database: each fact is edited in one place only, and every conversation that references it uses the new content the next time it is read.

Why they are also isolated by set. A card may only be referenced by conversations in one set (one of train/val/test). Otherwise the material behind the validation and test questions would already have been seen by the model during training, and the evaluation results would be optimistic.

Stage	Code location	How fact cards are used
Load and validate (D1)	data.py `validate`	`id` is unique; all fields present; `source_url` is https; `kind` must be fact/legend/current; every `fact_ids` entry in a conversation must exist; the same card cannot be shared across train/val/test
Build the prompt (D3)	common.py `build_messages`	Fetches cards by `fact_ids`, keeps only `id/text/kind/scope`, and writes them as JSON after `CURRENT REFERENCE FACTS:` in the system prompt; source and review fields do not enter the model
Notebook training and validation samples	Notebook step 2, `prepare-data` cell	All 1150 train samples and 115 val samples go through `build_messages` and carry their own fact cards
Notebook observation questions	Notebook step 4, `cases` cell	Each of the three observation questions specifies a `fact_id`, and the same card is used to compare answers before and after training
CLI snapshot	prepare.py	Written as-is into the snapshot's `facts.jsonl`; few-shot examples record their `fact_ids`
A/B/C comparison inference	infer.py	Reads `facts.jsonl` from the snapshot; all three groups get the same fact card for the current question (section 6.4)
Blind review	evaluate.py `blind`	The score sheet includes `fact_ids`; graders fill in `fact_claims`/`supported` by checking against the fact card (section 6.6)
Calls after deployment	Call example in section 5.5	`read_dataset` loads the facts, and `messages_for(fact_id, ...)` builds the same system prompt as in training

Source and review fields¶

source_url is the entry point for a human to go back and check the source; the program does not go online and read it automatically. checked_at only means the reference entry point has been read; source_review set to reference_read_claim_review_pending means the individual claims are still awaiting review. reuse is the source attribution and a note on original rewriting; it does not mean the source organization endorses this data. Official science outreach from NASA, NOAA, USGS and others, and OpenStax textbooks, are the reference entry points; the body text is original Chinese writing, and the analogies, children's reactions and observation designs are synthetic demonstrations. A URL being present does not mean every sentence is supported by a source.

2.3 The first complete conversation¶

{
  "id": "day-night-01",
  "group_id": "day-night",
  "category": "space",
  "topic": "昼夜",
  "grade_band": "1-2",
  "scenario": "repair_misconception",
  "fact_ids": [
    "day-night"
  ],
  "provenance": {
    "type": "synthetic",
    "author": "Codex",
    "date": "2026-09-29",
    "method": "按主题逐项撰写知识、误解、追问、比喻边界和安全观察，再组合为10种教学对话路径；不是儿童真实对话，部分主题内回答复用。"
  },
  "review": {
    "status": "draft",
    "reviewer": "",
    "reviewer_kind": "",
    "notes": "待逐段核查科学事实、来源支持、适读学段、观察可行性、近重复与上下文衔接；成人审阅不代表已验证儿童探索兴趣。"
  },
  "turns": [
    {
      "user": "为什么白天亮、晚上黑？",
      "assistant": "地球在不停转，我们住的地方转向太阳就是白天，转到背光的一面就是晚上。",
      "behavior": "explain_at_grade"
    },
    {
      "user": "太阳晚上关灯睡觉了吧？",
      "assistant": "太阳没有在晚上关灯。是我们所在的地面转到了背向太阳的一侧。还可以接着想：如果地球不转，就没有黑夜了吗？",
      "behavior": "repair_then_offer_specific_question"
    }
  ]
}

(The two turns in English: the child asks "Why is it bright in the day and dark at night?" and the assistant answers "The Earth keeps turning. When the place we live turns toward the Sun it is daytime; when it turns to the side away from the light, it is night." The child then asks "The Sun turns off its light and goes to sleep at night, right?" and the assistant answers "The Sun doesn't turn off its light at night. It's the ground we're on that has turned to the side facing away from the Sun. Here's something to think about next: if the Earth didn't turn, would there be no night at all?" provenance.method says the dialogues were written topic by topic and combined into 10 teaching paths, are not real children's conversations, and reuse some answers within a topic; review.notes lists what still needs human checking.)

The child first asks about day and night, then offers the guess that "the Sun goes to sleep". The reference answer first explains the phenomenon, then corrects the specific misconception, and leaves an optional question to keep thinking about. Don't just learn stock catchphrases like "你真棒" ("Great job!").

Field	Does it enter the model, and where is it used
`id`	Generates sample ids such as `day-night-01:0`, for tracing
`group_id`	Groups a science topic together; all grade bands, paths and rewrites must be in the same split
(containing array)	This conversation is in the `train` array; on load it is tagged `split="train"`, and D2 filters to train only; not written into the prompt
`grade_band`	D3 writes it into the system prompt as "小学学段" (primary-school grade band); it is visible input
`fact_ids`	D3 selects fact cards from this file, passing only id/text/kind/scope
`turns[].user`	The child's current question; earlier questions go into history
`turns[].assistant`	The current answer is the training target; earlier answers are context only
`turns[].behavior`	Lets a human see what behavior this turn demonstrates; not passed to the model
`category/topic/scenario`	Labels for statistics and review; we don't sneak "what the child got wrong" into the input
`provenance`, `review`	Synthesis method and review status, for humans only; all conversations are currently `draft`

If you remember just one thing from this table: the model sees only the system prompt (role + grade band + fact cards), the history and the current user turn; it learns only the current assistant turn. Section 2.5 turns this sentence into real tokens and masks for you to see.

2.4 Expanding a conversation answer by answer¶

day-night-01 has two turns, so it expands into two samples:

Sample	Context	Target that contributes to the loss
`day-night-01:0`	Role + grade band + day-night material + "why is it bright in the day and dark at night"	1st reference answer + EOS
`day-night-01:1`	Same role, grade band and material + turn 1 Q&A + the "Sun goes to sleep" guess	2nd reference answer + EOS

The first sample can't see the second turn. In the second sample, the earlier answer in the history does not contribute to this turn's loss. Of the 500 train conversations, 350 have two turns and 150 have three, which yields 1150 samples. 1150 is neither a token count nor a number of optimizer steps.

Does a two-turn conversation mean "training twice"? It isn't the same content trained twice; the conversation contributes two independent samples, and each is trained once per epoch. After the 1150 samples go into the trainer they are shuffled and the parameters are updated every 16 samples, so day-night-01:0 and day-night-01:1 usually land in different steps. The 1st answer is part of the loss only in the 1st sample; in the 2nd sample it is just history.

The same rule applies when there are more turns, just like the full chat log of a session: sample k treats all of the previous k−1 Q&A turns as history and learns only the k-th answer. This matches how calls work at deployment: the server keeps no state, and the client sends the full history every time (section 5.5). There are two differences you should know about:

During training the history contains the reference answers; at deployment it contains what the model itself said, which may differ from the reference answers. The multi-turn mode in section 6.5 specifically checks this difference.
Later samples are longer, and the history is recomputed in every sample. This project's validation accepts only conversations of 2–4 turns, and anything over 4096 tokens raises an error right away instead of being silently truncated.

2.5 Masks: which tokens the model actually learns¶

Here is day-night-01:0 rendered with this project's tokenizer (this is real output; a few lines of role rules in the middle of the system prompt are omitted). The whole sample is 408 tokens, and only the last 22 contribute to the loss:

mask │ 渲染后的文本
─────┼──────────────────────────────────────────────────────────
  0  │ <|im_start|>system
  0  │ 你是陪小学生探索科学的伙伴。用中文，先回应孩子真正的问题，再按需要解释。
  0  │ ……（角色规则共 10 行）
  0  │ 小学学段：1-2 年级。
  0  │ CURRENT REFERENCE FACTS:
  0  │ [{"id": "day-night", "text": "自转使地球表面……", "kind": "fact",
  0  │   "scope": "昼夜的入门解释；……"}]<|im_end|>
  0  │ <|im_start|>user
  0  │ 为什么白天亮、晚上黑？<|im_end|>
  0  │ <|im_start|>assistant
  1  │ 地球在不停转，我们住的地方转向太阳就是白天，转到背光的一面就是晚上。<|im_end|>

(The header reads "mask │ rendered text". The system prompt begins "You are a companion who explores science with primary-school students. Use Chinese; first respond to the child's real question, then explain as needed.", followed by "(10 lines of role rules in total)" and "Primary-school grade band: grades 1-2.")

The 386 tokens with mask 0 are visible to the model but no loss is computed on them here; only the last 22 tokens with mask 1 are the learning target. The fact card is among those 386 tokens: the model doesn't memorize it, but it has to learn to answer while looking at it, so it must be included during training too (section 2.2, Q2 and Q3).

Split into tokens, the last line is: 地球 在 不停 转 ， 我们 住 的地方 转向 太阳 就是 白天 ， 转 到 背 光 的一面 就是 晚上 。 <|im_end|>. During training the model makes one guess of the "next token" at each of these 22 positions, and how wrong it is becomes the loss; the 386 positions before them only provide context.

Why not let the model learn the system and user parts too? Because after deployment that content is always supplied by the caller; the model only needs to learn "given input like this, how should I answer". If the user turn were included in the loss, the model would spend effort imitating children's questions, and might even learn to make up the next question by itself.

rows keeps case_id/prompt/completion/input_ids/completion_mask so you can inspect them easily. Actual training uses only the last two columns. The notebook prints the full prompt, target and real token length of the first row; don't reuse length numbers from older scenarios.

completion_mask=0 marks context, and =1 marks the current answer plus EOS. TRL sets the labels of context and padding positions to -100, so no loss is computed directly at those positions, but the model can still see the context when predicting the answer.

EOS is <|im_end|>; it appears exactly once per target so the model learns to stop. PAD is different from EOS. Exceeding 4096 tokens raises an error and the sample must be rewritten; the answer is never silently cut off.

2.6 Recap: seven steps from one JSON file to a parameter update¶

The previous five sections covered the data in the order "what the data looks like → how it is split → what the model sees and learns". Strung together, they form the seven steps below. The notebook's code comments mark with D1–D7 which line does each step; follow along during training and you'll know which line is doing which part of this chapter.

Step	What it does	Where in this chapter
D1 Load	Read `dataset.json`, tag each conversation with its set, check fact-card references and cross-set isolation	Sections 2.1, 2.2
D2 Split	Take train only, split each conversation into samples by answer, with history containing only earlier turns	Section 2.4
D3 Build input	Role rules + grade band + fact cards + history + current question, assembled into one list of messages	Sections 2.2, 2.3
D4 Mark the target	Apply the chat template, tokenize, and mark the current answer and EOS as mask=1	Section 2.5
D5 Hand to the trainer	Pass only the two columns `input_ids` and `completion_mask` to TRL	Section 2.5
D6 Double-check	Check the labels the trainer actually generates: context and padding positions are all -100 (no loss)	Section 2.5
D7 Keep a record	Save a copy of this run's data and its fingerprint; only then does `trainer.train()` actually update parameters	Chapter 4

2.7 Changing the data, or switching to your own¶

You don't need to change the data for the exercises; read this section when you want to start editing.

Editing the existing data. To change the phrasing, edit assistant. When adding a new reaction path, adjust the turns before and after it too, so the child never quotes something the model didn't actually say. If you change a fact card, re-check every conversation that references it. After editing, run the structure check first:

python -m lab.data --input data/dataset.json --allow-draft

It should print {'train': 500, 'val': 50, 'test': 50}. It only checks structure: all fields present, fact references exist, no topic crosses sets, at most one question mark per answer. It does not check whether the science is correct.

Making changes take effect. An adapter you have already trained does not change because you edited the JSON. In the notebook, set RUN_DIR to a new directory (for example runs/science-002) and re-run from step 2. Each training run directory automatically keeps a copy of dataset.json as a record of "which data this run used"; don't edit it in day-to-day work.

Switching to your own data. Common open-source SFT data formats such as Alpaca (instruction/output) and ShareGPT (conversations) are essentially the same as this project's: "a list of conversation messages + which one is the expected output", just laid out differently in JSON. To switch data you mainly change two things: organize your data into three sets of conversations, train/val/test; and adjust D3's build_messages as needed to decide what goes into the system prompt.

2.8 Optional: run the data checks on your own computer¶

Everything above can be understood just by reading. If you want to see the tokenization results with your own eyes before renting a GPU, or check your data after editing it, you can install a lightweight environment without PyTorch on your own computer (Mac or Linux, Python 3.11). It takes a few minutes and needs no graphics card:

python3.11 -m venv .venv-data
source .venv-data/bin/activate
python -m pip install -r requirements-data.txt
python -m lab.download --tokenizer-only
python -m lab.data --allow-draft

The last line should print {'train': 500, 'val': 50, 'test': 50}. --tokenizer-only downloads only the tokenizer and config (about 15 MB), not the 8 GB of weights, so it is normal for your local models/qwen3-4b/ to have no weight shards. Transformers may warn None of PyTorch, TensorFlow ... have been found; locally you only use the tokenizer, so you can ignore it. Whenever you open a new terminal later, run source .venv-data/bin/activate first.