How We Know It Works
Does it do what we say it does?
A tablet that chooses a child's next exercise raises two questions. Does the software do what Fido says it does? And is it safe to put in front of a child? This article answers the first, and the part of the second that software can answer: how we check, as of November 2026, that the software keeps the rules it was designed to keep. It is written for parents, educators, funders, and anyone who might help build it.
The Shape of Fido's Software names the parts that are checked here. Whether what we designed helps children learn is a different question, answered with children rather than with code. The Pilot at an Existing School and How We Recognise Progress tell how we look for that evidence.
As in the whole series, we say what runs today, what is planned, and what is still undecided. That includes what our checks do not yet cover.
Built Right, and the Right Thing
Engineers separate two questions. Verification asks whether the software does what was designed. Validation asks whether what was designed is the right thing: here, whether it helps children learn. A program can pass every check of the first kind and still fail the second. This article is about verification.
For a parent, verification is concrete. A topic should count as mastered only after the child has answered it. A fact learned weeks ago should come back for review when it is due. A lesson that no person has checked yet should stay locked for a school class. Each of these is a promise the design makes. Verification is how we find out, on every change, whether the software still keeps it.
Much of Fido's code is written with AI assistants; How We Build the School tells how we use them. A test written right after the code, by the same hand, tends to confirm what the code does rather than what it should do. So the rule comes first, written from what the behavior should be and never read off the code. And no change joins the software until a person has reviewed it.
The checks come in layers: written rules, tests of each part, replayed learning histories, walks through the screens, checks on the curriculum, and speed limits.
Rules Written Down, Tests That Name Them
The first layer is writing the rules down. Each capability of the software has a spec: a short document that states what must be true now, as a list of numbered requirements. Each requirement comes with at least one worked case, in the form "when this happens, then that follows." As of November 2026 there are 58 specs, with 633 numbered requirements between them. A number is never reused, so a test or a report can point at a requirement for good.
Take one rule. A child sees a lesson only once it has reached their group's release stage. Its worked case: when a reviewer releases a newly drafted fractions lesson to testers, the testers can open it, marked as not yet checked by a person, while the school's children still find it locked. The test that proves this rule names it at its top, and it runs whenever the part of the software it guards changes.
When a spec and the code disagree, the change that finds it must fix one of them. Where the designs behind a spec disagree, the spec lists the question as open instead of hiding it.
A rule written down is not yet a rule tested. A test can sit under a rule and check something next door. So when a change adds or edits a requirement, every test listed under it must name that requirement. Whoever adds the name has to open the test and see that it drives the case the rule describes.
Each Part, and Whole Invented Histories
The second layer tests each part on its own. As of November 2026 the app has about 700 test files, the shared engine about 475, the server about 210, and the explorer about 30. Most of them check one piece with chosen inputs: this answer, this card, this sign-in.
Many problems show only over time. One choice of the next exercise can look fine while ten days of them do not: nothing ever counts as mastered, or a review never comes back. So the third layer replays whole learning histories. A file describes an invented learner: what they already know, how fast they forget, and when and how long they practice. The real mastery engine serves them day by day, the invented learner answers, and every choice the engine makes is written down. As of November 2026 there are 37 such learners across 14 curricula, three of them followed for a year. A change that moves even one choice for one of them shows up as a difference that a person must read and accept on purpose.
Beside these fixed learners, the engine is held to properties: statements that must hold for any learner. No topic counts as mastered without an answer to it. No fact counts as mastered on the day it was first practiced. A fact that is due for review is offered within its grace period. On every change to the engine or the curriculum, the properties are checked on 40 randomly invented learners plus one for each curriculum; each night, on 500. When one fails, the check shrinks the case to the shortest history that still breaks the property.
Whether a week of practice makes sense for a struggling child is a question no property can state. For it, a second reader looks at a replayed history: a separate AI reviewer, which did not write the change, reads it against what the invented learner really knows and what the course promises. Its verdict is advice and stops nothing; a finding that holds up becomes an ordinary fix, reviewed by a person.
The Screens, the Curriculum, and the Clock
Walking the screens. The fourth layer uses the app as its users do. A tool drives the app in a browser, finds every screen it can reach and every control it can press, and records them as a map. Paths through that map are walked automatically, as a child, a parent, a guide, a teacher, or an administrator would walk them. As of November 2026 there are 1,534 paths: 34 written by hand, each naming the person it serves and what they want to do, and 1,500 generated to cover the steps the map records. The walks run on demonstration data in a test copy of the app, never on children's records.
Checking the curriculum. The curriculum is data, and machines check it on every change: every link must lead somewhere, nothing may be left unreachable, and every page must build before it ships. People check it too. Review marks and release stages decide which content children see, as Curriculum as Data, Rendered Anywhere explains.
Timing it. A school tablet is slow, and a frozen screen looks broken to a child. So the software has speed limits, each written down as a number. No single stretch of work may hold up the screen for more than half a second without becoming a tracked problem. The code the app loads at the start must stay under a fixed size. Opening the app and answering a card must each stay within a measured time. These times are taken in a browser on a computer slowed to a quarter of its speed, standing in for the tablet. A limit is raised only by a reviewed change that states the new measurement.
When Something Still Goes Wrong
Checks miss things, and defects have reached our testers, children among them. So the software is built to hear about it. Nearly every screen of the app and the explorer has a Feedback button. A report records by itself where it was made, so a tester only has to say what is wrong. Guides see what the children of their own class reported.
Each report gets an answer. For a mistake in the curriculum, a proposed correction can appear on the tester's own screen, and the tester says whether it fixes the problem; only a person's approval makes it permanent. For a fix in the code, the tester is told once it has shipped and asked whether things are better. No report is closed without a note that says what was done.
A fix alone is not enough. When a defect has reached a person, the fix must come with the rule it broke, written down so that it also holds for cases nobody has thought of. In the mastery engine, that rule is a property checked on every invented learner, wherever it can be stated as one. Elsewhere it is a numbered requirement in a spec, with a test that names it, and the spec's history records which report led to it. The rule is stated for every case of its kind, not only for the one that was reported.
What Is Thin
Much of this runs on every change. Some of it does not, and some of it has not been built.
- Real devices. The automated checks run in a browser on ordinary computers, none yet on the Android tablets children use; the way the app runs the engine there is covered by no automated test. The speed limits, measured on a computer standing in for a tablet, tell us when something gets slower, not how fast it is in a child's hands.
- Real classes. The invented learners are deliberately simple, and their settings were chosen by judgment, not fitted to children. Replaying the real, anonymized records of a pilot through the same checks would be the strongest test; it waits on a decision about how such records may be used. No test yet puts the load of a whole class signing in at once on the server.
- Advisory checks. Some checks report without stopping a change: the walks through the screens, the timing of opening the app, and the AI reviewer's reading of replayed histories. That reviewer runs when someone asks; a weekly schedule is planned. Of the engine's 26 stated properties, 17 must hold: 14 are checked on every change and 3 each night. Another 3 only report, and 6 are written down but not yet checked.
- Rules and their tests. Of the 633 requirements, 58 list no test yet. Where a requirement lists a test, the test names the requirement back in about one case in four (237 of 1,040); the naming rule applies only when a requirement changes. And 66 requirements about how a person moves through the app have no walk that proves them yet.
- No measure of coverage. We do not currently measure what share of the code the tests run.
How a Small Team Builds Fido's Software with AI completes the software part of this series: the AI agents and routines that help us build it.
Questions, objections, or keen to help build it? Write to us.