The Webspinner AI Playground

15.1  Everything so far, in one place

Three findings from earlier lessons have been waiting to be assembled.

Put them together and the conclusion is forced. If the model reconstructs rather than retrieves, and reconstruction under uncertainty produces confident error, then the fix is not a better instruction or a better model. It is to stop asking for reconstruction.

15.2  Grounding your own material

Walk the pipeline in plain language first, and name it only afterwards. The naming is the least useful part and most explanations start there.

1. Documents in. The material you actually have — contracts, handbooks, notes, records. Yours, not the internet's.
2. Chunked. Split into passages small enough that a handful will fit in the context window alongside the question.
3. Embedded. Each chunk is converted into a list of numbers positioning it in a space where passages about similar things sit near each other. This is what makes “find the relevant passage” a geometry problem rather than a keyword problem — it can match a question about termination notice to a passage that says ending the agreement.
4. Retrieved on relevance. When you ask a question, the question is embedded the same way and the nearest chunks are pulled out.
5. Supplied as context. Those chunks go into the window alongside your question. The model now reads rather than recalls.

Now name it: this is retrieval-augmented generation, introduced by Lewis and colleagues in 2020 as a method for knowledge-intensive tasks.1 It is six years old, it is not exotic, and it is the single highest-leverage technique available to a non-specialist.

Notice what has changed in terms of Lesson 6. The task has moved from recall — where plausibility is all the model has to go on — to comprehension, where the answer can be pointed at a passage and checked. That is why the citation back to the source line in the capstone is not decoration. It is the mechanism that makes the whole workflow verifiable.

FIG-15-01

Recall versus grounding

Two paths from the same question to an answer. Top — recall: question goes straight into the model; the answer emerges with a confident tone and a dotted, unresolved line trailing off toward nothing where a source should be. Bottom — grounded: question goes first to a document store, drawn as a shelf; three passages are pulled out and travel into the context window alongside the question; the answer emerges with a solid line running all the way back to a highlighted passage on the shelf. The solid line versus the dotted line is the whole image.

Aspect 16:9 · 3200×1800 px · Brand palette: navy #00040c, silver-blue #a8c0d8, column gold #a87848 · Style: process diagram, no UI chrome

15.3  Where grounding still fails

A course that ended on the fix without the fix's limits would violate its own standard. Four failure modes, all of which you will meet:

• Retrieval failure. If the right passage is not retrieved, the model answers from recall anyway — and usually does not tell you it has. This is the most common failure and the least visible.
• Chunk boundaries. A fact split across two chunks may be retrieved as half a fact. Tables and cross-references suffer most.
• Stale index. The document store reflects what was ingested, not what is current. An answer can be correctly grounded in a superseded policy.
• Misreading. Grounding reduces fabrication; it does not eliminate misinterpretation, over-generalisation, or blending the supplied text with prior training.

Which means the verification habit from Lesson 6 survives into the grounded world unchanged, with one addition: follow the citation. An answer with a source link that nobody clicks is not more verified than an answer without one — it is merely more persuasive, which is worse.

15.4  Governed, portable agents

Lesson 5 identified three ways agents break: compounding error, no natural stopping condition, and unbounded permissions. The Webspinner framing defines agents as immutable, portable, and governed — and each term answers one of those failures directly.

Read that table as a specification you can apply to any agent product, not as a feature list for one. When a vendor offers you an agent, ask: can I pin its version, can I run it somewhere else, and are its permissions and stopping conditions written down where I can see them? Three questions, and they discriminate.

The capstone — one unbroken take

The whole course lands here. Do not cut, and do not retry until the failure in step 5 has been shown at least once across the course.

1. Ask the ungrounded question first. Something specific to a document the viewer could plausibly have — a policy, a contract clause, an internal figure. Show the model answer confidently and wrongly. Do not editorialise; Lesson 6 already supplied the mechanism.
2. Load the document. [PRODUCT: ingestion path and interface — fill from the current build]. Keep this short. The pipeline from §15.2 is the concept; the clicks are not the lesson.
3. Ask the same question again. Identical wording. Show the correct answer.
4. Follow the citation back to the source line. Click it. Show the passage. This is the beat the entire course has been building toward, and it should be given room.
5. Ask something the document does not cover. See whether the system says so or fills the gap from recall. Leave whatever happens in the recording — §15.3 predicts this failure, and showing it is what makes the preceding four steps trustworthy.
6. End on the viewer's next step, not on a product feature.

What comes next

Two things carry forward, and only one of them is a product.

The failure log from Lesson 14 is the ongoing assignment. This course ends; the calibration it was trying to build takes months and is produced by written feedback on tasks you can verify. If you finish these fifteen lessons and start nothing, you have acquired vocabulary rather than skill — and vocabulary is exactly what the illusion of competence feels like.2

And the questions travel. Every lesson here supplied a question rather than an answer, because the answers expire and the questions do not:

• Can I verify this faster than I could produce it? (Lesson 6)
• Which model is right for this task, at this price, this quarter? (Lesson 7)
• Where do my weights live, where does my data go, who can revoke my access? (Lesson 9)
• Do I know the steps in advance — in which case this is a workflow, not an agent? (Lesson 5)
• Have I re-architected anything, or installed a new motor on the old driveshaft? (Lesson 13)
• What would change my mind? (Lesson 12)

[PRODUCT: point to whatever follows this course — subsequent Webspinner Academy courses, community, or Playground onboarding. Fill from the current roadmap.]