Opening your learning space
Starting the app …
Opening your learning space
Starting the app …
This internal study tests quote matching in Quanta's Source-First pipeline: 1,997 of 2,042 generated candidates passed. It measures text binding, not factual accuracy.
100% source-backed · 97.8% pass the quote-match · 2,042 cards
Abstract
Across 100 generations (50 exam topics from upper-secondary and university level, 2 runs each) Quanta's source-first pipeline produced 2,042 flashcards. 45 did not pass the deterministic quote-match against their source and were dropped server-side; 1,997 were delivered, a hit rate of 97.8%. All 1,997 delivered cards carry a source evidence quote (100%, by construction). The study measures source fidelity (where a statement comes from), not factual correctness. Every number is recomputable from the open raw dataset. Measured on a German corpus; the method is language-neutral.
The 28 June 2026 measurement used Google Gemini and has not been repeated with the current Mistral model. It measures source binding in that test setup, not factual correctness or current product quality.
As of June 28, 2026. Full computation under "Results" below.
The study measures the same gate used before delivery in the Source-First path. Quanta retrieves real source text, generates from it and keeps the passage attached to the study content.
Confirmed evidence makes a statement traceable. It is not an automatic fact check.
Live search · Alpha-Zerfall
Serlo · „Radioaktivität"
Picked
Wikipedia · „Alphastrahlung"
Wikibooks · Physik
Quanta searches vetted, openly licensed knowledge sources for your topic and picks the best passage.
Definition first, so the number stays honest.
The quote-match hit rate is the share of model-generated, well-formed cards whose evidence quote a deterministic algorithm finds in the source text (exact, normalized or contiguous token match). It is not an accuracy or correctness rate; it measures provenance.
The decisive part is source selection: Quanta generates exclusively from established, vetted, openly licensed educational sources (see below) and makes every card traceable back to one. That reverses the burden of proof: instead of claiming "correct", we show you the vetted source and the exact location for every card. You can verify correctness yourself rather than taking our word for it.
hit rate = backed / (backed + dropped) = 97.8%
backed = 1,997, dropped = 45, generated (well-formed) = 2,042.
German upper-secondary (Abitur) and bachelor level. 50 typical exam/test topics across the subject spectrum, no adversarial inputs.
Per topic, real full text from verified, openly licensed sources (Serlo, Wikibooks, Wikipedia, Wikiversity). Substance and license gate per source.
Cards are produced ONLY from that text (temperature 0, no model knowledge). Quote-first: the verbatim anchor first, then the question and answer around it.
Every card is checked by quote-match against the source (no second AI model): exact substring, punctuation/formula-normalized (>=0.9), token containment (>=0.85). The unbacked is dropped.
Absolute numbers per group. Rate = delivered / generated cards.
| Level | n (gen.) | generated | delivered | dropped | rate |
|---|---|---|---|---|---|
| Abitur (upper-secondary) | 50 | 1011 | 1000 | 11 | 98.9% |
| University (bachelor) | 50 | 1031 | 997 | 34 | 96.7% |
| Subject | n (gen.) | generated | delivered | dropped | rate |
|---|---|---|---|---|---|
| Economics | 8 | 160 | 160 | 0 | 100% |
| Medicine | 6 | 120 | 120 | 0 | 100% |
| Philosophy | 4 | 80 | 80 | 0 | 100% |
| Geography | 2 | 40 | 40 | 0 | 100% |
| Chemistry | 12 | 239 | 238 | 1 | 99.6% |
| History | 8 | 162 | 160 | 2 | 98.8% |
| Physics | 14 | 284 | 280 | 4 | 98.6% |
| Biology | 14 | 284 | 280 | 4 | 98.6% |
| Law | 4 | 82 | 80 | 2 | 97.6% |
| Mathematics | 16 | 336 | 319 | 17 | 94.9% |
| Computer Science | 12 | 255 | 240 | 15 | 94.1% |
Weakest fields: mathematics (94.9%) and computer science (94.1%), formula- and algorithm-heavy material with thinner open German source coverage. No subject below 94%.
How it is computed
hit rate = delivered / generated
Total = 1,997 / 2,042 = 0.9780 = 97.8%
Abitur = 1,000 / 1,011 = 0.9891 = 98.9%
Univ. = 0,997 / 1,031 = 0.9670 = 96.7%
generated = delivered + dropped (well-formed cards). Non-parseable model outputs in the denominator: measured 0.
Share of backed cards per subject, descending. Honest 0–100% scale (not truncated); amber marks fields under 96%.
The score distribution shows: almost all drops were clearly unbacked, barely any borderline cases.
Straight from the dataset, including a dropped card. Fronts/backs are the model's German output (English gloss added); the evidence quote is verbatim from the German source.
Was ist der technologische Kern der dritten industriellen Revolution?
What is the technological core of the third industrial revolution?
Die mikroelektronische Revolution seit Mitte der 1970er Jahre wird als technologischer Kern einer neuen, dritten industriellen Revolution angesehen.
Evidence quote, found verbatim in the German source
"Die mikroelektronische Revolution seit Mitte der 1970er Jahre wird als technologischer Kern einer neuen, dritten industriellen Revolution angesehen"
Was versteht man unter Selektion?
What is meant by selection?
Unter Selektion versteht man Prozesse, die den Genpool verändern und bewirken, dass sich Arten umbilden.
Evidence quote, found verbatim in the German source
"Unter Selektion versteht man Prozesse, die den Genpool verändern und bewirken, dass sich Arten umbilden."
Source: Serlo · "Selektion" (CC-BY-SA)
Was ist die Form einer linearen Funktion?
What is the form of a linear function?
Claimed evidence, not found contiguously in the source
"Eine lineare Funktion hat die Form $f(x)=m\cdot x+b$ . Ihr Graph ist eine Gerade ."
Dropped (score 0.73 < 0.85): the multi-sentence quote fell just below the threshold because of the spaces before the periods. The system drops when in doubt rather than deliver something uncertain (stricter, not laxer).
Search the German measurement corpus by topic, subject, level and verdict. Every delivered card includes its evidence quote and source; dropped candidates include their rejection reason.
The report is translated; the measured cards remain German because this release used a German-language corpus.
Generation and verification use the same retrieved full text as both the basis and the match corpus (no circular AI-grades-AI setup). The quote-match is pure, deterministic string work; the full parameters are listed above under "Parameters & assumptions".
How to recompute it yourself: download the raw dataset, and for each entry count the cards in ausgeliefert (delivered) and verworfen (dropped). delivered / (delivered + dropped) gives the rate, per topic, summed per subject, per level and overall. Every delivered card carries its evidence quote and sourceUrl, every dropped card its score and reason.
Quanta only uses sources that are editorially or community-reviewed, versioned/citation-required and openly licensed. That is exactly what makes the reverse argument hold: a vetted source plus per-card traceability.
| Source | License | Why reliable |
|---|---|---|
| Serlo | CC-BY-SA 4.0 | Non-profit learning platform (Serlo Education e.V.), editorially curated by teachers and subject authors, focused on German STEM didactics. |
| Wikibooks (DE) | CC-BY-SA 4.0 | Open Wikimedia textbooks, community-reviewed with full version history and a sourcing requirement. |
| Wikipedia (DE) | CC-BY-SA 4.0 | Citation-required encyclopedia with a flagged-revisions system and complete public version control. |
| Wikiversity (DE) | CC-BY-SA 4.0 | Wikimedia teaching and course material, open and versioned. |
| Source (host) | Citations |
|---|---|
| de.wikipedia.org | 800 |
| de.serlo.org | 605 |
| de.wikibooks.org | 545 |
| de.wikiversity.org | 47 |
Across a realistic cross-section of upper-secondary and university material (50 topics, 2 runs, 2,042 cards), Quanta's source-first architecture delivers checkable study material: 97.8% of generated cards pass the deterministic source check, and 100% of delivered cards carry a source citation. The result is stable across both runs and all eleven subjects, none below 94%.
Not "trust us, it is correct", but "here is the vetted source and the location". Correctness is made checkable rather than asserted.
Lower rates (mathematics, computer science) come from formula- and algorithm-heavy material with thin open German sources. They show up as fewer cards, never as unbacked ones. The gate holds.
44 of 45 dropped cards sit at score 0, not just under the threshold. The system drops stricter rather than laxer.
Outlook: next are a human-labeled validation subsample (matcher precision), an English-corpus run, broader and deliberately adversarial topics, better sources for formula-heavy STEM, and a per-card match score in the dataset. The number reported here measures source fidelity, not factual correctness. It is the most honest figure we can prove against the open dataset.
Source-First cards show their evidence status; without a suitable source no flashcards are created. Eligible users can try Essential for 7 days.
Quanta Study editorial team · June 28, 2026