Citation extraction and form
Selected case, short-form and reporter patterns
18 public regression examples below; this is not a representative legal-accuracy estimate.
Public report · 2026-10-23
See what the checks cover
Inspect the examples, the results and the limits before relying on a check. We publish the scope of each feature alongside the evidence available for it.
Capability coverage
Authority identity
Available records at a reporter address
Found, ambiguous, not found and unavailable remain separate. Finding a record does not establish current legal treatment.
Quotation location
Supported quotations where a source can be retrieved
A located passage must be read in context. Missing or unavailable sources do not become successful checks.
AI support flags
Selected quoted propositions
Automated assistance that can be wrong. No independent, representative accuracy study is published yet.
Court and filing profiles
Two dated source checklists
Northern District of Texas civil briefs and California civil appeals. Permission to cite and precedential weight require separate human review.
Saved evidence reviews
Version history, human tasks and verified-account acknowledgments
The acknowledgment applies to the recorded text and review stage. Credentials, filing compliance and legal conclusions are not certified.
Bar practice
Original practice, timed sections and Prep Studio
Supplemental material, not licensed NCBE questions. Independent attorney review and exam-outcome validation are not complete.
Citation-form regression report
Original, engineering-authored examples for selected citation-form behaviors. Expectations were specified before running the fixture set. This is a small public regression set, not an independent legal-accuracy study.
Extraction and selected deterministic form checks only. No live identity, quotation-location, AI-support, treatment, publication-status or filing-compliance verification.
- Matched expectations
- 18 / 18
- Missed target findings
- 0
- Unexpected target flags
- 0
- Deliberate abstentions
- 2
The set contains 8 examples expected to trigger a specified rule, 8 negative controls and 2 cases expected to abstain on that rule. Counts apply only to each named target rule. A silent check is not a determination that a citation or proposition is correct.
Independent attorney-adjudicated benchmark: not yet available. No pass rate, professional-competence score or general legal-accuracy percentage is claimed.
Reproduce this report from the project source
Run node scripts/review-benchmark.mjs in the Citation Code checkout. It executes the actual extractor and form checks without a network provider and rejects a stale published report.
Source SHA-256: 231c324616cb911d19c383a4b6b2b07613667511ed5481c71d752ae127e3eece
Inspect every example
year-missing · court-year · Matched expectation
Brown v. Board of Education, 347 U.S. 483.
Expected: finding. Observed: finding. Extracted citations: 1.
All form flags: court-year.
year-present · court-year · Matched expectation
Brown v. Board of Education, 347 U.S. 483 (1954).
Expected: no-finding. Observed: no-finding. Extracted citations: 1.
All form flags: none.
reporter-spacing · reporter-form · Matched expectation
Smith v. Jones, 500 F.Supp.2d 123 (S.D.N.Y. 2007).
Expected: finding. Observed: finding. Extracted citations: 1.
All form flags: reporter-form.
reporter-spaced · reporter-form · Matched expectation
Smith v. Jones, 500 F. Supp. 2d 123 (S.D.N.Y. 2007).
Expected: no-finding. Observed: no-finding. Extracted citations: 1.
All form flags: none.
quote-no-pin · quote-pinpoint · Matched expectation
The opinion says “separate educational facilities are inherently unequal.” Brown v. Board of Education, 347 U.S. 483 (1954).
Expected: finding. Observed: finding. Extracted citations: 1.
All form flags: quote-pinpoint.
quote-with-pin · quote-pinpoint · Matched expectation
The opinion says “separate educational facilities are inherently unequal.” Brown v. Board of Education, 347 U.S. 483, 495 (1954).
Expected: no-finding. Observed: no-finding. Extracted citations: 1.
All form flags: none.
id-without-prior · id-first · Matched expectation
Id. at 495.
Expected: finding. Observed: finding. Extracted citations: 1.
All form flags: id-first.
id-after-prior · id-first · Matched expectation
Brown v. Board of Education, 347 U.S. 483 (1954). Id. at 495.
Expected: no-finding. Observed: no-finding. Extracted citations: 2.
All form flags: none.
id-period · id-form · Matched expectation
Brown v. Board of Education, 347 U.S. 483 (1954). Id at 495.
Expected: finding. Observed: finding. Extracted citations: 2.
All form flags: id-form.
id-period-present · id-form · Matched expectation
Brown v. Board of Education, 347 U.S. 483 (1954). Id. at 495.
Expected: no-finding. Observed: no-finding. Extracted citations: 2.
All form flags: none.
short-without-prior · short-form-first · Matched expectation
Brown, 347 U.S. at 495.
Expected: finding. Observed: finding. Extracted citations: 1.
All form flags: short-form-first.
short-after-prior · short-form-first · Matched expectation
Brown v. Board of Education, 347 U.S. 483 (1954). Brown, 347 U.S. at 495.
Expected: no-finding. Observed: no-finding. Extracted citations: 2.
All form flags: none.
supra-case · supra-case · Matched expectation
Brown v. Board of Education, 347 U.S. 483 (1954). Brown, supra, at 495.
Expected: finding. Observed: finding. Extracted citations: 2.
All form flags: supra-case.
supra-article · supra-case · Matched expectation
Kline, supra note 2, at 16.
Expected: no-finding. Observed: no-finding. Extracted citations: 1.
All form flags: none.
ordinal-court · court-ordinal · Matched expectation
Smith v. Jones, 100 F.3d 10 (2nd Cir. 1996).
Expected: finding. Observed: finding. Extracted citations: 1.
All form flags: court-ordinal.
ordinal-court-correct · court-ordinal · Matched expectation
Smith v. Jones, 100 F.3d 10 (2d Cir. 1996).
Expected: no-finding. Observed: no-finding. Extracted citations: 1.
All form flags: none.
id-intervening-record · id-first · Matched expectation
Record at 42. Id. at 43.
Expected: abstain. Observed: no-finding. Extracted citations: 1.
An intervening numbered source makes the antecedent uncertain; silence is not a resolved reference.
All form flags: none.
short-ambiguous · short-form-first · Matched expectation
Smith v. Jones, 100 F.3d 10 (2d Cir. 1996). Davis v. Evans, 100 F.3d 30 (2d Cir. 1996). 100 F.3d at 35.
Expected: abstain. Observed: no-finding. Extracted citations: 3.
Two same-volume authorities require a clear name; this target check should not assert that no full citation exists.
All form flags: none.