Test set
- 45 invented reports: 31 standard reports (Turkish, English and French; three layouts; as text PDF, scanned PDF and phone photo), 12 deliberately difficult reports and 2 negative cases.
- The difficult ones include a dense two-page report, three collection dates in one table, the same test in serum and urine, a previous-result column, English test names on a Turkish report, urinalysis and a culture result, tests not done or pending, an ink blot over a value, a row cut off at the page edge, category-based reference ranges and thousands separators.
- 481 expected test rows, written down by hand before the run.
- No real patient data was used.
Results (latest full run)
| What was measured | Result |
|---|---|
| Reports that should give results and completed | 43 of 43 |
| Negative cases handled (not a lab report, blank page) | 2 of 2 |
| Expected test rows found (recall) | 474 of 481 (98.5%) |
| Extracted rows that match an expected row (precision) | 478 of 479 |
| Rows with value, unit, range, flag and date all right | 472 of 481 (98.1%) |
| Right values located on their line in the page | 442 of 481 |
| Patient name, birth date and number read correctly | 63 of 63 |
| Report date correct | 39 of 39 |
| Critical errors (see below) | 0 |
| Wrong values kept out of "found" (corruption test) | 34 of 34 |
The reading step uses an AI model whose output can vary between runs; these numbers are one full run of all 45 reports.
What each number means
All five right. Value, unit, reference range, flag and test date each match what is printed. A right number with the wrong unit or date counts as wrong.
Critical errors. A wrong value marked as found on the page (it could be confirmed in bulk without anyone looking), a value presented for a test that was not done or is illegible, a wrong unit, a wrong date, a wrong patient detail, or a negative case that produced results. A wrong value that is marked for review is not critical: it costs a reviewer a look but cannot slip through.
Corruption test. Correctly extracted rows are changed in memory to another test's value from the same page or to a number printed nowhere, and the product's own source check is run again. None of them may be marked as found. This tests the source check only.
What happens when the system is unsure
A value is shown as found only when it is printed on the same line as its test. Anything else waits for a person, and the review panel says whether the location is exact, approximate (the test name only) or the page only. Nothing is marked verified until a person confirms it. Dates that could be read day-first or month-first are settled from the report's own dates, or marked for confirmation.
Known limits
- All results above are on invented reports; real reports vary more.
- Locating values on tilted phone photos is the weakest part; those values wait for a person.
- Long test names may be shortened (for example a method suffix dropped).
On your own reports
Clinics in a pilot get the same measurements on their own documents, field by field. Ask about a pilot.