OCR that looked like it worked

For months the OCR on this site returned a file. It took a believable four or five Nobody complained, because there was nothing to complain about. A searchable PDF with File "Searchable PDF" output "Text only" output scan-150dpi-5p.pdf 0.0% word recall 100.0% word recall Recognition was perfect. Everything downstream of it was broken. Below are the five Three-hundred-dpi scans failed on page one. Hundred-and-fifty-dpi scans went all the pdf.js renders through a canvas factory, and its default is DOMCanvasFactory, which document.createElement('canvas'). This code runs in a Web Worker, where document does not exist. But the default factory is not reached on every render. It ImageResizer, which engages once a page exceeds MIN_IMAGE_DIM, So the bug was invisible at the resolution anybody would use for a quick test, and the The fix is a canvas factory built on OffscreenCanvas, which a worker does have, With the crash gone, 300 dpi scans completed. They still had no text. The code read the recognised words from result.data.words. In tesseract.js v7 that data.blocks[].paragraphs[].lines[].words[], and What made this survive so long was the shape of the guard around it: if (result.words.length > 0) { // draw the invisible text layer } result.words was undefined, so (result.words || []) gave an empty array, so the Reading from data.blocks instead of data.words did not fix it. Recall stayed at 0%. data.blocks was there in the result object, and its value was null. The reason is in tesseract.js/src/worker-script/constants/defaultOutput.js: blocks You have to ask for it: const result = await worker.recognize(blob, {}, { blocks: true }) This is the one worth the article. A field that is absent tells you that you are on the Two hours went into checking the scan quality, the render resolution, and the language With words finally reaching the page, page.drawText() wrote them into the PDF. Called font, pdf-lib falls back to a standard font with WinAnsi Out of ąćęłńóśźż, exactly one character survived: ó. For Russian, Japanese, Chinese, every single word threw an encoding exception. The exceptions went catch, so the pages came out clean and wordless. Six of the twelve languages in the dropdown could not produce a text layer at all, The fix is a real embedded font, registered through fontkit and subset into the output. The final indignity. The error handler classified any message containing the word read as a file problem, and the message coming out of the broken worker was: Cannot read properties of undefined So a bug in our code told the person using it that their PDF was damaged, and advised Same corpus, same browser, measured on production: File Before After scan-clean-300dpi-3p.pdf crash on page 1 100.0% recall, 2.0s scan-150dpi-5p.pdf 0.0% recall 100.0% recall, 5.0s scan-300dpi-10p.pdf crash on page 1 100.0% recall, 5.0s The page copy went to the bin along with the bugs. /ocr claimed multi-language support Not one of these raised an error. Every one produced a plausible success: a factory that is only reached above a size threshold, so the bug is resolution-dependent and machine-dependent a renamed field read through || [], so a missing structure reads as an empty one an output that is off by default and comes back as null, indistinguishable from "found nothing" exceptions thrown per word into an empty catch, so a total failure looks like a blank page an error classifier matching on a substring, turning an internal bug into a user's fault The practical lesson is not "write more tests", because a unit test would have mocked One more, learned the hard way while fixing this: a late benchmark run from localhost before measuring, The corpus, the harness and the measurements are in the repository, and the current benchmarks page. Worth saying plainly, because it is the same lesson wearing different clothes. Most of That arrangement only survives because of a guard built on the same idea as the harness const invented = quoted.filter((value) => !allowedNumbers.has(value)) if (invented.length > 0) { await reject(`quotes percentages that were never measured: ${invented.join(', ')}%`) } It rejects "up to N" phrasing too, because that is the shape a number takes when it has The reason for the guard is the reason for this whole article. A language model will produce a plausible statistic exactly the way the OCR pipeline produced a plausible PDF: quickly, with no error, and indistinguishable from the real thing right up until somebody checks it against ground truth. Same failure mode, same fix. Measure the output, and refuse to ship what you did not measure.