OCR Accuracy Doesn’t Matter (The Way You Think It Does)

If you’re looking for the “perfect” optical character recognition (OCR) solution, there’s probably one thing you want it to be more than anything else: Accurate. And you’re probably trying to find “the most accurate” option. But OCR accuracy, while obviously important, doesn’t really matter in the way you might think.

The market is filled with OCR vendors boasting about their accuracy levels. That’s because they are all pretty accurate.

And that’s the issue (or, well, non-issue) — depending on the task, the differences in accuracy among the top OCR engines are minimal to non-existent. Accuracy is table stakes. The most important question you should be asking isn’t, “Which OCR software is the most accurate?” It’s, “What am I using this OCR software for?”

What are your requirements?

The use cases for OCR range from automating data entry and managing business documents to enhancing accessibility for the visually impaired.

But you can drill down even further than that. Are you building your own OCR-capable app, or just want to use someone else’s? Do you need to integrate with other third-party tools and services like QuickBooks or a laboratory information management system (LIMS)?

Suppose you want to release a product that helps pharmacists parse doctors’ signatures on prescription order forms. An OCR technology that uses a third-party dashboard with a case study asserting it can read French passports with 98% accuracy probably isn’t that useful to you.

In other words, people tend to conflate “OCR accuracy” with “performance,” but accuracy is just one piece of performance.

This is also where the distinction between OCR vs IDP (Intelligent Document Processing) becomes important – because modern document workflows often go beyond simple text extraction.

See what intelligent document processing looks like in real use cases

Evaluating OCR software

Everyone’s exact business requirements are unique. We’ve never seen the same ones twice. So, what sets apart the OCR solutions that are just “accurate” from the ones that actually help you meet your needs?

Yours vs Theirs

Most OCR products put an emphasis on their proprietary dashboards for document processing and data monitoring. But if you’re embedding OCR into your own app, look for a solution offering comprehensive APIs and UI components you can use inside your infrastructure.

It should be interoperable with other platforms in your data pipeline, too. In fact, interoperability is important whether you’re building your own app or not. Extracted text has to go somewhere. Can they store the data in your cloud? Your CRM? Somewhere else?

When the University of Cincinnati needed to build a custom OCR solution inside Salesforce to digitize student records, they used Onymos DocKnow for its flexibility and pre-built, embeddable UI.

“We were able to develop an application in Salesforce to manage the process more efficiently,” said Josette Riep, Assistant Vice President of Integrated Data, Engineering & Application Services at UC. “Onymos DocKnow was a key component of the system by providing the ability to OCR and conduct an initial tier of validation.”

Handwritten Text Recognition (HTR)

Today, the best computer vision can read 2,000-year-old charred papyrus scrolls… but unless you have a particle accelerator, you’ll have to rely on slightly less powerful alternatives.

Handwriting recognition can be tricky for most commercially available OCR tools. You can get great results under optimal conditions (and with a lot of model training). But how can you consistently ensure “optimal conditions”?

If you need to perform HTR, try to find a solution that can recreate the “optimal conditions” for you, if they aren’t there to begin with. Can your solution crop backgrounds, correct for skew, adjust contrast, or remove “noise”? Can it do all of that automatically? Can you control it manually?

At the height of the COVID-19 pandemic, Albertsons pharmacists used an Onymos-powered mobile app to scan images of vaccine consent forms at crowded pop-up clinics in places like parking lots and school gymnasiums (i.e., not optimal conditions). It saved so much processing time for their clinicians in the field that it was subsequently rolled out to 1,700 stores nationwide.

Data Security and Privacy

A few years ago, ABBYY, one of the market-leading OCR providers, left a server with around 142GB of digitized documents wide open to the Internet. It wasn’t even password-protected. Bob Diachenko, the independent security researcher who found it, said the exposed database contained “contracts, NDAs, memos, letters, and other internal documentation, properly OCR’d and stored.”

SaaS vendors mishandling data isn’t anything new, but supply chain attacks against third-party software providers are rising. Every vendor with your data is a new attack surface for threat actors. Finding a partner who doesn’t need access to your data to actually perform data extraction can keep you and your customers more secure.

Focusing on data security motivated CloudWave to choose Onymos for OCR and intelligent document processing.

“At CloudWave, we emphasize data security in all collaborations. When searching for an enhanced OCR technology for our solution, Onymos DocKnow stood out as the only product allowing our customers to operate it entirely within their own infrastructure,” said Prabhakar Ramakrishnan, CEO of CloudWave.

FAQ

Get instant answers to your other questions about OCR accuracy and performance below, then reach out to the Onymos team. We’ll show you how DocKnow, our AI document processing platform, can help you start automating even your most complex or sensitive workflows. It’s an OCR solution optimized for performance… not just accuracy.

What is “OCR accuracy,” technically speaking, and why is it often misunderstood?

“OCR accuracy” seems simple on the surface. Character-level accuracy, often measured as Character Error Rate (CER), is the most basic and common metric. It’s counting how many individual, extracted characters match the original text. It will also typically factor in missed characters, called “deletions.” Word Error Rate (WER) is similar, but at the word level.

But things get complicated quickly if you need to consider things like layout or formatting, or even the relationship between different data points.

That’s why, however it’s defined, OCR accuracy, by itself, is a synthetic benchmark. It’s not a reflection of how well your automation actually works in production. It tells you how sharp the “eyes” of the system are, but not whether the brain understands what it’s seeing.

If OCR accuracy is no longer the differentiator, why is it still treated like it is?

Because it’s the easiest metric to market and the hardest to verify.

There’s also a legacy problem. Ten years ago, accuracy was the differentiator because OCR technology varied wildly in quality. Vendors competed on who could read text better, period. That mental model stuck, even though the top models today are all clustered within a few percentage points of each other on most tasks.

It’s like how people still argue about megapixels in smartphone cameras. It mattered in 2010, but now it’s a solved problem, and other factors (lens quality, image processing, low-light performance) actually determine whether your photos look good.

But a more uncomfortable truth is that many buyers don’t know what questions to ask beyond “how accurate is it?” So vendors keep answering the question they’re being asked, even if it’s the wrong question.

My documents are pretty standard, like invoices, receipts,and forms. Do I really need to worry about all this?

“Standard” documents are where most OCR vendors make their money because everyone assumes they’re easy.

But your invoices might come from dozens of different vendors with different layouts. Your receipts might be faded, crumpled, and photographed at odd angles. Your forms might have handwritten notes in the margins. The gap between “works on clean test data” and “works on real documents” is massive.

What’s the difference between OCR and IDP (or AI document processing)?

OCR does one thing: it looks at an image and tells you what text it sees. That’s it. You get a string of characters back.

IDP (also called AI document processing) makes OCR “intelligent.” OCR can tell you there’s a number “847.23” on the page. IDP knows that’s the total amount due on line 47 of an invoice, that it matches the sum of the line items above it, and that it should be entered into your accounts payable system as a pending payment.

What capabilities should I prioritise when choosing an OCR/IDP solution?

Performance is holistic. Accuracy, speed, integration capability, security, and usability all matter. Optimizing for just one metric is how you end up with technically impressive solutions that don’t actually solve your problem.

Most importantly, you should deeply understand your own requirements, so you know which tradeoffs you can (potentially) safely make. You may not require on-prem deployment options or an OCR model that reads handwriting.

External sources

Buried by the Ash of Vesuvius, These Scrolls Are Being Read for the First Time in Millennia · ABBYY ‘temporary data breach’ exposed 200,000 scanned docs