STP vs NTP

By
Amit Jnagal
Founder & CEO

A decade of running startups has taught me one thing about myself - I am wrong a lot of times. I have been so sure about having hired rock star team members who proved me wrong when they did not deliver. More than a few times, I was sure about some investors putting money in my ventures that ultimately did not sign the dotted line. I have personally lost a lot of money on companies that I thought would change the world. Life has taught me that I am wrong more often than I am right.

Around this time last year, I wrote a post about how 2022 was going to be the year of No Touch Processing (NTP) for Infrrd. Fast forward 12 months, and I am delighted to report that on this one occasion, I was not wrong. Yes, 2022 was an amazing year for No Touch Processing. Infrrd delivered on this promise and led a lot of our customers to the land of AI certainty. This was largely thanks to our stupendous Research Team that kept delivering marvels. It was a rocky ride, but we did it!

STP vs. NTP

While we had impossible technology to deliver in order to achieve no-touch processing, our biggest challenge turned out to be something else. There are a lot of solutions in the market that claim to deliver 90% straight through processing (STP). So, a lot of customers believed that they already had what we were pitching to them. But there is a huge difference between NTP and STP… massive! It was a big challenge explaining that to customers.

Allow me to elaborate.


Most solutions work on the concept of a “confidence score” for the data they read. They use this score to tell you that they are 90% sure that they have extracted the data correctly. You test it and see that most of the time when the confidence score is 90%, the system is correct. So you configure a rule that says when the confidence is 90%, go with what the system says. This is called Straight Through Processing. But there are two huge problems with STP:

Problem #1 - Where is my 10%?

You will need to refresh your 8th-grade Algebra books to understand the first problem. When a system tells you that it is 90% sure about the extracted data, it is also saying that there is a 10% chance that it is 100% wrong. Otherwise said, one in 10 documents that have a 90% confidence score could be completely wrong. Imagine if you took an important financial decision based on this data. For example, you paid out an invoice of $90,000 instead of $9,000! Or approved a mortgage loan for $3 million instead of $300,000.

So, customers usually do not make decisions based on STP documents. They still need teams to verify these documents manually. And that is a costly investment.

Problem #2 - STP and Semi/Unstructured Data

Second, STP rates of 90% only work for straightforward documents that fit into a template. As documents grow in variations from hundreds to millions, the 90% confidence rates do not hold. Now, I am not saying that no one provides 90% confidence for semi-structured data, but as documents become more complex, this 90% confidence becomes less reliable. So most customers use it for fixed format documents such as forms, etc.

Problem #3 - Document Level Accuracy vs. Field Level Accuracy

To understand this problem, you will need a piece of paper, a pen, and a scientific calculator. Ready? Here goes - if a document has 10 data points for extraction and each of them is extracted with 90% confidence, then what is the confidence level for the entire document?

If your answer is 90%, then you are wrong. Indulge me for a moment. The probability of two events happening at the same time is the product of their individual productivity. So, if event one has a 90% probability, it must occur with a second event that also has a 90% probability, which then combined probability for both of them happening together is:

0.9 X 0.9 = 0.81 or 81%

So, when you apply the same math to 10 fields in a document extracted with 90% accuracy each, then the result is:

0.9 X 0.9 X 0.9 X 0.9 X 0.9 X 0.9 X 0.9 X 0.9 X 0.9 X 0.9 = 0.3486 or 34.86%

Will you make financial decisions on a document with a 34% probability of being correct?

It’s not likely.

No-Touch Processing

NTP addresses all these problems. NTP ensures that you get 100%, not 90%, not 99% reliable data that does not need human reviews. It works for semi-structured documents as well as unstructured documents. Basically, it is AI’s guarantee that it has done a really good job with the extracted data and that you do not need to look at it. We have put 7 years of fundamental research toward this concept, as well as have filed and granted several patents into making this happen.

We started this year by delivering 18% NTP processing rates for our customers. That meant that every 18 documents out of 100 did not need any review and that our customers have taken financial decisions based on this extraction, without any human review. As I write this, a few of our customers have crossed the 57% NTP rate. In 2023, we will continue to make investments in AI research around reasoning and certainty to take this rate beyond 70% and eventually to our old friend - the beloved 90%.

Not too far out in the future, when you apply for a mortgage online and upload your documents, you will get a loan decision instantly because the AI has perfectly read your documents and there was no need to wait for a human review. Infrrd is on a mission to make that happen!

Happy Holidays and a Very Happy New Year, Everyone!

Frequently asked questions

What does your pricing model look like?

We price based on the annual volume of pages and complexity of document type.  We can get you preliminary pricing once we outlined a solution.  Let's do this.

To know more, book a 15-min session with an IDP expert

How can I try Infrrd before I commit to a full deployment?

Sure.  The first step is to schedule a guided demo where you get to jump into the thick of it.  After you explore our solution you can try a proof of concept. When you're ready, you can deploy the system to one use case.  Then more use cases.  Then across your enterprise.

To know more, book a 15-min session with an IDP expert

How does your system integrate with others in my enterprise?

We play nice.  Our solutions are API-based.  Your documents are feed into the solution using APIs. And extracted data is sent out through APIs.  We use REST APIs.

To know more, book a 15-min session with an IDP expert

Does your solution run in the cloud or on premise?

Our solution is cloud-native but is also design for premise deployments.  Your choice on how you want to deploy it.

To know more, book a 15-min session with an IDP expert

Does Infrrd run on mobile or desktop device?

Glad you asked.  Our data extraction process runs on servers.  We have found performance and accuracy decline when running on a desktop or mobile device. (Remember Infrrd is running a powerful AI stack).

To know more, book a 15-min session with an IDP expert

Does your system work out of the box or does it require training?

Common documents and use cases work out of the box.  The cool thing is your solution will improve as the system learns from your documents upfront and over time.

To know more, book a 15-min session with an IDP expert

How does your solution handle corrections?

Did you know no system is 100% accurate all the time?  When extraction errors occur you want to correct them.  We provide a simple UI that your business analyst will use to make corrections.

To know more, book a 15-min session with an IDP expert

Does your solution work with handwriting?

Our solution excels at data extraction from handwriting.  We've got proprietary methods and techniques that do the trick.  It's pretty cool.  See for yourself.

To know more, book a 15-min session with an IDP expert