by Amit Jnagal, on December 21, 2021 9:56:41 AM PST
2021 is slowly coming to an end. This was a big year for IDP platforms. They gained a lot of traction and made inroads into a lot of operations teams, improving their efficiency and saving costs along the way. Analyst recognition of this segment broadened in 2021 with Gartner, ISG, Quadrant, and many other analysts joining Everest and Zinnov to cover this space. While 2020 was the year of early adopters, in 2021 a lot more customers started to notice the possibility that this technology presents and actively started pilots for wider adoption across enterprises.
Over the last few years, customers were cautiously adopting IDP, skeptical of the efficiency gains that can be achieved by this technology. Now that a lot of them have seen what is possible, they are asking questions about what else can be done and also pushing the boundaries of what is possible. This year, Straight through processing showed up in a lot of our customer conversations. Customers started asking about what it would take for the IDP solutions to handle documents completely independently of human intervention.
What is the big deal with no-touch?
No-touch processing is the holy grail of IDP and AI in general. The more AI can rely on itself rather than an external input, the more scalable it becomes. We have posted in detail about the case for low-touch mortgage processing in one of our past posts. Look, algorithms do not sleep and do not need to take a break. They can process data 24 hours a day. The big obstacle that they need to manage is the human intervention they need when they are not sure about what they have read. If there is a way to bypass that intervention, the efficiencies will skyrocket.
No easy path
It’s time for a short riddle. You will need your probability and math skills to solve this one. Ready? Here goes - let’s say there is a document that you need to extract 5 fields from. Your highly accurate algorithms can extract each of these 5 fields with 90% accuracy. If any of these fields is wrong, then someone has to manually look into the document to see what the real value should be. Let’s call this number document-level accuracy. The question is, if each of your 5 fields is extracted with 90% accuracy, what is the document-level accuracy that you can achieve?
If you said 90%, you are wrong and not alone.
Most people make that mistake. It is easy to assume that if you get 90% accuracy on all fields, your overall accuracy should be 90%. But it does not work like that. When you get 90% accuracy, your algorithms are wrong 10% of the time. If you had 5 fields extracted, each of the 5 fields would be wrong 10% of the time, each. Even if 4 of these 5 fields are correct and only one is wrong, your document level accuracy is zero.
So, if you have 5 fields extracted with 10% error, your maximum error compounds to 5 times 10% = 50%. If you have 10 fields extracted at 90% accuracy, your overall document-level accuracy could be 0%.
For IDP systems to produce Straight Through Processing, they have to be correct for all the fields, all the time. This is mathematically and technically very hard to achieve. That is why this problem has not been solved yet to this extent.
2022 looks very promising
There are three key factors that look very promising for dramatic progress in no-touch document processing in the new year. These chess pieces are all lined up for the end game to begin:1. Proven Capability
In the years leading up to 2022, IDP platforms have shown what they are capable of and have surprised many customers and industry pundits alike. Especially for customers who had tried OCR solutions in the past and not seen the results that they were hoping to see, the last couple of years have delivered a lot of promise in efficiency and cost-cutting. The question about this technology holding water has finally been answered. This has emboldened both the customers and the IDP players to aspire to bigger goals.
2. Advancement in Machine Learning
This was also the year that massive advancements were made in the field of machine learning. Algorithms' capabilities to see, read, and understand data have scaled new limits. AI algorithms started writing poems and articles this year. This one seems to be the Queen of all of the chess pieces. A lot of what looked like a scene from a Sci-Fi movie found its way into real-life this year. All these advancements have brought no-touch processing in closer reach of IDP platforms.
3. Incentives
Innovative customers have started incentivizing IDP deals based on outcomes, rewarding higher risk of automation with higher reward. This works both ways - the customers save on their operational costs and pass on a share of that to the IDP vendors. As a result, product engineering teams are pushing harder than ever to race to be the first ones to capture no-touch document processing.
All in all, 2022 seems to be a really promising year for no-touch document processing. Our research and engineering teams are very excited about what they will put in front of our customers in the new year. We hope to set even higher standards for no-touch processing and accuracy.
