How Three AI Failures Made Spectacular Recoveries In 2019
by Amit Jnagal, on December 31, 2019 7:52:21 AM PST
If at first, you don’t succeed...Trust me. You’re not alone. Most AI projects fail to deliver anything close to the spectacular results promised way back at the beginning when allocating all those resources seemed like such a good idea. Cutting-edge projects don’t always turn out as planned.
Someone Has to Pay the Price for Your AI Failure. Let It Be Your Competitors.
An initial failure in AI doesn’t have to be a career killer. It can be a springboard to success. Most AI failures generate a boatload of lessons learned, and you can turn that into an advantage if you make the right moves as a leader. This actually works, and we’ve got proof.
We discovered this not long ago when Infrrd’s leadership team hunkered down at a secret off-site location for several days to reflect on 2019 and do some planning for 2020. Among other things, we asked ourselves: Which of our customers are the most innovative?
We were looking for the ones who pushed us the hardest to solve the biggest challenges and create the most value. When that happens, we both win. Those are the customers we love to work with.
A Surprising Pattern Led Us to the AI Innovators.
So we looked for patterns from many AI deployments, taking into account the scope of the projects and the people who championed them.
We thought our most innovative customers would be the ones with boundless excitement about the promise of AI as a game-changer for their business process automation and insights for decision making. Surely they would be the ones to push the envelope.
The ones who pushed us to solve the hardest problems, the ones who achieved staggering Return on Investment (ROI) after deployment and kept getting better over time, the spectacular overachievers among AI Illuminati were NOT the super-excited teams who screamed “Let’s see what you got!” and dove headfirst into a deep learning project.
Skeptical AI Leaders With Burnt Fingers and Holes in Their Pockets.
Our elite AI Innovators were the skeptical ones with burnt fingers and holes in their pockets. You’d think the ones who failed would be the timidest. Maybe some of them are. But there was no mistaking the fact that our most innovative customers had in common the lessons learned from failure, and they absolutely found ways to exploit that advantage.
If you’ve experienced an AI failure, you too might be on the cusp of spectacular success. Check out these three short stories and tell us if you see a little bit of yourself in any of them. The stories are all true, but the names have been changed to protect the innocent.
Engineering Documents: Bailed Out by a Bake-Off
A multi-billion-dollar manufacturing company was struggling to find a way to automate a process involving the extraction of data from complex engineering documents. The process was taking two weeks to do manually, and a competitor that had found a way to partially automate the same process was beating the pants off them. So they approached a familiar big-name IT outsourcing company that had some AI skills and dove in headfirst. They launched a big AI project aiming to solve the automation problem and neutralize their competitor’s advantage.
After spending millions of dollars, they got nowhere. Worse, they burned a year and a half of precious time and didn’t gain a step on their competition. Epic AI Fail. But instead of throwing in the towel, they launched an innovative AI challenge to solve the keystone problem that was blocking their progress. It was a well-constructed 3-month bake-off that pushed the envelope of what AI could achieve.
Infrrd was one of several AI companies that participated. Yes, we crushed the competition by a pretty wide margin and we’re proud of that. But that never would have happened if an innovative and persistent leader in this company hadn’t gotten everyone to step back, reassess the situation, and create a challenge to try a completely different approach to solving their critical problem.
Fast forward to 2019, and Infrrd is their platform of choice for most things AI and the innovative leader who championed the project is a hero. The epic AI failure has been overshadowed by a spectacular success: The critical process that once took two weeks now takes less than a minute.
This happened because both Infrrd and the customer got real about the problem. Instead of promising the moon, we had realistic conversations about what AI could do and what it could not do reliably. We pushed the envelope with AI, but we also kept ourselves grounded on solving the problem. AI is just a means to that end. Where AI wasn’t yet the best approach, we came up with innovative ways to solve the remaining challenges in a non-AI way for now.
Financial Documents: To NLP and Beyond
A bank was taking five days to process complex financial documents, and they wanted an AI solution to help them reduce that processing time. These documents were, unsurprisingly, loaded with words. So the bank concluded that they needed to solve a natural language problem. They hired a company that specialized in Natural Language Processing (NLP) and kicked off their AI project with great expectations.
After burning a year and some insane amount of money that nobody likes to talk about, they managed to solve only a fraction of the problem and made no meaningful progress toward their automation goals. Epic AI Fail.
But an innovative leader also failed to give up. She realized they were trying to solve the wrong problem. NLP was just one piece of the puzzle needed to understand these complex documents. Before they could extract meaningful insights from their documents, they also had to understand complex tables, graphs, and charts. They needed to find the other pieces and assemble them in just the right order to solve their problem.
Armed with this epiphany, they started looking for a new partner that could bring multiple AI technologies to bear on this complex problem. That is how they found Infrrd. We applied an innovative sequence of computer vision, predictive analytics, OCR, natural language, and machine learning to understand the bank’s documents and provide the insights they needed.
Fast forward to 2019 and the innovative AI leader is a hero. Processing complex financial documents that once took five days to process get done in just 15 minutes today.
Insurance Documents: Impossible? Got you Covered.
A back-office supporting the insurance industry processed millions of documents every month, relying on a small army of manual data entry specialists. This manual process was expensive, time-consuming, and difficult to scale fast enough to accommodate their growth.
Every time they hired a new employee, it took about a year to train them to the point at which they were “fully productive". Each employee would have to learn how to handle dozens of document types that showed up in hundreds of different formats. With thousands of doctype/format variations, it was a challenge for humans to accurately pick out the important data in a document and quickly key it into a computer. This wasn’t an easy job. And to make matters worse, employee churn was high - so training costs soared and productivity dragged.
This company needed an AI solution to automate this process.
Their first attempt was to do this in-house with their own technology team. After spending nine months getting nowhere, they were still stuck on the problem and fully dependent on manual labor. Epic AI Fail.
An innovative AI leader recognized the magnitude of this problem and the competitive implications of solving it (or not solving it!) So they initiated a six-month-long search for an AI partner that finally led them to Infrrd.
Fast forward to 2019 and you can guess the rest. First, they underestimated the problem, then they thought a solution was impossible. Now they’re implementing what they never thought possible until they looked at a different kind of solution to their problem. The innovative AI leader is a hero, and the company is on track to make spectacular improvements in its ability to automate, scale, and accelerate document processing for the insurance industry.
Lessons Learned from Failed AI Survivors
Each of these three innovative leaders embraced the lessons learned from their first bite at the AI apple. They used what they learned to turn early AI failures into spectacular successes that improve their ability to compete. That’s how you become a hero, and that’s how you make the competition pay for your mistakes!
So what are these lessons? Across all of our customers who have innovative AI leadership like the companies in these three examples, we see these recurring themes:
Gain a clear understanding of what AI can do today and what is not yet possible.
Find some real AI experts and have real conversations with them about the problem you want to solve. Don’t get fooled by impostors. There are a lot of companies out there that changed their website extensions from .com to .ai and became AI experts overnight.
Most of these companies picked up some open-source frameworks and started loading them with customers’ data. They gave the customers a good start but failed to deliver a good finish. This is the most prevalent and in a way the most dangerous reason for AI failure because the initial good results can lead you to start throwing a lot of money at the problem before you really have an effective solution.
You can spot these companies by looking for the following:
- Ask them about their proprietary IP or patents that they hold on the AI capabilities you need to solve your problem
- Figure out what is unique about their people, process or technology that enables them to do this task better than anyone else.
- Seriously... Look for signs of a website name change from .com to .ai
Engage AI Specialists, not IT Generalists
You may have a technology partner who has done a great job helping you with (pick your poison) CRM, Web Development, Analytics, Systems Integration, Digital Transformation - whatever. When you hear they now have an AI practice, it’s tempting to just engage the trusted partner who’s proven themselves in a different area.
But AI is a completely different beast, so past results with a generalist don’t do a good job of predicting their future work in AI. There isn’t even much to code in the beginning. A lot of it is data play and how to get enough of the right sets of data you need for training. Even though it is all technical work, the skills, tools, and processes needed to deliver a successful AI implementation are very different than what is used in development work.
You’ll know you’re dealing with a generalist when:
- You realize that for them Harder Problems = More People. Whose business model does that benefit?
- They offer to support your SAP implementation, your Mainframe system, debug your COBOL, and also do AI. It’s like shopping for designer clothing in the same place you buy Craftsman tools. Not a winning strategy.
- Their conversations start with ‘What are the requirements?” instead of “Where is the data to train the models?” Yes, you need to define the finish line, but if they don’t understand what fuels AI it won’t be much of a race.
Solve for the Bigger Problem without anchoring on a single pillar AI solution
This one is particularly difficult to get your head around. Maybe you find a promising company that passes the first two lessons above. It is an AI company but it focuses on one super-specialty around AI. Let’s say it’s automated conversations. This company was born an AI company. They can understand natural language and generate automated conversations. So far, so good.
But when you decide to use them to understand natural language in your contracts, that’s a completely different ballgame. To a natural language company, all that text in your contracts might look like a clear-cut natural language problem. After all, to a hammer company, every problem looks like a nail, right?
NLP will have no clue how to interpret the structure of complex documents. You’re going to need computer vision, predictive analytics, and OCR for that. It might even struggle with the text itself because in conversations and documents the words flow very differently. They need completely different techniques to handle context and correlate information.
So you have to make sure you understand the bigger problem and don’t fall into the trap of focusing on just one pillar of AI, like NLP.
To avoid this:
- Ensure that your AI partner has specific case studies that are related to your problem. Make sure they have more than a hammer in their toolbox and understand how to use multiple tools.
- Talk to customers who have solved the exact same problem. Ask them to share the hidden complexities that weren’t apparent when they started.
- Do a silicon valley styled, “Fail Fast” experiment. Run a proof of concept around your keystone problems to help you fully understand the complexity of the problem you want to solve.
To All Who Failed at AI in 2019...
Look, it takes a strong heart to take risks that have the potential to propel your business forward. If you took a risk and didn’t get the results you wanted, you’re still a hero in my book as long as you keep trying.
Let me tell you a side story from 2019 that reflects this. This year, India launched its moon landing mission and failed in front of the entire world.
If the mission had succeeded, India would have been only the fourth country in the world to have a successful moon landing. (It’s really hard.)
When that failure happened there were two clear camps of critics. The first ones mocked how the country should give up on the whole idea and stick to making movies. Pakistan’s Minister of Science took this jab:
But the other camp praised India for experimenting:
Quite a few people from Pakistan rebuked the minister and made him see that India is at least trying to land on the moon. Those who keep trying eventually succeed. Sometimes spectacularly.
You see the same level of criticism for failed AI implementations too. The hype and noise around AI can lead you to believe that it’s either unpredictable black magic or a simple advancement in technology that most IT companies can help you deploy.
When you venture out to make sense of the world in this noise, you are likely to meet failure. But every failure teaches you how not to try it the next time around. Thomas Edison knew this well.
So… Here's to all the experimenters, the risk-takers, and the brave champions who failed to try to bring AI into their company. You are our heroes and Infrrd will always root for you guys. May 2020 bring you more success through the wisdom to learn from failures and the courage to keep trying.
Happy New Year!