What does customer obsession have to do with AI? According to a recent study from Forrester Research, companies that put customer obsession at the center of their strategy and operations, outpace the competition in both revenue growth and profitability. The secret? They’re insight driven. This is where AI and natural language processing (NLP) come into play. It’s about finding insights in ALL of your data.
In a recent webinar, we spoke with Principal Analyst Boris Evelson from Forrester Research to talk about the connection between successful, customer-obsessed companies and the technologies that they’re relying on to solve complex business challenges, namely NLP and text analytics.
AI and the Challenge of Enterprise Data
The enterprise runs on data. Invoices, reviews, contracts and legal documents, purchase orders are common to most every business, as are reports and presentations, product data sheets, and any other enterprise data. This data is “organized” in a combination of written text and numbers, forms, tables and more that can be pages—hundreds of pages—in length. This is all standard data that organizations have been working with for years.
But today, there are new sources of information that offer streams of data coming in real time. Live customer service—made possible through chat channels that offer real-time communication—are simply a necessity for companies in most industries, and social media and news are valuable sources of breaking trends, events happening in real time that companies cannot afford to ignore.
In fact, the formats mentioned above are all unstructured data and organizations need to be able to access this information in addition to all the structured data in spreadsheets and applications that are neatly matrixed for analysis throughout an enterprise.
The common denominator in the unstructured examples we shared above is that they are dominated by text, and what is text but language, what we like to call language data. Not surprisingly, this unstructured data constitutes more than 80% of all enterprise data.
In today’s enterprise world, you cannot be insight-driven and only focus on the 20% of enterprise data stored neatly in rows and columns. In fact, you must be able to consider data residing within, and related to, your whole organization—with a 360-degree view—all of your enterprise data.
So, how do you to go from data to insight?
According to Forrester, being insight driven is more than just collecting the data and using the insights that you discover from your data, but using these insights to make decisions and to learn from them in order to improve continuously. This is where NLP comes into the picture.
NLP and Text Analytics to the Rescue
When we talk about artificial intelligence, it’s more than just about the machine learning aspects. This difference is clear if we look at how different forms of AI deal with documents, for example.
Knowledge-based AI—what we refer to as symbolic AI—is based on what humans know, and for applications like text analytics it uses linguistic rules that humans write to analyze sentences and entire documents to achieve desired outputs, such as identifying entities, topics, numbers, or sentiment in data. As you can imagine, you can do a lot with this information!
Machine learning (ML) based AI turns this inside out. Here, we don’t write rules but provide the algorithm with the inputs and the algorithm learns from this to provide outputs. The ease of access, readiness for use and low cost of some options makes an ML the approach of choice for some use cases. But because this learning is based on matching and pattern recognition and not an understanding of the text that it’s seeing, you need a large volume of documents to train it and time to make sure it’s accurate enough to meet your business requirements.
Thus, a single approach rarely solves the complex language problems enterprises face. That’s why hybrid AI—the combination of ML and symbolic AI—is recognized by Forrester as a “best of both worlds” approach for natural language AI applications. A hybrid approach allows you to take a range of tools, from knowledge-based, ML and even large language models like GPT-3, and use the right ones for your business requirements.
You Can Achieve ROI with NLP Now
NLP is not an emerging technology but a here-and-now technology that is delivering ROI for companies today.
“NLP benefits are almost immediate. According to Forrester’s 2021 data, 70% of data and analytics decision-makers whose firm is adopting AI say they expect their firm to use natural language technologies in 2021.” – Brian Hopkins, VP Emerging Tech Portfolio, Forrester Research
Not surprisingly, multiple studies confirm that organizations are increasing their NLP investments in the next year. In its 2022 Data and Analytics Survey, Forrester found that 71% of decision makers expect to increase spending on NLP and text analytics technologies. An expert.ai commissioned AI Journal survey of NLP practitioners in business and technical roles across North America and Europe found that more than 70% of organizations surveyed expect to spend more on NLP projects in the next 12-18 months.
So, how can companies stay ahead of the competition and start taking advantage of these technologies today?
1. Understand the inner workings of text analytics platforms
Start by having a deep understanding of the technologies and/or platforms you’re evaluating to make sure that you know how it works, the components it offers. Which AI technique does it use? Is it a black box or is it explainable? Think about the lifetime of a platform’s use, such as the implications (including costs) for when the data changes and the ease of adaptability for future capabilities and additional use cases.
2. Recognize that only few platforms can handle all your enterprise text analytics requirements
Organizations have many different kinds of data and the way you will use it depends greatly on your business. This is why understanding your requirements and the functionality you will need is so important. According to Evelson, there are few platforms that do it all, which is why it’s worth considering all of the use cases you will need to address before you buy.
3. Match the capabilities to the use case
As we mentioned above, there are many, many different use cases for NLP and it’s important to match the capabilities of any platform or technologies you’re evaluating with the use cases that are or will be important for your business.
Text analytics use cases fall into two different categories. People-oriented text analytics are the high-volume, high-velocity unstructured text that are typical of use cases like customer experience and employee experience, contact center conversation intelligence, market intelligence and social media. Here, the functionalities that you might look for would include AI in many forms (knowledge based, ML based as well as pre-trained models), multilingual capabilities, natural language understanding, etc.
Document-oriented text analytics use cases like intelligent document extraction and processing (IDEP), data protection and info governance, knowledge management and those that are domain-specific (discovery, contract analytics) involve unstructured and semi-structured text with greater complexity. Think about the complexity of lengthy contracts, legal documents or insurance claims containing forms and lots of text, or scientific research or in-depth reports that contain lots of technical language as well as charts and tables. In addition to the above capabilities, you would also want industry specific knowledge models, image-to-text, form/table recognition and process automation functionalities.
4. Keep humans in the loop (HITL) to increase accuracy and adoption
One of the biggest mistakes that people make in AI is not accounting or budgeting for humans in the loop. Regardless of the AI approach, human expertise will always be necessary to increase accuracy. Accuracy drift occurs in all AI systems, especially as the data changes and evolves with your business, your products, your customers, the market. Few engines will be accurate out of the box, and they will always need humans to help them learn, tune, continuously monitor and to add corrective actions. Choose a text analytics approach that keeps humans in the loop.
5. Today, hybrid AI is likely your best approach for text analytics
Hybrid AI offers the best of both worlds when it comes to text analytics. As we mentioned above, knowledge-based AI based on linguistic rules is more accurate out of the box and it does not require labeling and training for data, saving considerable time, effort and resources. When combined with machine learning or other AI approaches, this allows you to solve even your most complex language challenges in a single environment.
“Bottom line: If you’d like to have the cake and eat it too — have a future fit, adaptable NLP solution, that is accurate out of the box, only requires moderate support, and can continuously learn and self-improve — look for an NLP solution based on hybrid AI.” – Boris Evelson, VP and Principal Analyst, Forrester Research
6. Buy text analytics solutions rather than build them
Finally, Evelson counsels that, unless you have a really special use case and a lot of in-house expertise, it’s better to buy than build.
The end-to-end processes that text analytics requires are extensive and it should not be a black box because you need to understand what happens when and how. You need connectors and more importantly, scripts, to clean up documents and text depending to avoid duplicating information over and over (email chains are a great example of this), you may need to transcribe audio to text or pull information out of images, and then you will need to enrich and integrate data… and this is just the beginning of what’s really happening with NLP in a text analytics scenario. So, do you want to spend time and effort building all the connectors and scripts you will need or do you want to focus your efforts on higher value activities like fine tuning and addressing more use cases in your business?
We go much deeper into the Build vs. Buy discussion over on the blog, and we created a guide to help navigate the decision-making process.
Finally, you can listen to the full discussion with Forrester on the on-demand webinar, 6 Tips for Text Analytics Success with NLP.
