Text analysis has become essential to the modern enterprise. It enables common internal processes such as search and document classification and external applications such as chatbots. However, there is a common misunderstanding of the artificial intelligence approaches that make textual analysis possible — and how they impact explainability. So what are the core differences between machine learning/deep learning and a semantic analysis-based approach?
Machine Learning/Deep Learning vs. Semantic Analysis
The fundamental difference between ML/DL and a semantic analysis-based approach is the level of understanding each approach provide. Using the classic machine learning and deep learning algorithms, text is not considered to have structure and meaning. Instead, text is simply a sequence of symbols (keywords) that occur together with a certain frequency.
In essence, these algorithms recognize the most statistically frequent and relevant patterns but do not “understand” anything about the text. As a result, a text that doesn’t make sense or that is syntactically incorrect is considered identical to one that is written correctly.
Instead, a semantic approach analyzes text similarly to a human by imitating some of the cognitive processes we instinctively use to understand the meaning of a text. To do this, the software must have a rich and deep knowledge of the world and language (usually stored within a knowledge graph) and use an algorithm written specifically for understanding text.
It is a more specific and complex approach — requiring more initial investment — but it is the only one that can go beyond simply counting word sequences to understand structure, relationships and meaning as a human does. Because of this, semantic analysis can be trusted and easily understood by humans. That’s called “Explainable AI” (as opposed to black-box AI).
How Explainable AI Impacts Your Organization
Explainable AI helps you to understand how software arrives at a decision and why it provides specific outputs by making clear the steps it takes to get there. Given the persistent challenges companies face with respect to bias, accuracy and more, understanding the inner workings of your AI system needs to be a top priority.
How important is human activity for algorithm training?
Simple: it’s fundamental. Without human intelligence (and knowledge), an algorithm cannot work, except in the most trivial cases. Even the most sophisticated artificial intelligence software cannot operate without the help of experts who have special technical knowledge. Software that can program itself, learn complex things by itself and maintain itself does not yet exist.
How reliable are machines when analyzing complex texts?
Given the complexity and breadth of the themes that could be covered, there is not a single answer. In general, machines cannot yet reach the level of reliability of people. For the most complex problems, I don’t expect this to be achieved anytime soon.
On the other hand, people are not perfect and are bound to make mistakes due to fatigue, distraction or lack of knowledge. So while software cannot completely replace manual analysis of text, it can complement it. Even under the best manual conditions, software can reduce the work of human analysis by 90-95%. And, for a variety of scenarios, the average reduction is around 30-40% with an opportunity for significant growth over the next four to five years.
Where has semantic technology made the biggest contribution?
Semantic technology is the only technology capable of addressing problems that require an understanding (or even a partial understanding) of the content of a text, whether in a short email or a 50-page report. It can understand the meaning of words and phrases, identify relationships between concepts and/or entities and make inferences from elements extracted from a text.
In principle, the non-trivial problems of understanding text can only be solved with semantic technology. This does not mean that all problems of this type can be solved, but rather that other technologies stop at a lower level of complexity.
What many people fail to realize is that machine learning fails to impart any semblance of human knowledge or common sense into an AI model. While this may suffice for some organizations, those that truly seek to maximize the value of their unstructured data need more. They need AI they can trust. They need explainable AI.
