This is the second of two articles on why knowledge matters in NLP. You can find the first here.
In the first article I described what a deep and wide knowledge graph is. And here’s why this is important and why at expert.ai we chose this approach.
- First of all, a knowledge graph with the above characteristics enables any natural language software to start from a higher level of accuracy and therefore requires much less work to reach the level of performance required to be useful. This is true for any NLP software, including those based purely on machine learning.
- A second advantage is that, in any domain, new concepts or new knowledge requirements are introduced all the time. An NLP software with a knowledge graph like the one I described can maintain the same level of performance by easily and incrementally expanding the knowledge graph because subject matter experts can understand the structure of the repository and easily identify where to add the new concepts. On the contrary, humans cannot understand how a ML-based NLP system works. It’s a black box. The only way to make it work is by retraining it—this means going back to square one every time there is a change in how you want the model to perform.
- It should also be clear at this point that, contrary to what the majority of data scientists claim, ML is not the solution for ALL NLP requirements, and more traditional systems based on engines that perform deep linguistic analysis and rely on a rich knowledge graph offer many practical and pragmatic advantages. The consensus seems to be finally shifting towards re-evaluating this position.
Renewed attention on the knowledge graph will be good for the general objective of having NLP systems handle complexity, and, indirectly, for companies like expert.ai who have invested in this approach from the beginning.
Because the trained model is a black box, as humans, we don’t know what’s inside. Therefore, additional knowledge cannot be easily added, and the only way the system can “learn more” is for it to be retrained (even if only partially in some cases). Instead, when you have an understandable knowledge structure, you can easily identify where the new concept should be added in the graph and therefore keep the level of performance pretty stable with limited cost.
As the accurate comprehension of text content always requires domain-specific knowledge, expert.ai’s knowledge graph can be extended and expanded upon based on the specific requirements of any use case. Unlike other text analytics platforms, expert.ai can add domain-specific knowledge through both subject matter experts and through our proprietary machine learning algorithms that leverage the analysis of a training set of content, both unsupervised and tagged.
Luca Scagliarini
Executive Vice President Strategy and Business Development at expert.ai
https://www.linkedin.com/in/scagliarini/
