Artificial intelligence is hard. Anyone who has worked on an AI project can attest to that. It is because of this that companies have not found success with AI at the rate they expect. A number of factors contribute to these struggles, but three stand out above the rest:
It is with these challenges (and more) in mind, that we developed the expert.ai Platform. The Platform is an easy-to-use, hybrid natural language platform that provides a deep understanding of language in all forms, from complex documents (e.g., contracts, emails, reports, etc.) to social media posts, and transforms it into knowledge and insight.
We recently held a webinar to discuss the many facets of the expert.ai Platform and provide a live demonstration on how it can be used to develop more accurate and powerful natural language projects. Though we were unable to answer every question during the webinar, we do not want to leave anyone hanging. With that said, here are the answers to your questions outstanding.
Q: Does your product work with vendor products like AWS, Microsoft or Google?
This question can be answered at two levels. From an infrastructure perspective, the platform runs on either AWS or Microsoft Azure. So, in terms of using a cloud natural language service from Amazon or IBM, the answer is yes.
The expert.ai Platform architecture has an open approach that allows for different services to be used as part of the production workflow. So, in pre- or post-processing activities, you can actually plug in any kind of third-party product. Our philosophy is to keep the platform architecture as open as possible. This not only enables you to use different techniques but the different API services you want.
Q: What machine learning algorithms do you support (e.g., conditional random fields) within the expert.ai Platform?
The expert.ai Platform was built with the belief that no single natural language AI technique is a fit for every project. Teams need the flexibility of a hybrid approach that integrates symbolic and machine learning (ML) AI techniques to achieve the success metrics most valuable to each use case (e.g., explainability, scalability, and accuracy). The machine learning models outlined below are currently available and we are adding more with each release.
For categorization models we support:
- Decision Tree Ensembles: Random Forest; Gradient Boosting; XGradient Boosting
- Support Vector Machines: Linear SVM; Probabilistic SVM; Support Vector Classifier; Stochastic Gradient Descent
- Linear Models: Logistic Regression
- Naïve Bayes Models: Multinomial NB; Complement NB
For entity extraction models we support:
- Conditional Random Fields (CRF)
- Sliding-Window Classifiers: Support Vector Machines
Q: How do you build quality models without an army of human labelers?
This is the million-dollar question. However, when you talk about practical implementation in the enterprise, data availability is actually the biggest problem. Many teams are unable to simply collect the volume of data they need. No amount of people annotating data can make up for insufficient data sets. This shows why merging AI techniques is so important. For example, symbolic AI is a much less demanding approach in terms of data requirements because it uses knowledge-based rules to make decisions. By merging techniques, you can implement a high-quality natural language solution with less data reliance.
The second challenge is that when you have lots of data, it’s not only an army of annotators that you need but an army of expert annotators, especially for complex business problems. For example, when you want to extract data from contracts, you cannot outsource the annotation to someone who has no legal expertise. This forces you to convince your subject matter expert to do the annotation for you. Providing them with easy-to-use, time-saving approaches to annotation makes it easier to incorporate human expertise more efficiently and a lower cost.
Fundamentally, the idea is to leverage hybrid AI to help both scenarios. In doing so, you can decrease the amount of data that you need and, at the same time, decrease the amount of annotation needed.
Q: How is the expert.ai Platform deployed? Is it done as Infrastructure-as-Code or does it require manual provisioning?
The expert.ai Platform is available on Amazon or Microsoft Azure public cloud or as an enterprise private account on their public cloud services. An instance is activated for each customer who wants to use it, or it can also be implemented on their private cloud. We are working to deliver an on-premises offering before the end of 2022.
Q: What is your approach for testing and monitoring in production?
Monitoring happens at two different levels: data performance and technical performance. In terms of data performance, we do live monitoring on the actual performance. For example, if you have a certain category in a classification that never gets clicked, that could indicate that there might be a problem in your model. We provide a dashboard that helps to identify potential data issues. For performance monitoring, we track workflow statistics in terms of “OK” versus “Failed” calls, latency and throughput, and we monitor services within the ELK stack on Elastic Cloud.
Q: How are you, expert.ai, different from what Stanford CoreNLP or TokensRegex offers?
Our proprietary technology offers a much deeper level of understanding, including domain-independent disambiguation of terms, relationship identification among concepts, most relevant concepts and more. This significantly shortens the time required to implement a solution, even if you only use our NL API and not the full expert.ai Platform.
Here are some additional articles highlighting how to get started with the NL API:
Q: Do you build multilingual models that require data of different languages?
It depends on the project. Out of the box, we provide a ready-to-use and easily customizable knowledge graph that includes hundreds of thousands of general- and domain-specific concepts and business terms connected through semantic relationships. Expert.ai’s proprietary technology leverages this embedded knowledge graph (available in multiple languages) together with natural language understanding algorithms to read, comprehend and learn from any text. The knowledge graph is open (not a black box), meaning its content and structure can be understood by humans, making it easy to incrementally adjust to maximize performance in any domain and across languages.
Q: What is the list of domains / industries for which there are specific NLU models?
Knowledge Models leverage out-of-the-box domain knowledge that provides a head start for building custom NLP solutions with the expert.ai Platform. They are NLP rules-based models that contain industry, role or use case-specific concepts and relationships that can be used to quickly improve the accuracy of natural language classification and extract projects. The expert.ai Platform has knowledge models that support:
- Finance: currencies, commodities and macroeconomics
- Environment, Social and Governance (ESG) Sentiment: environment-, social- and governance-related content
- Behavioral Traits: personality traits such as curiosity, honesty or negativity
- Emotional Traits: emotions such as joy, surprise, and anger with emoji support
- PII: personal identifiable information with redaction features
- Life Sciences: biomedical entities, drugs, diseases, and signs or symptoms
Q: In what format are the models published to the PRD environment? Does the PRD environment / server need to be on the same instance as the other environment?
The models operate in a complete, containerized workflow (including pre- and post-processing tasks) that can be invoked through an API. The format used is proprietary (.cpk for symbolic and .mlpk for ML). It is preferable that authoring and runtime are on the same instance, although the runtime module can work also as standalone service.
The expert.ai Platform can unlock so much for you and your organization. See for yourself by watching our recent webinar, “Unlock Natural Language Value: 3 Keys for Success”. And if you have more questions for us, keep asking! After all, knowledge is the key to becoming an expert.
