In 2023, AI and NLP are no longer considered optional, but crucial technologies for achieving competitive advantage—now. In its trends report on the top 10 emerging technologies of 2022, Forrester identified NLP as one of six technologies that “will create significant ROI starting today.”
Natural language processing (NLP) is the key that unlocks the potential of AI to comprehend and utilize unstructured language data, bridging the automation gap between humans and technology. Together, they help leverage company assets—emails, contracts, customer service interactions, anything that contains text—for new insights that were previously unavailable. Its uptake is being driven by big data, digital transformation, and the rise in human-machine communications. Its use cases stretch across all business operations, from marketing to finance, customer care to sales.
Our recent state-of-the-industry report on NLP found that most—nearly 80%— expect to spend more on NLP projects in the next 12-18 months. Yet, organizations still face barriers to the development and implementation of NLP models. Our data shows that only 1% of current NLP practitioners report encountering no challenges in its adoption, with many having to tackle unexpected hurdles along the way.
Here, we will take a closer look at the top three challenges companies are facing and offer guidance on how to think about them to move forward.
1. High Development Costs
In its 2022 Global AI Adoption Index, IBM reports that natural language capabilities are generally perceived by organizations as expensive to implement. Over half (54%) of IT professionals cited costs as a large or medium barrier to introducing NLP. Our research confirmed these findings, with 38% of respondents citing the financial cost associated with NLP modelling and tools as the largest challenge faced when adopting the technology.
So, how can we address costs? Let’s start by looking at the main cost contributors to NLP development / implementation.
For many companies, the first consideration is often whether to build or buy your own model. Some may see “build” as a cost-saving path, but the technology expertise and integration required, as well as the necessary expertise to bring everything together, can be much more expensive and time-consuming than you bargained for. Also, the solution may not be extendable to your next use case (the one-trick pony scenario), and it may not factor in the costs of integration, hosting, management, deployment, security and monitoring (the slippery slope scenario). Lastly, if you build using large language models (LLM) that rely on data that is not vetted, results may be hard to explain and lead to costly, if not irrecoverable, reputation damage. Building is clearly an option, but one that should be taken with caution and full transparency into the associated hidden costs. For a deeper dive on build versus buy, see our blog post on Why You Don’t Need to Decide Between Build versus Buy Part 1 and Part 2.
Next, the cost to train your model depends on many factors. Do you have enough of the required data to effectively train it (and to re-train to get to the level of accuracy required)? Are you prepared to deal with changes in data and the retraining required to keep your model up to date? Finally, AI and NLP require very specific skills and having this talent in-house is a challenge that can hamstring implementation and adoption efforts (more on this later in the post).
So, rather than “why is this so expensive?” a better question may be “what is the right size of model and training necessary for our needs?”
The “bigger is better” mentality says that larger datasets, more training parameters and greater complexity are what make a better model. “Better” is debatable, but it will certainly be more expensive and require more skilled staff to train and manage.
Luckily, there are options that can do much more with less. Hybrid platforms that combine ML and symbolic AI perform well with smaller data sets and require less technical expertise. This means that you can use the data you have available, avoiding costly training (and retraining) that is necessary with larger models. With NLP platforms, the development, deployment, maintenance and management of the software solution is provided by the platform vendor, and they are designed for extension to multiple use cases.
2. Building the Business Case and ROI for NLP
Building the business case for NLP projects, especially in terms of return on investment, is another major challenge facing would-be users – raised by 37% of North American businesses and 44% of European businesses in our survey.
Traditional calculation of ROI – gains minus cost of investment divided by cost of investment – is challenging to determine due to the fact that the data required to estimate potential variable operating costs is based on as-yet undeveloped NLP solutions.
One approach to overcome this barrier is using a variety of methods to present the case for NLP to stakeholders while employing multiple ROI metrics to track the success of existing models. This can help set more realistic expectations for the likely returns from new projects.
Our data shows that more than half of organizations are using one or more metrics to measure ROI: time to production (54%), efficiency improvements (53%) and cost reduction (53%).
Building the business case doesn’t stop here. There are several other considerations that you’ll want to put in the right perspective.
Can the model be scaled and reused in other areas of the business? For example, a knowledge graph provides the same level of language understanding from one project to the next without any additional training costs. Also, amid concerns of transparency and bias of AI models (not to mention impending regulation), the explainability of your NLP solution is an invaluable aspect of your investment. In fact, 74% of survey respondents said they consider how explainable, energy efficient and unbiased each AI approach is when selecting their solution.
3. Lack of In-House NLP Expertise
Locating NLP expertise isn’t always easy, and 29% of our respondents cite finding employees with the skills to use NLP tools effectively as a hurdle to implementation. So, how can companies address their skills gap?
While having certain technical skills is important, the answer is more about your choice of NLP technology: It’s about having accessible technology. Platforms offering low-code and no-code solutions provide multiple benefits. For one, it provides an opportunity for less skilled employees to be exposed to complex NLP and AI technology. And, for companies interested in building their own solution (in whole or part), it makes it easier for developers to build their own natural language models.
The expert.ai Platform does both. We’re both hybrid and low-code: this level of accessibility means that users can apply both machine learning and symbolic techniques to their models. With end-to-end language tools and intelligent data processing built in to the platform, it’s easy for anyone to train your NLP models out of the box. If you have subject matter experts in-house, our collaboration tools make it easy to lead a team of non-experts with the assistance of the platform (a great tool for training). Otherwise, you still have the low-code and no-code tools at your disposal to simplify onboarding for anyone.
