We previously outlined six key steps to follow to kick-off a successful AI project. Given its importance, we wanted to explore one of these in a bit more detail: getting the right skills in place.
In most situations, AI doesn’t live in isolation. It generally forms part of a wider software application or ecosystem, or requires supporting software and infrastructure to feed data to it and consume its outputs. This means that as well as data scientists or AI experts, you need data engineers, software developers and DevOps specialists. And in addition to these technical and analytical skills, your project will also require expertise to navigate the many legal and ethical complexities associated with AI.
The increased need for cross-departmental collaboration
Organisations who have developed data systems in the past will be familiar with the cross-departmental collaboration required to make them a success. Product, development, marketing and other functions will work closely together. AI increases this need for collaboration across numerous parts of the business, and it’s therefore more important than ever to have C-level buy-in for what you’re doing.
Let’s look at each of the key roles.
Preparing the data: Data engineers
Successful AI projects are built on timely, high-quality data. Responsibility for sourcing and preparing this data often falls on the data scientists. However, this reduces the amount of time they’ll spend on the work where they can add the most value (more on this below).
Instead, data engineers should be responsible for building the pipelines that get the right data in the right place at the right time, in line with the organisation’s wider data strategy.
This can involve exploring data sources, validating, transforming and organising information, and labelling the often very large quantities of data that will be used to train the algorithm. Labelling the training data is often a particularly time-intensive and manual process, and offshoring this aspect can be a quick and cost-effective solution.
The data engineers consequently play an absolutely critical role early in the process, by putting in place the essential foundations.
Building the AI algorithm: Data scientists/AI engineers
The data scientist or AI engineer is a multi-faceted role, requiring an understanding of the business opportunity or challenge, expertise in statistical analysis, deep knowledge of the various AI techniques and models, and programming skills.
Understanding the business issue is essential if the AI solution is to provide real value to the organisation.
Statistics are key because you need to be able to understand when and how to apply different analytical techniques, in ways that often won’t be obvious. A good statistician will ensure you don’t fall into one of the many pitfalls, such as handling data in unrepresentative ways, or testing your model using data you trained it with (which would yield excellent test results but potentially poor results when used with other data in production).
The knowledge of the different AI models and techniques ensures you pursue the right avenues, though there will always be a degree of experimentation involved to identify which works best in a given situation.
And programming skills are also essential: even though the AI engineer generally isn’t building the final, production-ready system, creating and refining the AI model still involves coding.
Linking the AI model to other systems: Software developers
While the data scientist or AI engineer will be writing a lot of code as part of their job, they won’t generally have the knowledge of how to build production-grade software. This is where the developers come in, to connect the AI model to the rest of the ecosystem in a way that’s robust, secure and meets user and business needs.
This can involve enhancing the code from the AI engineers, creating the necessary APIs to accept input data and then deliver outputs to other systems, and putting in place the platforms to support a production-grade system.
There’s a significant DevOps aspect to the role, to take care of infrastructure provisioning and the continual releases that will be essential as your data scientists refine the AI model. Depending on your software developers’ skillsets, the DevOps side of things may be handled by a dedicated individual or team.
Navigating the ethics and legalities
Any project handling potentially sensitive data needs to be aware of what’s allowed, both legally and ethically. Bringing AI into the mix adds a further layer of complexity. For example, could using AI with a given set of training data result in unintended consequences, as is
This is why it’s extra-important for your team to be aware of the potential issues, and to have support and oversight from your legal and ethical specialists. This will help ensure you use AI in line with the law and best practices.
Identify the gaps in your talent
With AI being a relatively new discipline, the pool of people with the requisite skills is limited. Moreover, demand for these individuals is high, given the clamour from pretty much every sector to start harnessing AI.
If you’ve identified a use case for AI and are keen to get started, assess which skills you have already and where any gaps are. Once you have an organisation-wide picture of your AI skills, decide whether to recruit some or all of the missing roles yourself, or to draw on the resources of a partner organisation.
The latter is a cost-effective way of getting proofs of concept up and running quickly and validating the effectiveness of AI. Doing so will help you establish whether there’s a case for in-house recruitment, while giving you time to bring in your own people, if you choose to.
Building out your AI capabilities
We hope this gives you an idea of the unique and varied blend of skills you need to make an AI project a success. In our next piece, we’ll take a look at how to progress beyond your early AI initiatives, and transfer the knowledge and experience you gain into other parts of your organisation.
