Banner Image
avatar icon

Introduction to Machine Learning in Recruitment


Banner Image

Machine learning is one step further towards revolutionizing Talent Acquisition programs. Recruiting the right candidate from a pool of candidates has never been easy. Needless to say, '' bad hires' come at a cost and there is hardly any organization across the globe that has not been hurt by high headcount turnover / low productivity ratio.  

The U.S. Bureau of Labor Statistics has projected that over the decade from 2019 to 2029; labor productivity is expected to increase from 1.1 percent to 1.8 percent. This increase is approximately equivalent to a 0.4 percent annual growth rate comparatively slower than the 2009-2019 annual growth rate. 

Then, how can higher productivity be achieved if the headcount turnover rate remains high?

Obviously, there is a correlation between high productivity, low headcount turnover rate and the digital transformation many businesses are going through. Usually, the ''digital transformation'' is associated with automating those processes that are time-consuming and prone to human error. Though this perception is somewhat correct in many cases, when it comes to recruitment and talent acquisition, the digital transformation is rather to be associated with gathering data and insights and then analyzing the data to empower informed decisions and remove bias as much as possible. Speaking of bias, in the past, many ATS providers have been accused of using algorithms that filter out certain candidates either because of deficiencies in the algorithms or because of designing the algorithms based on very strict ( aka biased) requirements of the end clients. To enhance the level of accuracy, AI-driven ATS providers like Transformify constantly upgrade their algorithms that have been designed to never filter out any job candidate. Instead, the algorithms guide the recruiters during the selection process and help remove unconscious bias.

Why is leveraging AI, machine learning and predictive analytics vital in the recruitment process?

Let's take a look at the following statistical report data excerpt:

Healthcare and social services have added  the most to the recruitment hype in the last few years. 6 out of 10 fastest-growing occupations have seen contributions from the health care sector. 

The decade has seen a decline in women’s participation and a slight increase in men's labor participation.  Besides, the growth in the health care sector, computer occupations like IT security and software development will see fast job growth. This increase is expected due to the increase in internet products. 

Is this data sufficient and can it be used to empower data-driven recruitment decisions?

Very unlikely. Although the report provides some data, it is definitely insufficient and can lead to bias, positive discrimination if out of a sudden women are preferred for certain roles only to address the statement above, etc. 

To use machine learning in recruitment, one needs sufficient, accurate data that has been gathered over a period of time. On top of that, the ML algorithms need to be designed by data scientists and engineers who understand the drivers behind certain data patterns.

Common problems faced in the recruitment industry and their solution

The major question in your mind must be on how to improve recruitment and selection while eliminating bias. But before moving into that you must be aware of the problems related to recruitment and hiring that can potentially be solved by adopting AI in the recruitment process and the available solutions. These are broadly classified into three types:

  • AI-driven ATS ( Applicant Tracking System) - The legacy ATS systems based on natural language processing and resume parsing may filter out candidates who do not use certain language and keywords in their resumes. Some legacy ATS systems filter out candidates whose CVs can not be fully parsed or are in a format that is not supported, others filter out candidates whose experience is in a different industry, etc. To the contrary, a contemporary AI-driven ATS does not filter out candidates. Even more, the cutting-edge  AI-driven ATS of Transformify does not use CV's at all !!! Some of the most innovative companies like Tesla and Accenture also believe that the CV is a thing of the past.

  • AI-driven decision-making process- The number of applicants is increasing at a very fast pace as seen in the U.S. Bureau of Labor reports and it is hard to screen them. How does machine learning in recruitment help? The machine learning algorithm learns from the data that is already available and predicts the likelihood for a candidate to be a good match to the job's requirement, be shortlisted and offered the job. At the same time, a well-designed ML algorithm shall NOT filter out candidates. Instead, it shall help recruiters to make informed data-driven decisions.

  • AI-backed competency scorecards- During the interview process, the recruiters fill in competency scorecards for each candidate. The data is analyzed and compared to data about the candidates that were shortlisted and eventually offered the job, the skills the candidates have listed and the skills and level of proficiency defined by the recruiters during the interviews, the tenure of each candidate with the company once offered the job, etc. Over time, once enough data is gathered, it becomes clear what is the profile of the ''ideal candidate'' for each role within the organization and the added value of diversity of thought.

Machine Learning enabled Hiring and Recruitment trends

Even though AI is still in its infancy and there is a long way to go before any recruitment decisions are fully automated, material progress has been witnessed in the last few years. IBM Watson Talen pioneered AI-driven recruitment software that analyzes big data from various sources to remove bias and keeps collecting and analyzing data about the candidate during their time with the company once offered the job.

A variety of recruitment software providers use data to automate parts of the recruitment process as this approach is easier to address and cheaper to implement. However, there are many limitations associated with the quality and quantity of data used for training the machine learning algorithms. This is especially valid if the data is collected by a third party upon the request of the recruitment software provider. 

As in the past, there have been cases of candidates filtered out by machine learning algorithms either due to deficiencies in the algorithms or due to ambiguous business requirements, there trends towards regulating the usage of AI in recruitment to guarantee the rights of all job seekers and ensure transparency in the recruitment process. The city of New York already advised employers to inform job candidates if they used AI-driven ATS or recruitment software and clearly outline how are these HR tools used. 

Data protection regulations are also relevant to the development and usage of AI-driven ATS as in most cases the data used to train machine learning algorithms is in the scope of GDPR ( General Data Privacy Regulation) in the European Union or similar regulations across the globe. Data protection issues may arise of the job candidates and employees are not informed about the ways their private data is used and can not exercise their rights. 

Although regulations add complexity to the already complex process of developing AI-driven ATS and recruitment software, there is no doubt that a regulatory framework is much needed to protect the rights of all parties involved.