Recruitment Intelligence
The Top 10 Candidates Scored Below 3/10. Keyword Screening Nearly Missed Them All.
Nearly 380 people applied for a complex regional IT Director role. Traditional keyword ranking buried the strongest candidates. Contextual AI and recruiter-led conversations revealed who was actually equipped for the job.
Built from the field experience of Dr. Ali Raza and the byteSpark.ai recruitment team
The Ranking Gap
The same candidates. Two radically different conclusions.
Every candidate in the eventual top 10
The same top 10 candidates
The comparison is based on the eventual contextual top 10. It does not imply that BM25 and contextual evaluation measure the same thing. That difference is precisely the point.
Nearly 380 people applied for the role. When the eventual top 10 candidates were examined through a popular lexical ranking method, every one of them scored below 3 out of 10. When byteSpark.ai evaluated the same candidates contextually, every one scored 7.5 or higher.
One of those candidates was hired. Two years later, performance feedback remains strong, stakeholder acceptance is high, and the team has recorded no resignations since the appointment. People value having even a dotted-line relationship with the new leader.
This was not a story about a broken search algorithm. It was a story about asking a search algorithm to answer a question that hiring cannot reduce to word overlap.
The Actual Mandate
This was not a normal IT Director role.
On paper, the brief looked familiar: appoint an IT Director with the technical depth and leadership experience to oversee a regional function. The reality was considerably harder.
The outgoing director had served for 25 years. Over that time, he had accumulated trust, loyalty, and informal authority across teams in Saudi Arabia, the UAE, Egypt, and Qatar. Some people followed his lead even when they did not formally report to him.
The replacement would not simply inherit a job title. The successful candidate would need to inherit a network of relationships without assuming that the predecessor's authority transferred automatically.
- 01
Earn authority after a respected predecessor had led for 25 years.
- 02
Build trust across Saudi Arabia, the UAE, Egypt, and Qatar.
- 03
Lead experienced employees, including people older than the incoming director.
- 04
Manage the expectations of internal candidates who had hoped for the promotion.
- 05
Preserve continuity while establishing a credible new direction.
The visible job was IT leadership. The actual job was preserving trust while transferring authority across four countries.
Where Keyword Ranking Failed
BM25 answered the wrong hiring question correctly.
BM25 is a respected information-retrieval method. It ranks documents using signals such as the occurrence of query terms, how rare those terms are, and document length. It remains useful when the task is to find text that resembles a search query.
But CVs do not use a standard language. Two candidates can describe equivalent capability using different terminology. More importantly, leadership readiness, political awareness, cultural fluency, and the ability to inherit a loyal team are not simple term-frequency problems.
BM25 asks: How strongly does this document match these search terms?
The hiring team needed to ask: Who can succeed in this specific environment?
The distinction matters. A low lexical score is not necessarily a low-fit score. When recruitment systems treat keyword retrieval as the final evaluation layer, capable people can disappear before a recruiter has the opportunity to speak with them.
Technical background: Apache Lucene BM25Similarity and The Probabilistic Relevance Framework: BM25 and Beyond.
Academia and R&D
Better recruitment AI starts by modelling the real decision.
byteSpark.ai's evaluation architecture uses seven AI models working in concert. Instead of trusting one score, the architecture examines candidate evidence through multiple lenses, including role criteria, career context, relevance, and comparative strength.
This is where proper research and development matter. Adding an AI interface to an old keyword filter does not change the underlying decision model. The system must be designed around what success in the role actually requires.
The Human Core
Seven AI models were still not enough.
Every surfaced candidate was contacted by a recruiter. The screening call confirmed important CV evidence, tested understanding of the mandate, explored motivation, and gave the candidate space to explain experience that a document could not fully express.
Recruiters could challenge the system's interpretation, confirm what mattered, and identify where a promising profile did not survive human conversation. AI focused attention. A person established whether the evidence held up.
No matter how much AI we apply to recruitment, HR remains a human-to-human discipline.
The Final Three
Credentials created the pool. Context revealed readiness.
Three candidates ultimately demonstrated the leadership profile required for the real environment around the role. Their value was not confined to technical credentials or the density of familiar terminology in their CVs.
Leading through trust, not only formal reporting lines
Managing resistance without creating unnecessary conflict
Navigating regional and cultural differences
Taking over from a deeply respected predecessor
Handling disappointed internal candidates constructively
Balancing continuity with a new leadership direction
Two Years Later
The appointment worked where it mattered.
The selected candidate was hired and remains in the role two years later. Performance feedback has been consistently strong. Stakeholders accepted the new leader, and employees value having even a dotted-line relationship with him.
The team has recorded no resignations during that period. That outcome cannot be attributed to one evaluation score alone. It is, however, meaningful evidence that the hiring process identified someone capable of succeeding in the environment the organisation actually had.
Responsible Recruitment AI
The goal is better human judgment, not fewer humans.
Recruitment technology should help teams see more clearly, direct attention intelligently, and make evidence easier to interrogate. It should not disguise a shallow matching method behind an AI label or remove accountability from the people making the decision.
- 1
Treat keyword retrieval as one signal, not the hiring decision.
- 2
Define the real operating context before evaluating candidates.
- 3
Use multiple models to examine different dimensions of evidence.
- 4
Make the reasoning visible enough for recruiters to question it.
- 5
Validate important claims, motivation, and fit through conversation.
- 6
Keep accountable humans in control of every consequential decision.
- 7
Review performance and retention after the appointment.
The best recruitment technology does not eliminate the recruiter. It helps the recruiter see who an inadequate search method would have missed.
A Difficult Role to Close?
Bring us the mandate conventional recruitment is struggling to solve.
See how byteSpark.ai combines contextual talent intelligence, a seven-model evaluation architecture, and recruiter-led validation to identify candidates traditional workflows can miss.