When the algorithm selects the candidate: the obligations of the HR operator under the high-risk systems regime of the AI Act
A candidate applies for a position. They upload a CV, answer a few questions on a platform, perhaps even sit a video interview. A few days later, they receive a standard message: “Thank you for your interest, but your profile does not match our current criteria.”
The awkward question is a different one: who actually decided? The recruiter, a department of the employer itself, or the AI system that analysed the CV, assigned a score, ranked the candidates and passed on only the top ten selected profiles?
In recruitment, Regulation 2024/1689 on artificial intelligence (hereinafter the “AI Act” or the “Regulation”) does not prohibit the use of algorithms, but it radically changes the standard of legal comfort. A tool that filters applications, assesses candidates, builds compatibility scores or recommends a shortlist can no longer be treated as mere administrative software; under the logic of the AI Act, it falls within the category of high-risk systems listed in Annex III.
Why recruitment is a sensitive area
Access to work is not a simple commercial interaction. For the candidate, the outcome can mean access to income, professional status, social mobility and, in certain cases, the effective right to compete for a position. This is why the AI Act treats employment and worker management as fields in which algorithmic errors can affect fundamental rights.
Not every application used by the HR department is, by its mere existence, a high-risk AI system: an interview calendar, a passive database or an online form do not automatically become a “high-risk”. The difference arises where the system is intended to generate predictions, recommendations, scores, rankings or other outputs that influence the way in which a person is selected or rejected. The AI Act defines an AI system by reference to its capacity to generate outputs such as predictions, recommendations or decisions that can influence physical or virtual environments.
The relevant test is not the commercial label attached by the provider (“smart recruitment”, „talent intelligence”, „CV ranking”, „candidate matching”), but the intended purpose and the actual effect of the system within the recruitment process. Article 6 of the AI Act classifies as high-risk the systems set out in Annex III, and systems intended for recruitment and selection are expressly listed in that annex.
The HR operator: between the GDPR and the AI Act
In GDPR terms, the employer or recruitment agency that determines the purposes and means of processing candidates’ data is, as a rule, a controller. In AI Act terms, the same entity becomes a “deployer”, that is, the natural or legal person operating an AI system under its authority. The Regulation also uses the broader notion of “operator”, which covers, as the case may be, the provider, the manufacturer, the deployer, the importer or the distributor.
This distinction is not cosmetic. The provider (the company that develops the candidate scoring platform) is responsible for compliance, documentation, risk management and technical transparency. The employer or recruiter using the system has obligations of its own: to use it in accordance with the instructions, to ensure human oversight, to monitor its operation, to keep logs and to inform the data subjects.
There is a further practical trap: the HR company may itself become a “provider” within the meaning of the AI Act if it substantially modifies the system, rebrands it (as happens in the case of licences of the whitelabel) or deliberately changes its purpose. For example, a recruitment agency acquires a candidate scoring platform, adds its own filtering criteria, integrates it with its internal database and offers it to clients. From the perspective of the AI Act, the agency is no longer a mere “deployer”, because it has intervened in the logic and presentation of the system to such an extent that it takes on the provider’s obligations, including those relating to risk management, technical documentation and the declaration of conformity.
The first question: does the system actually influence the selection?
An HR department should start with a simple question: what exactly does the system do in the recruitment process?
If the system merely centralises applications, the AI Act risk may be limited. If, however, the system automatically extracts skills from the CV, compares the candidate’s profile with the history of previous hires, awards scores, recommends candidates, eliminates applications below a threshold or ranks the list sent to the recruiter, it enters sensitive territory. The algorithm need not have the formal final say; it is sufficient for its output to significantly influence the human decision.
The AI Act allows, under certain conditions, for a system listed in Annex III not to be treated as high-risk where it does not pose a significant risk of harm to health, safety or fundamental rights and does not materially influence the outcome of the decision. That exception must, however, be documented, and systems that carry out profiling of natural persons remain, in principle, high-risk systems.
In practical terms, the more the AI score counts in the selection, the harder it becomes to sustain the argument that “it was only an auxiliary tool”. If the recruiter sees only the candidates pre-selected by the system, human oversight is no longer real; it is decorative.
Procuring the system: due diligence, not a sales demo
For the HR operator, compliance begins before a software procurement contract is signed. The AI Act requires high-risk systems to be sufficiently transparent to enable deployer to be able to interpret the results and use them appropriately. The instructions for use must set out, among other things, the intended purpose, the system's characteristics and limitations, accuracy levels, known risks, the expected input data, how the results are to be interpreted and the human oversight measures.
The questions that matter for the provider are not merely technical; they are legal and operational. What exactly does the system assess? What data was it trained on? What are its known limitations? How does it behave across different categories of candidates? How can the results be challenged or explained? What logs does it generate? What human intervention is possible? What happens when the system gets it wrong?
An HR-tech contract that fails to answer these questions creates a twofold risk: the risk of non-compliance with the AI Act and the risk of non-compliance with the GDPR. The two regimes are not mutually exclusive: the AI Act expressly preserves the application of the GDPR and of the other rules on data protection and privacy.
Use in accordance with the instructions and the problem of the “extended purpose”
One of the core obligations of the deployer is to use the high-risk AI system in accordance with the provider's instructions. The rule sounds obvious, yet in HR it is frequently disregarded.
For instance, a system acquired for the initial screening of applications should not also be used to appraise the performance of existing employees. A tool built to recommend candidates should not be turned, informally, into an automatic rejection mechanism. A model calibrated for one type of position should not be applied, without validation, to different roles, different industries or different labour markets.
The AI Act requires the deployer to use the system in line with the instructions and to ensure that the persons assigned to human oversight have the necessary competence, training, authority and support. Deployer must also monitor the operation of the system and notify the provider and the competent authorities where use of the system may give rise to a relevant risk.
Human oversight is not a signature at the end of the process
One of the most damaging misconceptions in automated recruitment is that having a recruiter in the process solves the problem. If the system screens candidates out before a human ever sees them, the decision is no longer genuinely human. If the recruiter invariably endorses the algorithmic score, the intervention is purely formal. If the platform does not adequately explain the criteria assessed, the person responsible has no real way of overseeing the process.
Article 14 of the AI Act requires high-risk AI systems to be designed so that they can be overseen by natural persons. Oversight must make it possible, as the case may be, to understand the system's capabilities and limitations, to monitor its operation, to interpret the output correctly, to avoid the psychological automatism known as “automation bias” and to disregard, override or reverse the system's output output.
The legal question is not “is there a human in the process?”, but “can that human actually change the outcome?”. If the answer is no, the organisation must treat the process as automated in substance, however its internal reporting lines describe it.
Article 22 GDPR remains central. A data subject has the right not to be subject to a decision based solely on automated processing, including profiling, where that decision produces legal effects or similarly significantly affects them. The exceptions are subject to strict conditions and appropriate safeguards, including the right to obtain human intervention, to express one's point of view and to contest the decision. In recruitment, an automatic rejection may have a significant impact on the individual. Even where the company maintains that the final decision rests with a recruiter, the analysis must look at how the process actually works. If the algorithm has removed the candidate from the visible shortlist and the human has had no genuine opportunity to assess the file, human intervention is illusory.
The case law of the CJEU on automated scoring, including the SCHUFA case, shows that an automatically calculated score may fall within the scope of Article 22 GDPR where it plays a determining role in a decision subsequently taken by another entity. The same logic applies in HR: it is not enough for the final decision to be, formally, the employer's, if the AI score in fact dictated the outcome.
Informing candidates and workers
The AI Act adds express information obligations, so that, before putting into service or using a high-risk AI system in the workplace, the deployer must inform workers' representatives and the workers concerned that they will be subject to the use of the system. Likewise, natural persons who are subject to a decision taken, or assisted, by a high-risk AI system must be informed that such a system is being used.
For candidates, this information should not be buried in a generic privacy policy. A useful wording should explain, in plain language, that the application may be analysed by an AI system, what role that system plays in the process, whether the output may influence selection, who reviews the output and how the candidate may request clarifications or human intervention.
This is where the AI Act and the GDPR overlap. If the system processes personal data, as it almost invariably does in recruitment, the employer must comply with the GDPR principles of lawfulness, transparency, data minimisation, storage limitation and the exercise of data subjects' rights.
Moreover, the AI Act introduces a distinct right to an explanation for persons affected by certain decisions taken by deployer on the basis of output of a high-risk AI system listed in Annex III. That person is entitled to receive clear and meaningful explanations regarding the role of the AI system in the decision-making procedure and the main elements of the decision, where the decision produces legal effects or significantly affects them as regards their fundamental rights, health or safety.
In practice, this does not mean that the employer must disclose the source code, the full algorithm or the provider's trade secrets. It does mean, however, that it cannot fob the candidate off with a sentence such as: “the system generated an internal score”. A useful explanation should indicate the role of the system, the main criteria assessed, the way in which the outputwas used and the existence of a human review.
For HR departments, this right must be catered for before the system goes live, not after the first complaint. If the organisation cannot explain internally why a candidate was rejected, it will not be able to explain it satisfactorily externally either.
Logs, monitoring and incidents
The AI Act requires deployers of high-risk AI systems to keep the automatically generated logs, to the extent that those logs are under their control, for a period of at least six months, unless EU law or applicable national law provides otherwise. Deployermust monitor the operation of the system and take action where it has reason to consider that its use may give rise to a risk.
In recruitment, logs can show which version of the system was used, which criteria were applied, what score the candidate received, whether the recruiter reviewed the result and whether there was any genuine human intervention. Without logs, the company cannot reconstruct a process that it is required to justify.
Monitoring should not be purely technical: the HR department must also review periodically whether certain categories of candidates are being screened out disproportionately, whether the system unjustifiably favours particular educational or professional backgrounds, whether the score thresholds are correctly calibrated and whether recruiters rely exclusively on the automated recommendation.
Where an incident arises (for example, a candidate demonstrates that they were excluded on the basis of a discriminatory criterion encoded in the system, or an entire batch of applications is screened out because of a configuration error), the deployermust be able to react quickly: the AI Act obliges it to suspend use of the system where it considers that the system presents a risk, and to inform the provider and the competent authority. In practical terms, the HR department should have an internal protocol answering three questions: who decides to shut the system down, how the affected applications are reinstated, and who notifies the supervisory authority. Without such a protocol, the incident does not remain a mere technical error but becomes a governance problem.
Checklist - the minimum for the HR operator
Before deploying an AI system in recruitment, the HR operator should be able to answer, on a documented basis, at least nine questions:
1. Does the system fall within the AI Act's definition of an AI system? 2. Is it intended for recruitment, selection, the filtering of applications or the assessment of candidates? 3. Who is the provider, who is the deployerand who is the data controller under the GDPR? 4. What is the intended purpose of the system, and is its actual use confined to that purpose? 5. Does the Outputof the system materially influence the rejection, ranking or selection of candidates? 6. What input data are used, and are they relevant to the position concerned? 7. Which individual is able to interpret, disregard, amend or override the result? 8. How are candidates informed, and how can they request explanations or human intervention? 9. What logs are kept, and for how long?
If these questions seem excessive for a simple recruitment tool, that is precisely the signal that the tool is no longer as simple as it appears.
Conclusion
The AI Act does not require recruitment to revert to paper files and managerial intuition; it does require organisations not to outsource the decision on access to a system that is opaque, unverified and difficult to challenge.
In human resources, an algorithm can help: it can organise information, reduce repetitive tasks and flag possible matches. What it should not become is the silent judge of a person's career.
For the HR operator, the underlying rule is straightforward: if the algorithm takes part in selecting the candidate, the organisation must be able to explain what the system does, why it is legitimate to use it, who supervises it and how an incorrect decision can be put right.
In the age of the AI Act, automated recruitment is no longer merely a question of efficiency. It is a question of governance, evidential documentation and legal liability.
Lawyer Victor Buju, Buju Stanciu & Associates