Algorithmic power and the fragility of accountability

We find ourselves at an inflection point where the rhetoric of technological progress races ahead of the slower, steadier disciplines of law and regulation. Algorithmic systems, including machine learning systems, predictive analytics, automated decision-making and decision-support, have been embraced with remarkable enthusiasm by policymakers eager to demonstrate efficiency and innovation. Yet, as with previous moments of digital transformation, from early internet governance debates to the rise of platform power, we would be unwise to confuse enthusiasm with legitimacy, or automation with authority.

The promise of algorithmic regulation is seductive. Its champions speak of data-driven precision, neutrality, and the removal of human frailty from administrative decision-making. This vision, so often marketed with the glossy optimism found in certain corners of Silicon Valley, obscures a more uncomfortable reality. Algorithmic systems entrench human judgments, institutional aims, and historical biases, as expressed through a mathematical grammar that gives them the appearance of inevitability. Cathy O’Neil’s Weapons of Math Destruction remains a stark reminder that the authority of numbers can be as political as the authority of law.

Legal systems are, of course, no strangers to complexity. But they are premised on intelligibility. A citizen may disagree with a law; they may dislike a regulatory outcome; but the system is designed, normatively and institutionally, to allow them to understand it, challenge it, and seek redress. Algorithmic governance disrupts this equilibrium. A decision taken by an automated welfare assessment tool or predictive policing model can feel, to the individual subject and in its consequences, both inscrutable and unassailable. The Dutch SyRI judgment illustrates the tension: when the rationale for decision-making disappears into proprietary code and inferred correlations, reviewers including judges, struggle to grasp the object of scrutiny.

This is not merely a technical problem; it is a constitutional one. The rule of law requires more than outcomes – it requires processes that are visible, contestable, and reasoned. Yet opacity, whether due to commercial secrecy or model complexity, has become the quiet default of algorithmic regulation. The UK Information Commissioner’s Office has repeatedly emphasised, in its Guidance on AI and Data Protection, that transparency is not an optional extra. But we must recognise that even well-intentioned regulatory guidance risks being overwhelmed by the scale and speed of contemporary computational systems.