Rethinking AI Compliance: From Checkbox to Continuous Monitoring

The traditional approach to AI compliance has long been about ticking boxes at audit time. But with the EU AI Act shaking things up, the focus is shifting. Instead of a binary 'compliant or not' verdict, there's a growing need for continuous, measurable oversight. This is where the idea of 'governance from metrics' enters the scene.

A New Compliance Model

Governance from metrics suggests that compliance shouldn't just be a snapshot in time. Instead, it should be a continuous signal, derived from runtime observability. This means AI systems are monitored in real-time, allowing for ongoing adjustments based on how they're actually behaving in the field. That's a significant pivot from the static assessments most companies have relied on so far.

Enter govllm, an open-source framework that aims to implement this continuous governance. Its architecture relies on a panel of regulatory judges, LLM evaluators tailored to specific criteria like the EU AI Act, GDPR, and more. These judges don't just flag issues. they provide a compliance score that can influence model selection, moving beyond cost and latency as sole decision factors.

The Uncertainty Factor

But what happens when the judges can't agree? The developers of govllm argue that inter-judge disagreement shouldn't be seen as noise. Instead, it's a signal for regulatory uncertainty, a call for human arbitration. That’s a novel way of looking at the variability inherent in AI evaluations.

The team tested this approach with 49 annotated prompt/response pairs across five regulatory criteria, using four small language models. Agreement rates varied, from 51.5% with the mistral:7b model to 69.1% with phi4-mini. No single model was a jack-of-all-trades, which suggests their Profile-as-jury design might actually be onto something.

Challenges and Opportunities

Of course, the system isn't without its challenges. Three structural failure modes in the small regulatory judges were documented, along with a judge-specific position bias that degraded agreement by up to 25 percentage points. These hiccups highlight the complexity of creating a reliable compliance framework.

So why does this matter? Because in production, AI compliance can't just be a checkbox. The real test is always the edge cases. Regulatory frameworks like the EU AI Act demand systems that can adapt and remain compliant as they evolve. The demo's impressive. The deployment story is messier, but that's where the real progress happens.

Will this approach catch on? It's too soon to call it a revolution, but it’s a step in the right direction. As AI systems become more integrated into our lives, continuous oversight will be key. The catch is ensuring these systems are as strong in the real world as they're in theory. Here's where it gets practical.