NoRA Benchmark: A New Era for AI Normative Competence

AI, there's a new benchmark that's turning heads: NoRA. This isn't just another test. It's a visual first-person video benchmark that challenges AI models to generate actions and justify them with a fact-reason-action support graph. The focus is on normative competence, a critical aspect for AI systems operating in social environments.

Why NoRA Matters

Traditionally, AI models have been evaluated on their ability to select actions from a predefined list. But let's face it, the real world doesn't hand you a menu of options. NoRA forces models to think from scratch, grounded in the visible facts of a scenario. This shift is monumental for AI development. It moves the evaluation from mere action selection to justifying why an action is the right choice based on observable reasons.

The Benchmark's Structure

NoRA comprises 1,420 annotated video clips, split into segments called HumanGold-190 and LLMSilver-1230. It evaluates AI systems through action alignment, factual grounding, and support binding, culminating in a single grounded reasonableness score. The results are telling. Current Visual Language Models (VLMs) frequently recover plausible actions and relevant scene facts. However, they struggle to construct a comprehensive reasonable action space and bind these actions to the correct local support. That's a significant gap.

The Real-World Implications

The benchmark results speak for themselves. As AI systems increasingly interact with social environments, the need for normative competence becomes essential. Can these systems justify their actions in a human-like manner? That’s the question NoRA puts forth. Western coverage has largely overlooked this kind of in-depth evaluation. The English-language press missed the importance of grounding AI actions in real-world facts and visible reasons, something NoRA is designed to address.

The question isn't just whether an AI model can pick an action. It's whether it can justify that choice convincingly, much like a human would. This is the direction AI evaluations need to head if we want truly reliable systems in our social spaces. And with NoRA, that future feels a little closer.