Unpacking the Complex World of VLMs in Physics

Vision Language Models (VLMs) are the latest darlings of AI research, promising to bridge the gap between visual perception and symbolic reasoning. Yet tackling physics problems, they're not exactly hitting home runs compared to human performance. Recently, researchers have been looking into how different reward designs might influence VLMs' abilities to reason about physical scenarios.

The Study's Approach

A recent study explored the impact of reward signals on VLMs using a systematic ablation approach. Using IBM Granite Vision 3.3, a VLM with 2 billion parameters, they tested how various rewards affected performance on a physics benchmark called PhyX. This benchmark isn't trivial. It spans 3,000 problems across six different domains and reasoning types, from multiple-choice to open-ended questions.

Think of it this way: when training VLMs, the type of reward you use can dramatically shift how these models 'think'. This study compared four types of rewards, ranging from basic format compliance to more nuanced ones like attention-weight rewards. Each offered a unique twist on how the model approached a problem.

Results: Not All Rewards Are Equal

The analogy I keep coming back to is that of a chef crafting a meal. The ingredients (or rewards, in this case) used can make all the difference in the final dish. Accuracy-based rewards, unsurprisingly, led to the strongest results, significantly outperforming Supervised Fine-Tuning (SFT) across most physics domains. But here's the thing: while they improved overall accuracy, they didn't do the same for structured reasoning, which is where rubric-based rewards came into play.

On the flip side, attention-based rewards boosted spatial reasoning. Sounds great, right? But they also degraded performance in symbolic reasoning domains. It's like boosting one skill but at the expense of another. If you've ever trained a model, you know the trade-offs are real.

Why This Matters

Here's why this matters for everyone, not just researchers. The study suggests that the way we train these models can fundamentally change how they approach complex tasks. For instance, the internal attention-weight reward, which doesn't require spatial annotations, improved spatial relation accuracy from 0.27 to 0.50. That's a pretty significant leap, especially when you consider scaling such models for practical applications.

So, why should you care? If VLMs are ever to be truly useful in real-world applications like robotics or autonomous systems, understanding how to best train them is essential. The current study highlights that a one-size-fits-all approach to reward design simply won't cut it. The type of reward influences the model's strengths and weaknesses, shaping it into what could either be a trusty tool or a glorified calculator.

Ultimately, the challenge isn't just about making VLMs better at physics problems. It's about understanding the nuances of model training and using that knowledge to build systems that can think, reason, and act in visually grounded environments. And isn't that the goal many of us are striving towards?