AI Hunts Swedish Heritage, Leaves Questions in Its Wake

As the European Union marches towards implementing the Energy Performance of Buildings Directive by 2025-2026, Sweden finds itself in a precarious position. While its neighbors forge ahead with National Building Renovation Plans, Sweden grapples with a missing piece: a comprehensive national register of buildings with heritage values. And what does Sweden turn to? Multimodal Large Language Models (LLMs) analyzing street view images.

The AI Approach

As innovative as it sounds, this approach involves zero-shot predictions made by LLMs to identify potential heritage buildings. In numbers, 154,710 buildings across Sweden are under the AI microscope, scanning through five million square meters of heated floor area. But here's the kicker, can we trust AI to recognize cultural significance?

Ask who funded the study. It's not just about performance. it's about power. Swedish authorities aim to use AI's findings to help craft these renovation plans. But are they banking too much on tech that doesn't quite grasp the nuances of heritage?

Transparency and Trust

The method's transparency, or lack thereof, leads to essential questions about governance and accountability. How does an algorithm weigh the significance of a historical facade? The risks of relying on AI aren't trivial. Transparency issues, error detection, and the troubling tendency of AI to simply agree (a.k.a. sycophancy) present real challenges. The benchmark doesn't capture what matters most, the human stories behind these bricks and mortar.

And then there's the question of who benefits. Heritage recognition isn't just a checkbox in renovation plans. it's about preserving cultural identity. Is AI capable of that kind of subtlety, or are we risking a whitewashed version of history?

Lessons for the Future

While the aim is noble, the execution needs work. The paper buries the most important finding in the appendix, but what's clear is that AI isn't infallible. Instead, it should be seen as one tool among many in a broader strategy. There's potential here, sure, but let's not give AI the keys to our cultural vault just yet.

Ultimately, as Sweden navigates these murky waters, one must ask: Are we ready for machines to define our heritage? The stakes are high. Because if AI gets it wrong, we're not just losing buildings. We're losing a part of ourselves.