LLMs and Privacy: The Forgotten Risk of Interdependent Data

This week in 60 seconds: Large language models (LLMs) are reshaping our digital landscape, but there's a privacy snag lurking in the shadows. As personal AI assistants become a staple in our lives, they not only manage our data but also intermingle it with others'. This gives rise to a pesky issue called interdependent privacy (IDP), where your data could spill the beans without you even knowing it.

Introducing IDP-Bench

In the area of AI privacy, a new player has entered the game. It's called IDP-Bench, and it's setting the stage for addressing this overlooked privacy dilemma. Designed as the first benchmark for IDP scenarios, IDP-Bench roots itself in the Contextual Integrity (CI) framework to assess how LLMs handle the complexities of interdependent data.

Eight open-source LLMs were put to the test, navigating through various IDP scenarios. Two LLM judges evaluated them on their ability to recognize co-ownership, a task most models nailed, with 6 out of 8 scoring over 90%. But when it came to identifying the nitty-gritty details like information attributes and who else might be involved, the results weren't as rosy. A whopping 7 out of 8 models stumbled, scoring below 74%.

The Takeaway

Why should you care? Well, imagine sharing a photo with a friend. Now think about that friend sharing it with their circles, and suddenly your image is practically public. That's IDP in action. These LLMs showed a credible understanding of sharing appropriateness, yet their struggles with IDP-specific questions highlight a critical gap in privacy research. Five out of the eight models scored below 77% on judging sharing appropriateness. Scale helps, but smaller models fell flat, proving that size matters when privacy's on the line.

So, what's the one thing to remember from this week? If LLMs want to earn our trust, they'll need to beef up their IDP handling. After all, wouldn't you want your AI assistant to respect your privacy as much as it respects your requests?

Looking Forward

This benchmark isn't just a report card for LLMs, it's a wake-up call for developers to focus on these privacy oversights. The high sensitivity to prompts in IDP-related queries suggests that LLMs are still shaky on solid ground. As the tools for privacy protection continue to evolve, tackling IDP should be top of the list. The data and code from IDP-Bench are out there now, ready to spark more research. Will it be enough to drive change? Let's hope so. That's the week. See you Monday.