User Habits in AI Chat: Unmoving Yet key

AI chatbots, like Microsoft's Bing Copilot, are supposed to evolve our interactions with technology. However, a closer look at user behavior reveals a different story. An analysis of around 12,000 Bing Copilot users shows that individual habits with these chatbots are surprisingly sticky.

Habitual Stagnation

Despite the buzz around advancements in AI interactions, user habits seem to cling to the status quo. The study finds that while there are significant trends at the population level, individual habits are far less dynamic. It's like we've given users a Formula 1 car, but they're still driving in first gear.

Why does this matter? Because if individual users aren't changing their behavior, then who really benefits? The benchmarks don't capture what matters most: actual behavioral change. If the goal is to make these tools universally useful, then it's clear more needs to be done to encourage diverse ways of engaging with them.

Power Users vs. Casual Users

The data shows sharp differences between heavy users and casual ones. Active users, unsurprisingly, have more successful interactions and tackle more complex tasks. They're the ones benefiting from these AI advancements. But what about everyone else?

Enter WildChat-4.8M, another dataset used for comparison. It's heavily skewed towards proficient 'power' users. This suggests a huge gap between those who can fully exploit these tools and those who can't. Whose data is this really serving? Not the average user, clearly.

The Bigger Picture

The real question is how these findings impact the future development of AI tools. Do tech giants like Microsoft focus on making tools for the few who can adapt quickly, or should they consider the everyday user who struggles to shift gears?

The paper buries the most important finding in the appendix: that WildChat doesn't represent typical user-AI interactions. This is a major caveat for anyone using this data for further studies or product development. If we want AI to be inclusive, we need to rethink how we analyze and interpret user data.

So, what's the takeaway? If AI is to truly democratize information and interaction, it has to evolve beyond serving the tech-savvy elite. Ask who funded the study and why these disparities persist. Because, this is a story about power, not just performance.