Aligning User Stories with Interviews: A New Approach

Aligning interview transcripts with software requirements is no small feat. Yet, it's essential for ensuring those requirements truly reflect stakeholder needs. This new research proposes a structured approach to tackle the challenge.

Formalizing Alignment Tasks

The key innovation? Formalizing the task of mapping interview transcripts to user stories. The researchers introduce two heuristic metrics: requirements faithfulness and interview coverage. Requirements faithfulness measures how many user stories are supported by the interview transcript. Interview coverage, on the other hand, checks how much of the transcript is backed by at least one story.

Why should this matter to developers and project managers? Because aligning these narratives ensures that what gets built matches what’s needed. It’s about precision and avoiding costly misalignments later on.

Experimenting with Large Language Models

Testing this framework involved large language models (LLMs) and embedding models. With experiments across four datasets, one LLM-based solution achieved a macro-F1 score of 0.86 on manually labeled data. That's impressive, showing these models can automatically assess alignment with high accuracy.

embedding models can act as blockers, adding scalability to the process. This means larger projects can still benefit without manual bottlenecks.

Implications and Future Directions

The paper's key contribution is more than just methodology. It paves the way for automated systems that trace requirements back to their conversational origins. Imagine tools that can generate requirements from interviews, bridging the gap between discussion and documentation.

But let's ask the pressing question: can this really replace the nuanced understanding of a skilled analyst? While automation offers speed and breadth, human insight remains critical, especially in complex, high-stakes projects. The ablation study reveals where models excel and where human intervention remains essential.

Code and data are available at the project repository, inviting further research. This builds on prior work from natural language processing, illustrating how AI can enhance but not entirely replace human expertise in requirements gathering.