Code Sharing in AI Models: A Promise Unfulfilled

Analytical code is the backbone of reproducible research in diagnostic and prognostic prediction models. Yet, despite its importance, sharing this code in published research is far from common practice. A recent review examined the current state of code-sharing practices, focusing on studies citing the TRIPOD or TRIPOD+AI guidelines, and the findings are eye-opening. Let's apply the standard the industry set for itself.

The Current State of Code Sharing

In an analysis of 3,967 eligible articles, only 12.2% included code-sharing statements. Over time, this number increased slightly, reaching 15.8% by 2025. Notably, studies citing TRIPOD+AI were more likely to share code than those citing only TRIPOD. Sharing prevalence varied widely depending on the journal and country of publication. The real issue, however, lies in the quality of what's shared.

When code is made available, it often falls short of being truly reusable. The study found substantial heterogeneity in reproducibility-related features across repositories. While 80.5% of repositories included a README file, a critical component for understanding code, only 37.6% specified dependencies, and even fewer (21.6%) had version-constrained dependencies. Just 42.4% of repositories were modular, highlighting a significant gap between available code and usable code.

Beyond Availability: The Real Challenges

These findings underscore an urgent need for more than just code availability. Clear expectations must be set for documentation, dependency specification, licensing, and executable structure. Otherwise, we're left with a facade of transparency without the substance. Show me the audit.

Why should researchers and developers care about this? Because the burden of proof sits with the team, not the community. If code can't be easily reused and verified, the credibility of the research itself is called into question. Are we building a house of cards under the guise of scientific advancement?

: Setting the Right Expectations

The development of the TRIPOD-Code extension aims to address these issues by establishing clearer guidelines for code sharing. But as always, the execution will be key. Will these guidelines make a tangible difference, or will they become another set of standards that looks good on paper but flounders in practice?

Skepticism isn't pessimism. It's due diligence. And AI research, there's no room for complacency. As we move forward, the industry must hold itself accountable, ensuring that promises made are promises kept. The future of transparent and reproducible research depends on it.