FigEx2: Turning Unusable Scientific Figures into Data Goldmines

Scientific compound figures, those multi-panel images prevalent in research papers, often come with a frustrating pitfall: inadequate captions. A staggering 16.3% of these figures lack captions entirely, and 1.8% have captions shorter than ten words. This oversight renders them unusable in current caption-decomposition processes, essentially wasting a treasure trove of potential data.

FigEx2: A Game Changer

Enter FigEx2, a visual-conditioned framework designed to salvage these neglected figures. It localizes panels and generates captions directly from the image, turning what was once trash into treasure. This process not only revives discarded data but significantly enriches resources for downstream pretraining and retrieval tasks.

Why does this matter? In an era where AI's hunger for data is insatiable, every data point counts. FigEx2's ability to create panel-text pairs from unusable figures gives researchers a critical advantage, potentially accelerating breakthroughs across disciplines.

Innovative Techniques

The framework introduces a noise-aware gated fusion module. This module adapts the way caption features condition the detection query space, thus mitigating linguistic variance in open-ended captioning. Additionally, a staged SFT+RL strategy employs CLIP-based alignment and BERTScore-based semantic rewards.

These sophisticated techniques aren't just academic exercises. They're the backbone of high-quality supervision, leading to the curation of the BioSci-Fig-Cap benchmark for panel-level grounding. This isn't just about improving metrics. it's about setting a new standard for data quality in scientific research.

Performance and Impact

FigEx2 achieves a 0.728 mAP@0.5:0.95 for detection, outperforming the Qwen3-VL-8B model by 0.44 in METEOR and 0.22 in BERTScore. More impressively, it transfers zero-shot to out-of-distribution scientific domains without any fine-tuning. This cross-disciplinary agility makes FigEx2 not just a tool, but a turning point asset in AI-driven research.

Is this the future of scientific data processing? FigEx2 certainly points in that direction. By transforming unusable data into a valuable resource, it opens doors to new possibilities in AI research and beyond. The question is, how quickly will the research community embrace this change?