Portuguese Clinical NER: Can mmBERT Outperform the Big Players?

Clinical notes in Portuguese are a goldmine of unstructured data, and mining them just got a whole lot more interesting. Enter named entity recognition (NER), the unsung hero of medical data extraction, now stepping into the spotlight in Portugal. But here's the kicker: benchmarks have been few and far between.

Who's Battling for Top Spot?

In the ring, we've the BERT-based contenders, BioBERTpt, BERTimbau, ModernBERT, and mmBERT, facing off against giants like GPT-5 and Gemini-2.5. The challenge? Extract medical concepts from Portuguese clinical notes as precisely as possible.

Using the SemClinBr corpus and a private breast cancer dataset, these models went head-to-head under the same conditions. Precision, recall, and F1-score were the weapons of choice. Spoiler alert: mmBERT-base came out swinging the hardest, scoring a micro F1 of 0.76. Not bad for a multilingual maverick.

Why Should We Care?

Sure, numbers are great, but why should anyone outside the clinical field give a hoot? Here's why: the ability to run powerful NER models like mmBERT locally, with modest computational resources, democratizes access to latest tech. We're talking serious implications for smaller medical institutions and researchers in resource-constrained environments.

tackling class imbalance in datasets with strategies like iterative stratification isn't just a technical win. It's a breakthrough for reliability and accuracy, vital in clinical settings where missteps can mean life or death.

The Bigger Picture

So, is mmBERT the future of Portuguese clinical NER? If retention and precision are anything to go by, it's certainly a strong candidate. But don't count out the big guns just yet. GPT-5 and Gemini-2.5 have the backing of massive resources and continuous updates.

Yet, the question remains: can smaller, more adaptable models like mmBERT continue to outperform, or will they hit a ceiling without the constant influx of new data that larger models receive? For now, mmBERT is punching above its weight, and that's worth keeping an eye on.