This study utilized the ThyroidEffi 1.0 dataset to compare frozen visual encoders for classifying thyroid nodules. The population included cases categorized as Bethesda II (Benign), Bethesda V (Suspicious), and Bethesda VI (Malignant). The analysis involved four models: MedSigLIP, a medical image–text pretrained encoder; and ImageNet-pretrained ResNet50, EfficientNet-B0, and ViT-Base. The primary outcome was Macro-F1, with secondary outcomes including balanced accuracy, Expected Calibration Error (ECE), and recall for the Suspicious class.
Regarding aggregate classification accuracy, EfficientNet-B0 achieved a Macro-F1 of 0.845 ± 0.021, outperforming ViT (0.817 ± 0.020) with a p-value less than 0.05. The difference between EfficientNet-B0 and MedSigLIP (0.836 ± 0.019) was not statistically significant after multiple comparison correction. ResNet50 scored 0.829 ± 0.015. For the recall of the Suspicious class, MedSigLIP achieved 0.808, the highest among the models. In terms of Expected Calibration Error, MedSigLIP showed the best performance at 0.025, compared to 0.044–0.082 for the general-purpose encoders.
Safety and tolerability data were not reported, as adverse events and discontinuations are not applicable to computational model evaluation. A key limitation is the lack of prospective validation in real-world triage workflows. The study notes that MedSigLIP did not yield a statistically significant advantage in aggregate classification accuracy compared to the best ImageNet-based model. Consequently, practice relevance dictates that encoder selection should be guided by a joint view of discrimination and safety, particularly calibration and Bethesda V sensitivity, rather than aggregate accuracy alone.
View Original Abstract ↓
BackgroundFine-needle aspiration biopsy (FNAB) cytology is central to thyroid nodule evaluation, yet reliable differentiation across Bethesda categories remains challenging, particularly for the indeterminate Bethesda V (Suspicious for Malignancy) class. While transfer learning with ImageNet-pretrained models is a standard approach, it remains unclear whether emerging domain-specific medical foundation models offer superior performance compared to general purpose baselines in this specialized domain.MethodsWe benchmarked four frozen visual encoders—ResNet50, EfficientNet-B0, ViT-Base (ImageNet pretrained), and MedSigLIP (medical image–text pretrained)—on the ThyroidEffi 1.0 dataset (N = 1,804), comprising Bethesda II (Benign), Bethesda V (Suspicious), and Bethesda VI (Malignant) cases. A unified evaluation protocol was employed using five-fold stratified cross-validation with a lightweight multilayer perceptron head. Performance was assessed using macro-F1, balanced accuracy, Expected Calibration Error (ECE), and McNemar’s test for statistical significance.ResultsEfficientNet achieved the highest macro-F1 (0.845 ± 0.021), followed closely by MedSigLIP (0.836 ± 0.019), ResNet50 (0.829 ± 0.015), and ViT (0.817 ± 0.020). Pairwise statistical testing revealed that while EfficientNet significantly outperformed ViT (p < 0.05), the difference between EfficientNet and MedSigLIP was not statistically significant after multiple comparison correction. Notably, MedSigLIP demonstrated superior reliability attributes, achieving the highest recall for the challenging Suspicious class (0.808) and the best model calibration score (ECE = 0.025) compared to the general-purpose encoders (ECE: 0.044–0.082).ConclusionsWhile domain-specific medical pretraining (MedSigLIP) did not yield a statistically significant advantage in aggregate classification accuracy compared to the best ImageNet-based model (EfficientNet), it provided superior calibration and sensitivity for borderline cases. These findings suggest that in thyroid cytology clinical workflow support, encoder selection should be guided by a joint view of discrimination and safety—particularly calibration and Bethesda V sensitivity—rather than aggregate accuracy alone, enabling threshold-based triage and selective expert review. In particular, well-calibrated models such as MedSigLIP suggest a potential benefit in reducing overconfident misclassification in borderline Bethesda V cases, pending prospective validation in real-world triage workflows.
