The DermLIP model is presented for the analysis of dermatological images

Developers presented DermLIP_PanDerm-base-w-PubMed-256 — a specialized multimodal model designed to solve problems in the field of dermatology. The system is built on the CLIP architecture and adapted to work with medical data.

The basis of the model's training is the Derm1M dataset, which includes 403,000 "image-text" pairs. The visual component is processed using the PanDerm-base encoder, and for analyzing text descriptions, a specialized PubMedBERT encoder is used. This combination allows the model to effectively perform classification and image search tasks in zero-shot mode.

The tool is aimed at automating the analysis of skin diseases, allowing clinical images to be matched with medical descriptions. The model is available for use through the Hugging Face platform, where the weights and architecture configuration are presented for integration into research projects.

The development is aimed at improving the accuracy of diagnosis when working with dermatological atlases. The use of specialized encoders trained on the PubMed corpus provides a deeper understanding of medical terminology compared to general-purpose models.