DERM-3R: Resource-Efficient Multimodal AI for Dermatology

The DERM-3R framework introduces a resource-efficient multimodal agentic approach to dermatologic diagnosis and treatment, addressing a significant global health burden. This innovation is particularly relevant for real-world clinical settings where data and computational resources are often limited. By reformulating complex dermatological decision-making into three distinct core issues—fine-grained lesion recognition, multi-view lesion representation with specialist-level pathogenesis modeling, and holistic reasoning for syndrome differentiation and treatment planning—DERM-3R provides a structured and scalable solution. This contrasts with brute-force scaling strategies often seen in general-purpose AI, demonstrating the power of domain-aware, multi-agent modeling.

The framework comprises three collaborative agents: DERM-Rec for recognition, DERM-Rep for representation, and DERM-Reason for holistic reasoning, each targeting a specific component of the diagnostic and treatment pipeline. Built upon a lightweight multimodal LLM and partially fine-tuned on a focused dataset of 103 real-world Traditional Chinese Medicine (TCM) psoriasis cases, DERM-3R exhibits strong performance across various dermatologic reasoning tasks. This targeted training on a smaller, specialized dataset, rather than extensive general data, highlights an effective strategy for developing high-performing AI in niche medical domains, especially when resource constraints are a factor.

Evaluations, including automatic metrics, LLM-as-a-judge assessments, and physician evaluations, indicate that DERM-3R matches or even surpasses larger, general-purpose multimodal models. This outcome suggests that specialized, structured multi-agent architectures can be a practical and efficient alternative to massive model scaling for complex clinical applications, particularly in integrative medicine where holistic reasoning is paramount. The implications are significant for improving access to advanced dermatological care, standardizing TCM practices, and potentially reducing the burden of dermatologic diseases worldwide, especially in regions with limited access to expert dermatologists. Future work will need to focus on expanding the dataset and validating its performance across a broader spectrum of dermatological conditions and patient demographics to ensure robust and equitable application.