A Multimodal AI and ML Framework for Fashion Image Segmentation, Recommendation, and Similarity Recognition

Abstract

This study presents a scalable multimodal Artificial Intelligence (AI) and Machine Learning (ML) framework designed to enhance decision making in the fashion industry. The proposed system integrates garment segmentation, multimodal feature extraction, and similarity recommendation into a unified pipeline. Using Segformer for segmentation, along with the convolutional neural network (CNN)-based feature extraction models ResNet152V2 and Xception, and the transformer-based vision-language model LLaVA, the framework generates visual and semantic embeddings of garments. These representations are processed through similarity detection using OpenAI embedding models and stored in the Pinecone vector database for efficient retrieval. Real-time similarity scoring is enabled through FastAPI endpoints, offering interactive search capabilities. Preliminary results demonstrate the system's strong ability to identify conceptually and visually similar items across a large catalog, providing actionable insights for designers. This framework lays the groundwork for intelligent, interpretable, and production-ready AI systems in the fashion industry. © 2025 IEEE.