A Hybrid Convolutional Neural Network Approach for Context-aware Fashion Recommendation

Abstract

This study aimed to develop a fashion recommendation system that classifies clothing items and recommends complementary outfit pieces based on user-selected style preferences, addressing the limitation of existing systems that rely solely on visual similarity without incorporating style-based personalisation. The study was carried out at the Department of Information and Communication Technology, Faculty of Humanities and Social Sciences, University of Sri Jayewardenepura, Sri Lanka, between 25th July 2025 and 2nd June 2026, and followed a quantitative experimental approach involving the design, development, and performance evaluation of a two-stage deep learning-based classification and recommendation system. The proposed system consists of two stages: classification and recommendation. In the first stage, the system classifies the query item using a hybrid convolutional neural network (CNN) model combining ResNet-50 and EfficientNetB0. In the second stage, Fashion CLIP and CLIP ViT-B/32 models retrieve complementary items, which are then filtered and re-ranked based on the user-selected style from four categories: Casual, Formal, Party, and Streetwear. The classification model achieved 94% accuracy on the main dataset and 89.17% on external validation, while the recommendation pipeline achieved a mean Precision@5 of 84.2% and Accuracy@5 of 94.4%. The proposed system achieved 100% style consistency compared with 56.2% for the baseline model. The proposed two-stage system combines item classification with style-aware recommendation and has practical potential for integration into fashion e-commerce platforms to enhance user experience and support cross-category sales.