CLOSETAI: A MULTIMODAL AI-DRIVEN FASHION ASSISTANCE PLATFORM USING REAL-TIME VIDEO ANALYSIS

Abstract

The access to personal stylistic advice is limited by the cost, geographical constraints, and the scarcity of trained professionals who can give advice.This paper introduces ClosetAI, which is a web browser application capable of giving feedback on the outfit chosen by the user via a live video chat interface.The system uses the WebRTC and MediaDevices APIs to capture video from the camera, takes samples of the video frame at regular intervals and sends each frame along with the user’s query and a serialized history of the conversation to the multimodal vision-language model using the Google Gemini API.The model provides feedback on the color combination, pattern matching, shape balance, and appropriateness of the outfit, and its recommendation is transmitted token by token to the client.Front end is developed using Next.js, React, TypeScript and Tailwind CSS, back end relies on Node.js API routes using WebSocket transport and Redis as a session storage.This paper contributes an architecture and design patterns for creating a web application for low-latency, multimodal consultations adaptive frame sampling, persona-based system prompting, and streaming token delivery, along with the evaluation protocol. We describe the system design and the planned evaluation in detail so that the results can be independently reproduced.