Automated Public Complaints Understanding: Managing Multimodal Inputs to Improve Authority-Level Decision Support

Abstract

Cities deal with many civic and environmental complaints including waste management, infrastructure issues, drainage problems and public utility issues. These reports are often unorganized, repetitive or incomplete, which makes it hard for authorities to respond effectively. Current systems depend on manual sorting and do not prioritize issues, causing delays and lowering citizen trust. To overcome this issues, this work suggest an AI-based framework CivicAI for understanding complaints automatically using multimodal inputs which uses Natural Language Processing to classify complaints texts, Computer Vision to analyze visual evidence from images, and Geolocation-based analysis to identify duplicate complaints and spatial hotspots. Transformer-Based Natural Language Processing was used for classification of complaints based on text, while YOLOv8n was implemented for detection of visual evidence. Shared-Space Late Fusion framework was employed to integrate predictions from text and images. In addition, the model allows summarizing complaints, calculating their prioritization level, detecting duplicates, recognizing hot spots, identifying escalations, and generating reliability scores using a dedicated dashboard for authorities. The RoBERTa text classifier achieved 86.61% accuracy, 0.8604% weighted-F1, and 0.6149 macro-F1.The YOLOv8n detector achieved 0.619 precision and 0.536 mAP@0.5.The findings show that multimodal complaint analysis can improve complaint understanding and provide practical support for faster, more organized, and evidence-based civic service response.