Year: 2024
Type: conference-paper
Abstract: Recent advances in Visual Language Models (VLMs) have significantly enhanced video analytics. VLMs capture complex visual and textual connections. While Convolutional Neural Networks (CNNs) excel in spatial pattern recognition, VLMs provide a global context, making them ideal for tasks like complex incidents and anomaly detection. However, VLMs are much more computationally intensive, posing challenges for large-scale and real-time applications. This paper introduces EdgeCloudAI, a scalable system integrating VLMs and CNNs through edge-cloud computing. Edge-CloudAI performs initial video processing (e.g., CNN) on edge devices and offloads deeper analysis (e.g., VLM) to the cloud, optimizing resource use and reducing latency. We have deployed EdgeCloudAI on the NSF COSMOS testbed in NYC. In this demo, we will demonstrate EdgeCloudAI's performance in detecting user-defined incidents in real-time.
Institution: Columbia University
Language: English
Topic: IoT and Edge/Fog Computing
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
Awards: CAREER: Adaptive Algorithms for Combinatorial Optimization in Stochastic Networks, CPS: Medium: Hybrid Twins for Urban Transportation: From Intersections to Citywide Management, NSF Engineering Research Center for Smart Streetscapes (CS3), G2274430557, RINGS: Enabling Wireless Edge-cloud Services via Autonomous Resource Allocation and Robust Physical Layer Technologies, G4578656615, IRNC: Testbed: COSMOS Interconnecting Continents (COSMIC)
OpenAlex ID: W4405013789