Year: 2004
Type: conference-paper
Abstract: Given an image (or video clip, or audio song), how do we automatically assign keywords to it? The general problem is to find correlations across the media in a collection of multimedia objects like video clips, with colors, and/or motion, and/or audio, and/or text scripts. We propose a novel, graph-based approach, "MMG", to discover such cross-modal correlations.Our "MMG" method requires no tuning, no clustering, no user-determined constants; it can be applied to any multimedia collection, as long as we have a similarity function for each medium; and it scales linearly with the database size. We report auto-captioning experiments on the "standard" Corel image database of 680 MB, where it outperforms domain specific, fine-tuned methods by up to 10 percentage points in captioning accuracy (50% relative improvement).
Institution: Carnegie Mellon University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: Collaborative Research: NetMine: Finding Patterns in Network Data, U.S.-Japan Joint Seminar: A Data Mining-Based Approach to Semantic Integration of Stream Data, ITR: Providing Intuitive Access to Human Motion Databases, ITR Collaborative Research: Indexing, Retrieval, and Use of Large Motion Databases, SENSORS: Placement and Operation of an Environmental Sensor Network to Facilitate Decision Making Regarding Drinking Water Quality and Security, ITR/IM: Capturing, Coordinating and Remembering Human Experience, Collaborative Research: Data Mining Meets I/O Performance Evaluation: Advanced Statistical Tools for Analyzing Bursty Traffic, Information Technology Research (ITR): Next-Generation Bio-Molecular Imaging and Information Discovery
OpenAlex ID: W2063049279
