Multilevel Representation and Query Processing in Multimedia Database Systems

Arif Ghafoor (PI), Rangasami L. Kashyap (Co-PI), Shankar Moni (Co-PI)
Purdue University, West Lafayette, IN, 47907

Contact Information

Arif Ghafoor
School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, 47907
Phone: (765) 494-0638    Fax: (765) 494-3371    Email: ghafoor@ecn.purdue.edu

WWW PAGE

Distributed Multimedia Systems Laboratory

Keywords

multimedia data representation, fuzzy queries, spatio-temporal modeling, content-based retrieval, image processing

Project Award Information

Project Summary

The goal of this project is to develop a multimedia database system with capabilities to handle heterogeneous media queries. This system caters to the computational and storage requirements while accommodating and exploiting the inevitable semantic and representational imprecisions. The design of this system is based on multilevel data models and search mechanisms. These methodologies facilitate the users for posing various types of queries, including: (i) low-level, such as finding objects in a multimedia database, (ii) mid-level, based on spatio-temporal semantics, such as locating events associated with multimedia data, and (iii) high-level, targeted towards searching pre-composed multimedia documents, based on constituent mono-media data, their spatio-temporal dimensions, and logical structure. The multi-level search mechanisms are tightly interlinked. Imprecision in the search results is modeled using a set of fuzzy parameters. The results of this research are helping develop a comprehensive framework for building wide variety of multimedia applications in commercial, educational, governmental and military sectors.

Goals, Objectives, and Targeted Activities

Our long term goals include the development of a generic multi-level representation of multimedia data that can overcome several challenges faced by the database community. In particular, for low-level data modeling our research will focus on evaluating the unsupervised classification approach for image segmentation using a log-likelihood function. Two image segmentation models, namely the facet model and the texture model, are being evaluated in terms of computation and exactness of match. We are also planning to develop and evaluate other models. For mid-level multimedia data representation, we are planning to evaluate both interval-based and trajectory-based models in terms of complexity of semantic modeling and fuzziness in query formulation.

Indication of Success

So far the stated objectives for this project have been met quite well, as demonstrated by the concrete research results produced to date. Continuing success of this project in terms of quality research results has been aided by the collaboration between two researchers with backgrounds in image processing and multimedia databases. This collaboration has been instrumental in providing a sharp understanding of research challenges astride these areas. As a result of this unique collaboration, several research ideas and publications are being produced. The project is expected to provide viable solutions for developing a general framework for developing multimedia databases needed for a broad range of applications. The NSF funding has provided an opportunity for this collaboration which would have been difficult, otherwise.

Project Impact and Output

Three students are currently pursuing their doctoral studies as part of this project. The research results of this project have been incorporated in a graduate level course on multimedia systems (EE 624), which is taught by the PI. The project and its planned implementation is a cornerstone of the cutting-edge research being carried out in our lab. We anticipate interest from industrial organizations as the project matures and implementation-worthy results are produced.

Project References

[1] Y. F. Day, A. Khokhar, S. Dagtas, and A. Ghafoor, ``A Multi-level Abstraction and Modeling in Video Databases'', to appear in ACM Journal on Multimedia Systems.
[2] R. L. Kashyap and S. Sista, "Unsupervised Classification: Likelihood and Bayesian Approach," submitted to IEEE Computer Society Conf. on Computer Vision & Pattern Recognition, Santa Barbara, California, June 1998.
[3] Shu-Ching Chen and R. L. Kashyap, "Empirical Studies of Multimedia Semantic Models for Multimedia Presentations," 13th International Conference on Computer and Their Applications, Honolulu, Hawaii, USA, March 25-27, 1998.
[4] Shu-Ching Chen and R. L. Kashyap, "Temporal and Spatial Semantic Models for Multimedia Presentations," 1997 International Symposium on Multimedia Information Processing, Dec. 11-13, 1997.

Area Background

Our project is concerned with data management and information retrieval technologies essential for developing future multimedia systems. The emphasis of our research is on developing an automated system to allow multi-level data representation to assist query processing at different levels of abstraction. It will allow users to access images and video data based on appearance of objects as well as events surrounding these objects. The key tradeoff for users is between the accuracy of matching and the computational cost of the query. The framework developed for this project will provide solutions for challenging problems in multimedia data organization and integration, indexing and retrieval mechanisms, intelligent searching techniques, information browsing, content-based query processing and so forth. A large variety of potential applications will benefit from this framework.

Area References

There are several journals and conferences which have excellent coverage of issues in this area. They include: IEEE Multimedia; ACM Journal on Multimedia Systems; IEEE Trans. on Knowledge and Data Engineering; ACM Multimedia Conference; IEEE Int. Conf. on Multimedia Computing and Systems. Several special issues from IEEE Computer and ACM Journal on Multimedia Systems have been specifically devoted to this topic. Several industrial projects undertaken by IBM, Siemens, NEC, Oracle, Fuji Electric Co., etc., are focused on this topic.

Potential Related Projects

Within the NSF IDM program, several projects related to multimedia data modeling and management are being conducted in Case Western Reserve University, University of Maryland, University of Nevada, University of Illinois at Chicago, University of California at Santa Barbara, University of Pittsburgh, University of Maine, and University of Washington.