DOMIR - Dialogs and Ontologies for Multimedia Information Retrieval

To go beyond text retrieval, the analysis of document components such as audio or image segments is required. The Metamedia prototype (FEUP) currently incorporates the extraction of audiovisual features and the automatic association of the corresponding descriptors to the document. Further work is required on audiovisual extraction, in order to create more expressive descriptors. Ontologies will be used at this point: both the inclusion of audiovisual descriptors in ontologies and their combination with domain ontologies will be explored.

A second component is dialog management. The retrieval task can be significantly improved by gathering information from the user interaction and analyzing the dialog to extract user intentions and plans. Retrieval is an intrinsically imprecise task and therefore this line has to be complemented by appropriate evaluation procedures and tools.

A third line of research is the refinement of the database model to encompass the association of metadata to objects at different levels, the compliance with audiovisual standards and the use of heterogeneous descriptors in the computation of similarity measures for retrieval. Audiovisual descriptors are commonly multi-dimensional and quantitative; the similarity measures required in retrieval open a large ground for new approaches.

Funding entity: Fundação Ciência e Tecnologia (MCTES).

Reference: FCT POSC/EIA/61109/2004

Funding: 90000 Euro.

Working prototype.