My thesis

What is it?

Needle is a platform for facilitating the finding of information in the content of video and other multimedia material of events: lectures, seminars, talks, meetings, and more. All the resources (e.g. presentation slides) are time-synchronized with the video. Needle provides semantic support to information retrieval through intelligent use of Wikipedia as background knowledge, in combination with statistical Information Extraction techniques.

Who is it for?

Needle is for almost everyone who deals with larger amounts of multimedial data. We recommend Needle especially for use in seminars, e-Learning and e-Meeting events that are to be re-produced by a larger audience at a later point in time. Needle provides a straight-forward access to information presented inside a video-recorded presentation and additional material the speaker refers to.

What does it offer?

Needle offers intelligent search on multimedia content displaying the most important topic presented in the video and in the presentation of an event, and the relationship between a searched topic and other topics presented in the multimedia content.

For every search query, segments of video lectures are displayed together with presentation material that has been presented during the lecture at the same temporal slot.  Every search result presents links to the relevant video segment, an audio-only version of the video, the presentation slide synchronized with the video, and a navigable combined view of the video and the presentation slides.

Additionally, it displays a summary of the Wikipedia definition used to annotate the most important terms of the recording, and some a suggestion of related topics that can be found in the material. In this way we are able to provide annotations semantically tailored to the domain of the material, while we propose an explanation of the term through the summarized Wikipedia definition.

How does it work?

Annotations are automatically generated analyzing the video transcript and the material which comes with the video. We use all the multimedia content (video, audio, presentation slides, text documents, etc.) referring to an recording as training data for discovering annotations in the Wikipedia taxonomy.

The annotation vocabulary is defined through Wikipedia, for the reason of which there is no ambiguity in the meaning of a term in the corpus. The annotations in combination with textual information retrieval are used for giving search suggestions and summarization of the topics presented in the material.

Future Challenges

Future R&D challenges include:

• the automatic generation of video transcripts;
• multilanguage support;
• the automatic generation of a topic map for a recording;
• the creation of an online version to produce, index and share your own presentation.

Want to learn more?

1. Angela Fogarolli, Giuseppe Riccardi, and Marco Ronchetti. Searching information in a collection of video-lectures. In Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007, pages 1450–1459, Vancouver, Canada, June 2007. AACE.
2. Angela Fogarolli and Marco Ronchetti. Discovering semantic in multimedia content using wikipedia. In Proceeding of 11th International Conference on Business Information Systems, Innsbruck, Austria 5-7 May 2008, Lecture Notes in Business Information Processing, pp. 48–57. Springer, Heidelberg (2008).