Speaker
Ms
Cristina Versino
(European Commission - Joint Research Centre)
Description
Surveillance streams contain thousands of images. Inspectors review them in order to find the safeguards-relevant events. Only a small fraction of the images is expected to be safeguards-relevant. Inspectors need tools to focus their attention directly to the relevant parts of the surveillance stream.
The VideoZoom approach to reviews detects scene changes on the whole image plane. Changes are summarised and rendered at different levels of abstraction in layers of summaries, each layer revealing complementary information about the changes. By means of a zooming interface, the reviewer navigates the summaries and decides which to examine in full. Reviewers can make best use of their time by investigating what really requires their attention.
Building on this concept, extensions to VideoZoom can be envisaged to increase the efficiency of video reviews even further by employing video retrieval techniques to model and find events so that an inspector looking at a specific event can quickly preview all other occurrences based on their similarity. Similarity is measured by comparing image features extracted during video summarization.
The accuracy of retrieval results can be improved by ‘implicit’ relevance feedback based on the inspector’s navigation pattern (e.g., user clicks, dwell time), as well as image annotations which are integral part to the video review process. We can use this relevance information to re-rank video parts by image feature re-weighting. Machine learning algorithms can be used to determine refined feature weights whereby events recommended for review become closer to those under analysis.
Finally, a smart interface to interrogate summaries would avoid zero-result queries by employing a faceted-search approach similar to that used in e-commerce sites: at any point during the review, the interface would show, in a look-ahead perspective, the distribution of ranked events to elicit the inspector’s interest and possible follow-up.
| Topics | TEC2 |
|---|---|
| Which "Key Question" does your Abstract address? | TEC2.8 |
| Which alternative "Key Question" does your Abstract address? (if any) | TEC2.5 |
Primary author
Mr
Fabrizio Ghiringhelli
(European Commission - Joint Research Centre)
Co-author
Ms
Cristina Versino
(European Commission - Joint Research Centre)
