1
LikeLines: Collecting Timecode-level Feedback for Web Videos through User Interactions Raynor Vliegendhart, Martha Larson, and Alan Hanjalic 20 th ACM international conference on Multimedia, Nara, Japan, 2012 Future Work For what kinds of video is timecode-level feedback useful? How should user interactions be interpreted? How to fuse timecode-level feedback with content analysis without encouraging snowball effects? Can timecode-level data be linked to queries to recommend relevant jump points? How to collect a critical mass of timecode-level data by incentivizing users to interact with the system? Contact: [email protected] Implementation Video player component implemented in JavaScript and HTML5. Out-of-the-box support for YouTube and HTML5 videos. Video player component communicates with a back-end server using JSON(P). Back-end server reference implementation is written in Python. Multimedia Information Retrieval Lab, Delft University of Technology @ShinNoNoir Problem Problem: Providing users with a navigable heat map of interesting regions of the video they are watching. Motivation: Conventional time sliders do not make the inner structure of the video apparent, making it hard to navigate to the interesting bits. Approach A Web video player component with a navigable heat map, that: Uses multimedia content analysis to seed the heat map. Captures implicit and explicit user feedback at the timecode-level to refine the heat map. System Overview Video player component, augmented with: Navigable heat map that allows users to jump directly to “hot” areas; Time-sensitive “like” button that allows users to explicitly like particular points in the video. Captures user interactions: Implicit feedback such as playing, pausing and seeking; Explicit “likes” expressed by the user. Combines content analysis and captured user interactions to compute a video’s heat map. Back-end interaction session server stores and aggregates per video: All interaction sessions between each user and player; Initial multimedia content analysis of the video. Source code: https://github.com/delftmir/likelines-player viewers content analysis LikeLines player play pause seek like ... interaction session server content analysis t (s) user feedback 1 t (s) user feedback n t (s) + + + < script type = "text/ javascript " > var player = new LikeLines.Player ( 'playerDiv ' , { video : 'http://www.youtube.com/ watch?v =wPTilA0XxYE' , backend : 'http:// backend:9090/' }); </ script >

ACMMM12 - LikeLines: Collecting Timecode-level Feedback for Web Videos through User Interactions

Embed Size (px)

DESCRIPTION

Poster of technical demo presented at ACM Multimedia 2012

Citation preview

Page 1: ACMMM12 - LikeLines: Collecting Timecode-level Feedback for Web Videos through User Interactions

LikeLines: Collecting Timecode-level Feedback

for Web Videos through User Interactions

Raynor Vliegendhart, Martha Larson, and Alan Hanjalic

20th ACM international conference on Multimedia, Nara, Japan, 2012

Future Work

● For what kinds of video is timecode-level feedback useful?

● How should user interactions be interpreted?

● How to fuse timecode-level feedback with content analysis

without encouraging snowball effects?

● Can timecode-level data be linked to queries to recommend

relevant jump points?

● How to collect a critical mass of timecode-level data by

incentivizing users to interact with the system?

Contact: [email protected]

Implementation

● Video player component implemented in JavaScript and HTML5.

● Out-of-the-box support for YouTube and HTML5 videos.

● Video player component communicates with a back-end server

using JSON(P).

● Back-end server reference implementation is written in Python.

Multimedia Information Retrieval Lab, Delft University of Technology

@ShinNoNoir

Problem

● Problem: Providing users with a navigable heat map of interesting

regions of the video they are watching.

● Motivation: Conventional time sliders do not make the inner structure

of the video apparent, making it hard to navigate to the interesting bits.

Approach

A Web video player component with a navigable heat map, that:

● Uses multimedia content analysis to seed the heat map.

● Captures implicit and explicit user feedback at the timecode-level

to refine the heat map.

System Overview

● Video player component, augmented with:

● Navigable heat map that allows users to jump directly to “hot”

areas;

● Time-sensitive “like” button that allows users to explicitly like

particular points in the video.

● Captures user interactions:

● Implicit feedback such as playing, pausing and seeking;

● Explicit “likes” expressed by the user.

● Combines content analysis and captured user interactions to compute

a video’s heat map.

● Back-end interaction session server stores and aggregates per video:

● All interaction sessions between each user and player;

● Initial multimedia content analysis of the video.

Source code:

https://github.com/delftmir/likelines-player

viewers

content analysis

LikeLines player

play

pause

seek

like

... interaction

session server

content analysis

t (s)

user feedback 1

t (s)

user feedback n

t (s) + + + …

<script type="text/javascript">

var player = new LikeLines.Player('playerDiv', {

video: 'http://www.youtube.com/watch?v=wPTilA0XxYE',

backend: 'http://backend:9090/'

});

</script>