Kate Saenko "Why did you say that? Explaining neural models of vision and language"

What:
Talk
When:
Friday Feb 02   02:30 PM to 03:50 PM (1 hour 20 minutes)
Where:
Doherty Hall - DH 2315
Discussion:
0

WATCH VIDEO OF LECTURE ► [Andrew ID login required to view video]


TITLE

Why did you say that? Explaining neural models of vision and language

ABSTRACT
Recent advances in neural models of Vision & Language have produced exciting new results in image and video captioning, visual question answering and other related tasks. For example, our previous work has shown that sequence-to-sequence networks can generate fluent captions for input video clips that describe the main activity happening in the video. However, our understanding of how a neural network arrives at a particular caption or question answer is very limited. Despite their excellent performance, neural nets are inherently much less explainable than some of the previously popular machine learning models for these tasks. In my talk, I will first present some of the neural models we have developed in my lab for Vision & Language problems and then discuss our recent work that tries to "look under the hood" of these networks and explain their decisions.

BIO
Kate Saenko is an Assistant Professor at the Department of Computer Science at Boston University, and the director of the Computer Vision and Learning Group and member of the IVC Group. She received her PhD from MIT. Previously, she was an Assistant Professor at the Department of Computer Science at UMass Lowell, a Postdoctoral Researcher at the International Computer Science Institute, a Visiting Scholar at UC Berkeley EECS and a Visiting Postdoctoral Fellow in the School of Engineering and Applied Science at Harvard University. Her research interests are in the broad area of Artificial Intelligence with a focus on Adaptive Machine Learning, Learning for Vision and Language Understanding, and Deep Learning.

Participant
Boston University
Assistant Professor

My Schedule

Add to Your Schedule