Loading...
CityU Talk
Information Theory and Multi-Modal Learning
Lizhong Zheng October, 2020
@ CityU HK
What is Multi-Modal Learning?
Modalities:
Human Senses:
Sensor Measurements:
Attributes:
Array Elements:
Applications:
Audio-visual speech recognition
Multi-media information retrieval
Image/Video captioning
Emotion recognition
5 Core Challenges
(Baltrusaitis,
Morency’17):
Representation
Alignment
Fusion
Translation
Co-Learning
Why is this an Information Theory Problem?
Multi-Variate Dependence
To measure the dependence between
x
{\mathsf x}
x
and
y
{\mathsf y}
y
: mutual information, HGR correlation
Three random variables?
Core Problem of Multi-Terminal Information Theory
Broadcast channel
The CEO problem
Secret key generation
Please turn on JavaScript to use Paper in all of its awesomeness. ^_^
Information Theory and Multi-Modal Learning
What is Multi-Modal Learning?
Why is this an Information Theory Problem?