Kyu S. Han

Jul 19, 2024 | amazon.science | Chandrashekhar Lavania |Prashant Mathur |Kyu S. Han |Georgios Chochlakis

Audio-visual representations leverage information from both modalities to produce joint representations. Such representations have demonstrated their usefulness in a variety of tasks. However, both modalities incorporated in the learned model might not necessarily be present all the time during inference. In this work, we study whether and how we can make exist- ing models, trained under pristine conditions, robust to partial modality loss without retraining them.