Hongxu Yin

May 3, 2024 | developer.nvidia.com | Yao Lu |Hongxu Yin |Ji Lin |Dustin Franklin

VILA is a family of high-performance vision language models developed by NVIDIA Research and MIT. The largest model comes with ~40B parameters and the smallest model comes with ~3B parameters. It is fully open source (including model checkpoints and even training code and training data). In this post, we describe how VILA performs against other models to deliver edge AI 2.0. Initial versions of edge AI involved deploying compressed AI models onto edge devices.

May 3, 2024 | developer.nvidia.com | Yao Lu |Hongxu Yin |Ji Lin |Pavlo Molchanov

Visual language models have evolved significantly recently. However, the existing technology typically only supports one single image. They cannot reason among multiple images, support in context learning or understand videos. Also, they don’t optimize for inference speed. We developed VILA, a visual language model with a holistic pretraining, instruction tuning, and deployment pipeline that helps our NVIDIA clients succeed in their multi-modal products.