Another Open Source Framework Of Facebook Pythia is Used For Image And Language Modelling!

By Jyoti Nigania |Email | May 25, 2019 | 2223 Views

Facebook's commitment to the wider dev community remains as strong as ever if recent developments are any indication. Following the open-sourcing of image processing library Spectrum in January, natural language processing modeling framework PyText late last year, and AI reinforcement learning platform Horizon in November, Facebook's AI research division today announced that Pythia, a modular plug-and-play framework that enables data scientists to quickly build, reproduce, and benchmark AI models, is now freely available on GitHub.

As Facebook explains in a blog post, Pythia which is built atop the company's PyTorch machine learning framework is principally intended for vision and language tasks, such as answering questions related to visual data and automatically generating image captions. It incorporates elements of Facebook AI Research's top entries in AI competitions like LoRRA, a vision, and language model that won both the VQA Challenge 2018 and Vizwiz Challenge 2018, and it's capable of showing how previous state-of-the-art AI systems achieved top benchmark results and comparing their performance to that of new models.

Pythia also supports distributed training and a variety of data sets, as well as custom losses, metrics, scheduling, and optimizers. Modules with implementations for commonly-used vision and language layers are present and accounted for, as is support for distributed training; built-in corpora including VQA, VizWiz, TextVQA, and VisualDialog; and features like multitasking, which allows training on multiple corpora together.

Facebook says that it'll expand on Pythia's initial open source release with forthcoming tools, tasks, data sets, and reference models.

Pythia smooths the process of entering the growing subfield of vision and language and frees researchers to focus on faster prototyping and experimentation. Our goal is to accelerate progress by increasing the reproducibility of these models and results, wrote Facebook in a blog post. This will make it easier for the community to build on, and benchmark against, successful systems. 

We hope that removing some of the obstacles will allow researchers to more quickly develop new ways for people and intelligent machines to communicate. This work should also help researchers develop adaptive AI that synthesizes multiple kinds of understanding into a more context-based, multimodal understanding.

Pythia's debut comes just weeks after the unveiling of PyTorch 1.1 during Facebook's F8 developer summit in San Francisco, which added support for Google's TensorBoard and performance improvements from its just-in-time (JIT) compiler. GitHub's 2018 Octoverse report last fall named PyTorch one of the most popular open source projects on GitHub, with over 31 million developers worldwide.

Source: HOB