natural-language-processing-5

Grounded language learning

3260 papers • 126 benchmarks • 313 datasets

Acquire the meaning of language in situated environments.

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in grounded-language-learning-5

Trend

Dataset

Best Model

Actions

No benchmarks available.

Libraries

i

Use these libraries to find grounded-language-learning-5 models and implementations

Datasets

IGLU

Subtasks

No subtasks available.

Most implemented papers

Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

S. Hoi, Akhilesh Deepak Gotmare, Junnan Li, Ramprasaath R. Selvaraju, Shafiq R. Joty, Caiming Xiong•Thu Jul 15 2021

A contrastive loss to ALign the image and text representations BEfore Fusing (ALBEF) them through cross-modal attention, which enables more grounded vision and language representation learning and proposes momentum distillation, a self-training method which learns from pseudo-targets produced by a momentum model.

2513

Content

0

Paper Graph

Pragmatically Informative Text Generation

Sheng Shen, Jacob Andreas, D. Klein, Daniel Fried•Mon Apr 01 2019

This work considers two pragmatic modeling methods for text generation: one where pragmatics is imposed by information preservation, and another where prag matics isimposed by explicit modeling of distractors.

69 0

Paper Graph

Visually Grounded Continual Learning of Compositional Phrases

R. Nevatia, Xiang Ren, Arka Sadhu, Xisen Jin, Junyi Du•Fri May 01 2020

VisCOLL, a visually grounded language learning task, which simulates the continual acquisition of compositional phrases from streaming visual scenes, and reveals that SoTA continual learning approaches provide little to no improvements on VisCOLL.

17 0

Paper Graph

Grounded Language Learning Fast and Slow

Felix Hill, O. Tieleman, Tamara von Glehn, Nathaniel Wong, Hamza Merzic, S. Clark•Wed Sep 02 2020

The results demonstrate that deep neural networks can exploit meta-learning, episodic memory and an explicitly multi-modal environment to account for 'fast-mapping', a fundamental pillar of human cognitive development and a potentially transformative capacity for agents that interact with human users.

84 0

Paper Graph

Grounded Language Learning in a Simulated 3D World

D. Hassabis, Karl Moritz Hermann, Phil Blunsom, Hubert Soyer, Max Jaderberg, Wojciech M. Czarnecki, Felix Hill, Denis Teplyashin, Simon Green, Fumin Wang, David Szepesvari, Marcus Wainwright, C. Apps•Mon Jun 19 2017

An agent is presented that learns to interpret language in a simulated 3D environment where it is rewarded for the successful execution of written instructions and its comprehension of language extends beyond its prior experience, enabling it to apply familiar language to unfamiliar situations and to interpret entirely novel instructions.

311 0

Paper Graph

Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game

W. Xu, Haonan Yu•Wed Apr 25 2018

A joint imitation and reinforcement approach for grounded language learning through an interactive conversational game is proposed and the agent trained is able to actively acquire information by asking questions about novel objects and use the just-learned knowledge in subsequent conversations in a one-shot fashion.

8 0

Paper Graph

A new dataset and model for learning to understand navigational instructions

Ozan Arkan Can, Deniz Yuret•Sun May 20 2018

A model that can learn to follow new instructions given prior instruction-perception-action examples and achieves the best results to date on the SAIL dataset by using an improved perceptual component that can represent relative positions of objects.

1 0

Paper Graph

Lessons Learned in Multilingual Grounded Language Learning

Marc-Alexandre Côté, Desmond Elliott, Ákos Kádár, Grzegorz Chrupała, A. Alishahi•Fri Aug 31 2018

It is demonstrated that a multilingual model can be trained equally well on either translations or comparable sentence pairs, and that annotating the same set of images in multiple language enables further improvements via an additional caption-caption ranking objective.

26 0

Paper Graph

Learning Latent Semantic Annotations for Grounding Natural Language to Structured Data

Guanghui Qin, Jin-ge Yao, Xuening Wang, Jinpeng Wang, Chin-Yew Lin•Sun Dec 31 2017

This paper attempt at learning explicit latent semantic annotations from paired structured tables and texts, establishing correspondences between various types of values and texts with an adapted semi-hidden Markov model.

10 0

Paper Graph

Adding a benchmark result helps the community track progress.