reasoning-2

Systematic Generalization

3260 papers • 126 benchmarks • 313 datasets

This task has no description! Would you like to contribute one?

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in systematic-generalization-2

Trend

Dataset

Best Model

Actions

No benchmarks available.

Libraries

i

Use these libraries to find systematic-generalization-2 models and implementations

LauraRuis/groundedSCAN

2 papers 64

Datasets

Subtasks

No subtasks available.

Most implemented papers

A Benchmark for Systematic Generalization in Grounded Language Understanding

B. Lake, Marco Baroni, Laura Ruis, Jacob Andreas, Diane Bouchacourt•Tue Mar 10 2020

A new benchmark, gSCAN, is introduced for evaluating compositional generalization in models of situated language understanding, taking inspiration from standard models of meaning composition in formal linguistics and defining a language grounded in the states of a grid world.

152

Content

S2B

0

Paper Graph

Multi-Object Representation Learning with Iterative Variational Inference

M. Botvinick, Daniel Zoran, Christopher P. Burgess, L. Matthey, Nicholas Watters, Rishabh Kabra, Alexander Lerchner, Klaus Greff, Raphael Lopez Kaufman•Thu Feb 28 2019

This work argues for the importance of learning to segment and represent objects jointly, and demonstrates that, starting from the simple assumption that a scene is composed of multiple entities, it is possible to learn to segment images into interpretable objects with disentangled representations.

546 0

Paper Graph

CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text

Joelle Pineau, Koustuv Sinha, William L. Hamilton, Shagun Sodhani, Jin Dong•Wed Jul 31 2019

A diagnostic benchmark suite, named CLUTRR, is introduced to clarify some key issues related to the robustness and systematicity of NLU systems, and highlights a substantial performance gap between state-of-the-art NLU models.

246 0

Paper Graph

Prioritized Level Replay

Edward Grefenstette, Tim Rocktäschel, Minqi Jiang•Wed Oct 07 2020

This work introduces Prioritized Level Replay, a general framework for estimating the future learning potential of a level given the current state of the agent's policy, and finds that temporal-difference errors, while previously used to selectively sample past transitions, also prove effective for scoring a level's futurelearning potential in generating entire episodes that an agent would experience when replaying it.

198 0

Paper Graph

CLOSURE: Assessing Systematic Generalization of CLEVR Models

Yoshua Bengio, Aaron C. Courville, Philippe Beaudoin, H. D. Vries, Dzmitry Bahdanau, Shikhar Murty, Timothy J. O'Donnell•Wed Dec 11 2019

Surprisingly, it is found that an explicitly compositional Neural Module Network model also generalizes badly on CLOSURE, even when it has access to the ground-truth programs at test time.

95 0

Paper Graph

The NetHack Learning Environment

Nantas Nardelli, Edward Grefenstette, Heinrich Kuttler, Alexander H. Miller, R. Raileanu, Marco Selvatici, Tim Rocktäschel•Sun May 31 2020

It is argued that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL, while dramatically reducing the computational resources required to gather a large amount of experience.

209 0

Paper Graph

Conditional Object-Centric Learning from Video

Alexey Dosovitskiy, Gamaleldin F. Elsayed, G. Heigold, S. Sabour, Aravindh Mahendran, Thomas Kipf, Klaus Greff, Austin Stone, Rico Jonschkowski•Tue Nov 23 2021

This paper introduces a sequential extension to Slot Attention which is trained to predict optical flow for realistic looking synthetic scenes and shows that conditioning the initial state of this model on a small set of hints is sufficient to significantly improve instance segmentation.

267 0

Paper Graph

Revisit Systematic Generalization via Meaningful Learning

Zhouhan Lin, Ning Shi, Boxin Wang, Wei Wang, Xiangyu Liu•Fri Mar 13 2020

It is suggested that models can successfully one-shot generalize to novel concepts and compositions through semantic linking, either inductively or deductively, and it is demonstrated that prior knowledge plays a key role as well.

2 0

Paper Graph

Systematic Generalization on gSCAN with Language Conditioned Embedding

Qi Huang, R. Mooney, Tong Gao•Thu Sep 10 2020

This model is the first one that significantly outperforms the provided baseline and reaches state-of-the-art performance on grounded SCAN, a grounded natural language navigation dataset designed to require systematic generalization in its test splits.

22 0

Paper Graph

Adding a benchmark result helps the community track progress.