Time-Contrastive Networks: Self-Supervised Learning from Video

Published in

IEEE International Conference on Robotics and A...(2017)

External Links:

Generate Graph DownloadPDF

TL;DR

A self-supervised approach for learning representations and robotic behaviors entirely from unlabeled videos recorded from multiple viewpoints is proposed, and it is demonstrated that this representation can be used by a robot to directly mimic human poses without an explicit correspondence, and that it can be use as a reward function within a reinforcement learning algorithm.

Abstract

We propose a self-supervised approach for learning representations and robotic behaviors entirely from unlabeled videos recorded from multiple viewpoints, and study how this representation can be used in two robotic imitation settings: imitating object interactions from videos of humans, and imitating human poses. Imitation of human behavior requires a viewpoint-invariant representation that captures the relationships between end-effectors (hands or robot grippers) and the environment, object attributes, and body pose. We train our representations using a triplet loss, where multiple simultaneous viewpoints of the same observation are attracted in the embedding space, while being repelled from temporal neighbors which are often visually similar but functionally different. This signal causes our model to discover attributes that do not change across viewpoint, but do change across time, while ignoring nuisance variables such as occlusions, motion blur, lighting and background. We demonstrate that this representation can be used by a robot to directly mimic human poses without an explicit correspondence, and that it can be used as a reward function within a reinforcement learning algorithm. While representations are learned from an unlabeled collection of task-related videos, robot behaviors such as pouring are learned by watching a single 3rd-person demonstration by a human. Reward functions obtained by following the human demonstrations under the learned representation enable efficient reinforcement learning that is practical for real-world robotic systems. Video results, open-source code and dataset are available at sermanet.github.io/imitate

Authors

Time-Contrastive Networks: Self-Supervised Learning from Video

Published in

IEEE International Conference on Robotics and A...(2017)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

S. Levine

59 papers

Eric Jang

3 papers

P. Sermanet

6 papers

Corey Lynch

2 papers

Yevgen Chebotar

2 papers

Jasmine Hsu

2 papers

S. Schaal

3 papers

References57 items

Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation

One-Shot Imitation Learning

Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learning

Third-Person Imitation Learning

Unsupervised Perceptual Rewards for Imitation Learning

Learning Features by Watching Objects Move

Improved Deep Metric Learning with Multi-class N-pair Loss Objective

Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

SoundNet: Learning Sound Representations from Unlabeled Video

Generic 3D Representation via Pose Estimation and Matching

Path integral guided policy search

Label-Free Supervision of Neural Networks with Physics and Domain Knowledge

Adversarially Learned Inference

LIFT: Learned Invariant Feature Transform

Unsupervised Learning using Sequential Verification for Action Recognition

Understanding Visual Concepts with Continuation Learning

Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimizing Global Loss Functions

Visually Indicated Sounds

Local Convolutional Features with Unsupervised Training for Image Retrieval

Discriminative Learning of Deep Convolutional Feature Point Descriptors

Rethinking the Inception Architecture for Computer Vision

Deep Metric Learning via Lifted Structured Feature Embedding

Deep multi-scale video prediction beyond mean square error

Learning Visual Feature Spaces for Robotic Manipulation with Deep Spatial Autoencoders

Pose Embeddings: A Deep Architecture for Learning to Match Human Poses

Unsupervised Visual Representation Learning by Context Prediction

Learning to compare image patches via convolutional neural networks

FaceNet: A unified embedding for face recognition and clustering

Unsupervised Learning of Spatiotemporally Coherent Metrics

Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics

Synthesis and stabilization of complex behaviors through online trajectory optimization

Online customization of teleoperation interfaces

Learning to select and generalize striking movements in robot table tennis

View-Based Encoding of Actions in Mirror Neurons of Area F5 in Macaque Premotor Cortex

A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

A Generalized Path Integral Control Approach to Reinforcement Learning

ImageNet: A large-scale hierarchical image database

Learning and generalization of motor skills by learning from demonstration

Deep Boltzmann Machines

Extracting and composing robust features with denoising autoencoders

Imitation learning for locomotion and manipulation

On Learning, Representing, and Generalizing a Task in a Humanoid Robot

Imitation: is cognitive neuroscience solving the correspondence problem?

The mirror-neuron system.

Apprenticeship learning via inverse reinforcement learning

Movement imitation with nonlinear dynamical systems in humanoid robots

Slow Feature Analysis: Unsupervised Learning of Invariances

Efficient Training of Artificial Neural Networks for Autonomous Navigation

Robotics and autonomous systems

On Learning

Time-Contrastive Networks : Self-Supervised Learning from Pixels

pybullet, a python module for physics simulation in robotics, games and machine learning

and Martial Hebert

Unsupervised Learning of Visual Representations using Videos

Self-Supervised Video Representation Learning with Odd-One-Out Networks

the second step, the residual cost-to-go S ( x t , u t ) − ˆ S ( x t , u t ) is optimized using a model-free PI 2 [48, 49] to produce an unbiased policy update

TL;DR

Abstract

Authors

TL;DR

Abstract

Authors

References57 items

Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation

One-Shot Imitation Learning

Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learning

Third-Person Imitation Learning

Unsupervised Perceptual Rewards for Imitation Learning

Learning Features by Watching Objects Move

Improved Deep Metric Learning with Multi-class N-pair Loss Objective

Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

SoundNet: Learning Sound Representations from Unlabeled Video

Generic 3D Representation via Pose Estimation and Matching

Path integral guided policy search

Label-Free Supervision of Neural Networks with Physics and Domain Knowledge

Adversarially Learned Inference

LIFT: Learned Invariant Feature Transform

Unsupervised Learning using Sequential Verification for Action Recognition

Understanding Visual Concepts with Continuation Learning

Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimizing Global Loss Functions

Visually Indicated Sounds

Local Convolutional Features with Unsupervised Training for Image Retrieval

Discriminative Learning of Deep Convolutional Feature Point Descriptors

Rethinking the Inception Architecture for Computer Vision

Deep Metric Learning via Lifted Structured Feature Embedding

Deep multi-scale video prediction beyond mean square error

Learning Visual Feature Spaces for Robotic Manipulation with Deep Spatial Autoencoders

Pose Embeddings: A Deep Architecture for Learning to Match Human Poses

Unsupervised Visual Representation Learning by Context Prediction

Learning to compare image patches via convolutional neural networks

FaceNet: A unified embedding for face recognition and clustering

Unsupervised Learning of Spatiotemporally Coherent Metrics

Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics

Synthesis and stabilization of complex behaviors through online trajectory optimization

Online customization of teleoperation interfaces

Learning to select and generalize striking movements in robot table tennis

View-Based Encoding of Actions in Mirror Neurons of Area F5 in Macaque Premotor Cortex

A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

A Generalized Path Integral Control Approach to Reinforcement Learning

ImageNet: A large-scale hierarchical image database

Learning and generalization of motor skills by learning from demonstration

Deep Boltzmann Machines

Extracting and composing robust features with denoising autoencoders

Imitation learning for locomotion and manipulation

On Learning, Representing, and Generalizing a Task in a Humanoid Robot

Imitation: is cognitive neuroscience solving the correspondence problem?

The mirror-neuron system.

Apprenticeship learning via inverse reinforcement learning

Movement imitation with nonlinear dynamical systems in humanoid robots

Slow Feature Analysis: Unsupervised Learning of Invariances

Efficient Training of Artificial Neural Networks for Autonomous Navigation

Robotics and autonomous systems

On Learning

Time-Contrastive Networks : Self-Supervised Learning from Pixels

pybullet, a python module for physics simulation in robotics, games and machine learning

and Martial Hebert

Unsupervised Learning of Visual Representations using Videos

Self-Supervised Video Representation Learning with Odd-One-Out Networks

the second step, the residual cost-to-go S ( x t , u t ) − ˆ S ( x t , u t ) is optimized using a model-free PI 2 [48, 49] to produce an unbiased policy update

a coloring by groundtruth labels. Each color is a unique combination of 5 attribute values deﬁned earlier, i.e. if each

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names