Motion Style Transfer: Modular Low-Rank Adaptation for Deep Motion Forecasting

Published in

Conference on Robot Learning(2022)

External Links:

Generate Graph DownloadPDF

TL;DR

This work proposes a transfer learning approach for efficiently adapting pre-trained forecasting models to new domains, such as unseen agent types and scene contexts, and introduces two components that exploit prior knowledge of motion style shifts, including a low-rank motion style adapter and a modular adapter strategy.

Abstract

Deep motion forecasting models have achieved great success when trained on a massive amount of data. Yet, they often perform poorly when training data is limited. To address this challenge, we propose a transfer learning approach for efficiently adapting pre-trained forecasting models to new domains, such as unseen agent types and scene contexts. Unlike the conventional fine-tuning approach that updates the whole encoder, our main idea is to reduce the amount of tunable parameters that can precisely account for the target domain-specific motion style. To this end, we introduce two components that exploit our prior knowledge of motion style shifts: (i) a low-rank motion style adapter that projects and adjusts the style features at a low-dimensional bottleneck; and (ii) a modular adapter strategy that disentangles the features of scene context and motion history to facilitate a fine-grained choice of adaptation layers. Through extensive experimentation, we show that our proposed adapter design, coined MoSA, outperforms prior methods on several forecasting benchmarks.

Authors

Alexandre Alahi

17 papers

Yuejiang Liu

4 papers

Parth Kothari

2 papers

References45 items

Safety-Compliant Generative Adversarial Networks for Human Trajectory Forecasting

SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Driving

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

Adaptive Trajectory Prediction via Transferable GNN

Transferable and Adaptable Driving Behavior Prediction

Motion Style Transfer: Modular Low-Rank Adaptation for Deep Motion Forecasting

Published in

Conference on Robot Learning(2022)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Alexandre Alahi

17 papers

Yuejiang Liu

4 papers

Parth Kothari

2 papers

References45 items

Safety-Compliant Generative Adversarial Networks for Human Trajectory Forecasting

SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Driving

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

Adaptive Trajectory Prediction via Transferable GNN

Transferable and Adaptable Driving Behavior Prediction

Vehicle trajectory prediction works, but not everywhere

Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective

Are socially-aware trajectory prediction models really socially-aware?

LoRA: Low-Rank Adaptation of Large Language Models

Compacter: Efficient Low-Rank Hypercomplex Adapter Layers

Interpretable Social Anchors for Human Trajectory Forecasting in Crowds

Social NCE: Contrastive Learning of Socially-aware Motion Representations

From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Deep Visual Domain Adaptation

Human Trajectory Forecasting in Crowds: A Deep Learning Perspective

Few-shot Human Motion Prediction via Learning Novel Motion Dynamics

One Thousand and One Hours: Self-driving Motion Prediction Dataset

SimAug: Learning Robust Representations from Simulation for Trajectory Prediction

Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction

Multi-source Domain Adaptation in the Deep Learning Era: A Systematic Survey

Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data

The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersections

Parameter-Efficient Transfer Learning for NLP

Residual Reinforcement Learning for Robot Control

Universal Sentence Encoder for English

Few-Shot Human Motion Prediction via Meta-learning

Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks

Efficient Parametrization of Multi-domain Deep Neural Networks

Deep Visual Domain Adaptation: A Survey

Universal Language Model Fine-tuning for Text Classification

Social Attention: Modeling Attention in Human Crowds

Modulating early visual processing by language

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

COCO-Stuff: Thing and Stuff Classes in Context

Learning Social Etiquette: Human Trajectory Understanding In Crowded Scenes

Social LSTM: Human Trajectory Prediction in Crowded Spaces

Revisiting Batch Normalization For Practical Domain Adaptation

Distributed Representations of Words and Phrases and their Compositionality

Discrete Choice Models for Pedestrian Walking Behavior

Social force model for pedestrian dynamics.

Motion Forecasting with Unlikelihood Training in Continuous Space

Improving Language Understanding by Generative Pre-Training

Long-term path prediction in urban scenarios using circular distributions

following the setup in [2]. The adaptation uses N target = {20} for fine-tuning and 20 trajectories for validation

Field of Study

Computer Science

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

Conference on Robot Learning

Type

conference

URL

N/A

Alternate Names

CoRL
Conf Robot Learn