3d-shape-generation

Gesture Generation

3260 papers • 126 benchmarks • 313 datasets

Generation of gestures, as a sequence of 3d poses

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in 3d-shape-generation

Trend

Dataset

Best Model

Actions

TED Gesture Dataset

BEAT2

BEAT

Libraries

i

Use these libraries to find 3d-shape-generation models and implementations

youngseng/diffusestylegesture

2 papers 117

Datasets

Talking With Hands 16.2M

BiGe

Subtasks

No subtasks available.

Most implemented papers

robosuite: A Modular Simulation Framework and Benchmark for Robot Learning

Yuke Zhu, Ajay Mandlekar, Roberto Mart'in-Mart'in, Josiah Wong•Thu Sep 24 2020

The key system modules and the benchmark environments of the new release robosuite v1.0 are discussed.

575

Content

DVS128 Gesture

0

Paper Graph

Learning Individual Styles of Conversational Gesture

Jitendra Malik, Caroline Chan, Shiry Ginosar, Amir Bar, Andrew Owens, Gefen Kohavi•Fri May 31 2019

A method for cross-modal translation from "in-the-wild" monologue speech of a single speaker to their conversational gesture motion is presented and significantly outperforms baseline methods in a quantitative comparison.

372 0

Paper Graph

BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis

Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, E. Bozkurt, Bo Zheng•Wed Mar 09 2022

The statistical analysis on BEAT demonstrates the correlation of conversational gestures with facial expressions, emotions, and semantics, in addition to the known correlation with audio, text, and speaker identity, and proposes a baseline model, Cascaded Motion Network (CaMN), which consists of above six modalities modeled in a cascaded architecture for gesture synthesis.

194 0

Paper Graph

The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation

Youngwoo Yoon, G. Henter, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov•Sun Aug 21 2022

This paper reports on the second GENEA Challenge to benchmark data-driven automatic co-speech gesture generation, using the same speech and motion dataset to build gesture-generation systems to benchmark human-likeness from gesture appropriateness.

98 0

Paper Graph

Generating Holistic 3D Human Motion from Speech

Yandong Wen, Michael J. Black, Timo Bolkart, Hongwei Yi, Qiong Cao, Dacheng Tao, Hualin Liang, Yifei Liu•Wed Dec 07 2022

A novel speech-to-motion generation framework in which the face, body, and hands are modeled separately, and a cross-conditional autoregressive model that generates body poses and hand gestures, leading to coherent and realistic motions is proposed.

198 0

Paper Graph

Speech gesture generation from the trimodal context of text, audio, and speaker identity

Minsu Jang, Jaehong Kim, Youngwoo Yoon, Bok Cha, Joo-Haeng Lee, Geehyuk Lee•Thu Sep 03 2020

This paper presents an automatic gesture generation model that uses the multimodal context of speech text, audio, and speaker identity to reliably generate gestures that are human-like and that match with speech content and rhythm.

341 0

Paper Graph

EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling

Haiyang Liu, You Zhou, Zihao Zhu, Giorgio Becherini, Yichen Peng, Mingyang Su, Xuefei Zhe, Naoya Iwamoto, Bo Zheng, Michael J. Black•Sat Dec 30 2023

Experiments demonstrate that EMAGE generates holistic gestures with state-of-the-art performance and is flexible in accepting predefined spatial-temporal gesture inputs, generating complete, audio-synchronized results.

79 0

Paper Graph

The GENEA Challenge 2023: A large-scale evaluation of gesture generation models in monadic and dyadic settings

G. Henter, Taras Kucherenko, Teodor Nikolov, Mihail Tsakov, Rajmund Nagy, Youngwoo Yoon, Jieyeon Woo•Wed Aug 23 2023

A large span in human-likeness between challenge submissions is found, with a few systems rated close to human mocap, and a dyadic system being highly appropriate for agent speech does not necessarily imply high appropriateness for the interlocutor.

60 0

Paper Graph

Analyzing Input and Output Representations for Speech-Driven Gesture Generation

H. Kjellström, G. Henter, Taras Kucherenko, Dai Hasegawa, Naoshi Kaneko•Thu Mar 07 2019

A novel framework for automatic speech-driven gesture generation, applicable to human-agent interaction including both virtual agents and robots, using a denoising autoencoder neural network and a novel encoder network.

168 0

Paper Graph

Gesticulator: A framework for semantically-aware speech-driven gesture generation

G. Henter, Simon Alexanderson, Taras Kucherenko, Patrik Jonell, S. V. Waveren, Iolanda Leite, Hedvig Kjellstrom•Fri Jan 24 2020

This work presents a model designed to produce arbitrary beat and semantic gestures together, which takes both acoustic and semantic representations of speech as input, and generates gestures as a sequence of joint angle rotations as output.

198 0

Paper Graph

Adding a benchmark result helps the community track progress.