speech-5

Visual Keyword Spotting

3260 papers • 126 benchmarks • 313 datasets

Spot a given query keyword in a silent talking face video

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in visual-keyword-spotting-11

Trend

Dataset

Best Model

Actions

LRS2

LRS3-TED

LRW

Libraries

i

Use these libraries to find visual-keyword-spotting-11 models and implementations

Datasets

LRW

LRS2

LRS3-TED

Subtasks

No subtasks available.

Most implemented papers

Zero-shot keyword spotting for visual speech recognition in-the-wild

Georgios Tzimiropoulos, Themos Stafylakis•Sun Jul 22 2018

This paper proposes the use of a grapheme-to-phoneme encoder-decoder model which learns how to map words to their pronunciation, and demonstrates that this system obtains very promising visual-only KWS results on the challenging LRS2 database, for keywords unseen during training.

40

Content

0

Paper Graph

LipLearner: Customizable Silent Speech Interactions on Mobile Devices

Zixiong Su, Shitao Fang, J. Rekimoto•Sat Feb 11 2023

LipLearner leverages contrastive learning to learn efficient lipreading representations, enabling few-shot command customization with minimal user effort, and exhibits high robustness to different lighting, posture, and gesture conditions on an in-the-wild dataset.

36 0

Paper Graph

Seeing wake words: Audio-visual Keyword Spotting

Andrew Zisserman, Triantafyllos Afouras, Liliane Momeni, Samuel Albanie, Themos Stafylakis•Tue Sep 01 2020

A novel convolutional architecture, KWS-Net, that uses a similarity map intermediate representation to separate the task into sequence matching, and pattern detection, to decide whether and when a word of interest is spoken by a talking face, with or without the audio.

47 0

Paper Graph

Visual Keyword Spotting with Attention

Andrew Zisserman, Triantafyllos Afouras, Prajwal K R, Liliane Momeni•Thu Oct 28 2021

A novel architecture is proposed, the Transpotter, that uses full cross-modal attention between the visual and phonetic streams in Transformer-based models that ingest two streams, a visual encoding of the video and a phonetic encoding of a keyword, and output the temporal location of the keyword if present.

16 0

Paper Graph

Adding a benchmark result helps the community track progress.