Edge Weight Prediction For Category-Agnostic Pose Estimation (2024-11-25T00:00:00.000000Z)

TL;DR

This work proposes integrating Markovian Structural Bias, which modulates the self-attention interaction between nodes based on the number of hops between them, and shows that this improves the model's ability to capture global spatial dependencies.

Abstract

Category-Agnostic Pose Estimation (CAPE) localizes keypoints across diverse object categories with a single model, using one or a few annotated support images. Recent works have shown that using a pose graph (i.e., treating keypoints as nodes in a graph rather than isolated points) helps handle occlusions and break symmetry. However, these methods assume a static pose graph with equal-weight edges, leading to suboptimal results. We introduce EdgeCape, a novel framework that overcomes these limitations by predicting the graph's edge weights which optimizes localization. To further leverage structural priors, we propose integrating Markovian Structural Bias, which modulates the self-attention interaction between nodes based on the number of hops between them. We show that this improves the model's ability to capture global spatial dependencies. Evaluated on the MP-100 benchmark, which includes 100 categories and over 20K images, EdgeCape achieves state-of-the-art results in the 1-shot setting and leads among similar-sized methods in the 5-shot setting, significantly improving keypoint localization accuracy. Our code is publicly available.

Authors

S. Avidan

7 papers

Or Hirschorn

2 papers

TL;DR

Abstract

Authors

References62 items

Recurrent Feature Mining and Keypoint Mixup Padding for Category-Agnostic Pose Estimation

CapeLLM: Support-Free Category-Agnostic Pose Estimation with Multimodal Large Language Models

SCAPE: A Simple and Strong Category-Agnostic Pose Estimator

Dynamic Support Information Mining for Category-Agnostic Pose Estimation

ESCAPE: Encoding Super-keypoints for Category-Agnostic Pose Estimation

CapeX: Category-Agnostic Pose Estimation from Textual Point Explanation

Probing the 3D Awareness of Visual Foundation Models

Meta-Point Learning and Refining for Category-Agnostic Pose Estimation

A Graph-Based Approach for Category-Agnostic Pose Estimation

Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence

DETR Does Not Need Multi-Scale or Locality Design

NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification

Matching Is Not Enough: A Two-Stage Framework for Category-Agnostic Pose Estimation

Graph Inductive Biases in Transformers without Message Passing

A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence

DINOv2: Learning Robust Visual Features without Supervision

Adding Conditional Control to Text-to-Image Diffusion Models

EVA: Exploring the Limits of Masked Visual Representation Learning at Scale

InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions

AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time

Pose for Everything: Towards Category-Agnostic Pose Estimation

APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking

Recipe for a General, Powerful, Scalable Graph Transformer

AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints

Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting

DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection

Weisfeiler and Leman Go Infinite: Spectral and Combinatorial Pre-Colorings

Representing Long-Range Context for Graph Neural Networks with Global Attention

High-Resolution Image Synthesis with Latent Diffusion Models

Deep ViT Features as Dense Visual Descriptors

Gophormer: Ego-Graph Transformer for Node Classification

AP-10K: A Benchmark for Animal Pose Estimation in the Wild

Human Pose Regression with Residual Log-likelihood Estimation

GraphiT: Encoding Graph Structure in Transformers

Rethinking Graph Transformers with Spectral Attention

Emerging Properties in Self-Supervised Vision Transformers

Vision Transformers for Dense Prediction

TransPose: Keypoint Localization via Transformer

A Generalization of Transformer Networks to Graphs

Deformable DETR: Deformable Transformers for End-to-End Object Detection

End-to-End Object Detection with Transformers

RAFT: Recurrent All-Pairs Field Transforms for Optical Flow

Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving

CarFusion: Combining Point Tracking and Part Detection for Dynamic 3D Reconstruction of Vehicles

Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition

Cascade R-CNN: Delving Into High Quality Object Detection

Graph Attention Networks

DeepPrior++: Improving Fast and Accurate 3D Hand Pose Estimation

Learning to Estimate 3D Hand Pose from Single RGB Images

Semi-Supervised Classification with Graph Convolutional Networks

What Makes for Effective Detection Proposals?

Edge Boxes: Locating Object Proposals from Edges

Articulated Human Detection with Flexible Mixtures of Parts

Measuring the Objectness of Image Windows

Harnessing Text-to-Image Diffusion Models for Category-Agnostic Pose Estimation

ViTPose+: Vision Transformer Foundation Model for Generic Body Pose Estimation

Do Transformers Really Perform Badly for Graph Representation?

MMPose Contributors

X-pose: Detection any keypoints

Weak-shot keypoint estimation via keyness and correspondence transfer

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names