Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding (2022-04-18T00:00:00.000000Z)

TL;DR

This work creates a large and diverse dataset, Animal Kingdom, that provides multiple annotated tasks to enable a more thorough understanding of natural animal behaviors, and proposes a Collaborative Action Recognition model that learns general and specific features for action recognition with unseen new animals.

Abstract

Understanding animals' behaviors is significant for a wide range of applications. However, existing animal behavior datasets have limitations in multiple aspects, including limited numbers of animal classes, data samples and provided tasks, and also limited variations in environmental conditions and viewpoints. To address these limitations, we create a large and diverse dataset, Animal Kingdom, that provides multiple annotated tasks to enable a more thorough understanding of natural animal behaviors. The wild animal footages used in our dataset record different times of the day in extensive range of environments containing variations in backgrounds, viewpoints, illumination and weather conditions. More specifically, our dataset contains 50 hours of annotated videos to localize relevant animal behavior segments in long videos for the video grounding task, 30K video sequences for the fine-grained multi-label action recognition task, and 33K frames for the pose estimation task, which correspond to a diverse range of animals with 850 species across 6 major animal classes. Such a challenging and comprehensive dataset shall be able to facilitate the community to develop, adapt, and evaluate various types of advanced methods for animal behavior analysis. Moreover, we propose a Collaborative Action Recognition (CARe) model that learns general and specific features for action recognition with unseen new animals. This method achieves promising performance in our experiments. Our dataset can be found at https://sutdcv.github.io/Animal-Kingdom.

Authors

Yun Ni

2 papers

Xun Long Ng

1 papers

Kian Eng Ong

1 papers

TL;DR

Abstract

Authors

References93 items

AP-10K: A Benchmark for Animal Pose Estimation in the Wild

Conducting Behavioural Research in the Zoo: A Guide to Ten Important Methods, Concepts and Theories

SyDog: A Synthetic Dog Dataset for Improved 2D Pose Estimation

Florida Wildlife Camera Trap Dataset

Discovering Relationships between Object Categories via Universal Canonical Maps

Generalizable Person Re-identification with Relevance-aware Mixture of Experts

The iWildCam 2021 Competition Dataset

VideoLT: Large-scale Long-tailed Video Recognition

Iterative human and automated identification of wildlife images

Filtering Empty Camera Trap Images in Embedded Systems

From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation

Action Recognition Using a Spatial-Temporal Network for Wild Felines

Pose estimation and behavior classification of broiler chickens based on deep neural networks

Action detection using a neural network elucidates the genetics of mouse grooming behavior

DeepEthogram: a machine learning pipeline for supervised behavior classification from raw pixels

Automated markerless pose estimation in freely moving macaques with OpenMonkeyStudio

MacaquePose: A Novel “In the Wild” Macaque Monkey Pose Dataset for Markerless Motion Capture

A computer vision-based method for spatial-temporal action recognition of tail-biting behaviour in group-housed pigs

Big behavior: challenges and opportunities in a new era of deep behavior profiling

Anipose: A toolkit for robust markerless 3D pose estimation

Heterogeneous Domain Generalization Via Domain Mixup

Local-Global Video-Text Interactions for Temporal Grounding

X3D: Expanding Architectures for Efficient Video Recognition

Revisiting Pose-Normalization for Fine-Grained Few-Shot Recognition

Span-based Localizing Network for Natural Language Video Localization

Learning Meta Face Recognition in Unseen Domains

Equalization Loss for Long-Tailed Object Recognition

Deformation-Aware Unpaired Image Translation for Pose Estimation on Laboratory Animals

Learning From Synthetic Animals

A spatio-temporal recurrent network for salmon feeding action recognition from underwater videos in aquaculture

Deep learning improves automated rodent behavior recognition within a specific experimental setup

Animal Detection in Man-made Environments

Distribution-Aware Coordinate Representation for Human Pose Estimation

An automatic behavior recognition system classifies animal behaviors using movements and their temporal context

Deep learning tools for the measurement of animal behavior in neuroscience

Pretraining boosts out-of-domain robustness for pose estimation

Cross-Domain Adaptation for Animal Pose Estimation

Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss

Deep High-Resolution Representation Learning for Human Pose Estimation

Episodic Training for Domain Generalization

SlowFast Networks for Video Recognition

Video Action Transformer Network

Feeding behavior recognition for group-housed pigs with the Faster R-CNN

A Benchmark for Action Recognition of Large Animals

Online Collective Animal Movement Activity Recognition

An open-source platform for analyzing and sharing worm-behavior data

Lions and Tigers and Bears: Capturing Non-rigid, 3D, Articulated Shape from Images

Fast animal pose estimation using deep neural networks

DeepLabCut: markerless pose estimation of user-defined body parts with deep learning

Visual Tracking of Small Animals in Cluttered Natural Environments Using a Freely Moving Camera

Learning to Generalize: Meta-Learning for Domain Generalization

Animal Recognition and Identification with Deep Convolutional Neural Networks for Automated Wildlife Monitoring

Focal Loss for Dense Object Detection

The iNaturalist Species Classification and Detection Dataset

What is my rat doing? Behavior understanding of laboratory animals

Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly

Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

TALL: Temporal Activity Localization via Language Query

Interspecies Knowledge Transfer for Facial Keypoint Detection

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Global Biodiversity Outlook 3

3D Menagerie: Modeling the 3D Shape and Pose of Animals

Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding

Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna

Toward a Science of Computational Ethology

Comparison of automated home-cage monitoring systems: Emphasis on feeding behaviour, activity and spatial learning following pharmacological interventions

Quick, Accurate, Smart: 3D Computer Vision Technology Helps Assessing Confined Animals’ Behaviour

Fast action recognition using negative space features

Articulated Human Detection with Flexible Mixtures of Parts

The fish tail motion forms an attached leading edge vortex

Financial Costs of Meeting Global Biodiversity Conservation Targets: Current Spending and Unmet Needs

The national centre for the replacement, refinement, and reduction of animals in research

Inferential and visual analysis of ethogram data using multivariate techniques

How Many Species Are There on Earth and in the Ocean?

Scikit-learn: Machine Learning in Python

Caltech-UCSD Birds 200