idtracker.ai: Tracking all individuals in large collectives of unmarked animals (2018-03-12T00:00:00.000000Z)

TL;DR

An algorithm and software is described that extracts from video all trajectories with correct identities at a high accuracy for collectives of up to 100 individuals, trained adaptively to conditions and difficulty of the video.

Abstract

Our understanding of collective animal behavior is limited by our ability to track each of the individuals. We describe an algorithm and software, idtracker.ai, that extracts from video all trajectories with correct identities at a high accuracy for collectives of up to 100 individuals. It uses two deep networks, one detecting when animals touch or cross and an-other one for animal identification, trained adaptively to conditions and difficulty of the video.

Authors

Francisco J. H. Heras

3 papers

Francisco Romero-Ferrero

2 papers

M. Bergomi

1 papers

idtracker.ai: Tracking all individuals in large collectives of unmarked animals

TL;DR

Abstract

Authors

References26 items

ToxId: an efficient algorithm to solve occlusions when tracking multiple animals

Robust tracking of fish schools using CNN for head identification

Domino-like propagation of collective U-turns in fish schools

Zebrafish tracking using convolutional neural networks

Closing the loop: tracking and perturbing behaviour of individuals in a group in real-time

Toward an Integrated Zebrafish Health Management Program Supporting Cancer and Neuroscience Research

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

A method for resolving occlusions when multitracking individuals in a shoal

Adam: A Method for Stochastic Optimization

idTracker: tracking individuals in a group by automatic identification of unmarked animals

Understanding the difficulty of training deep feedforward neural networks

Deep Learning

Deep Learning

If all these conditions are met, F will be added to A ¯ i +1 as a single individual fragment

ii) Iterate on the elements of σ ( G )

| N | > 1 . The blob b in frame i overlaps with more than one blob in frame i + 1 , hence the crossing represented by b is splitting

Accuracy: Number of images correctly identi ed over I S

We compute the speed of the animal in F by considering the distance in pixels between subsequent centroids. Namely, v i = d ( c i , c i +1 ) for i ∈ { 1 , . . . , n − 1 } . 40

The identification network is trained using stochastic gradient descent, setting the learning rate to 0 . 005 . This network is trained in mini batches of 500 images

Accuracy during protocol cascade: Number of images correctly identi ed during the ngerpint protocol cascade, over the total number of individual images used to train the idCNN in S

iii) Train the model using the parameters learnt during the previous iteration for the convolutional part idCNN c (cid:0) Γ σ ( i − 1) (cid:1) and reinitialise idCNN f

| C | > 1 : F is identified by considering the identity i ∈ C realising the minimum boundary speed

the identity of F is coherent with all the individual fragments in γ ( F ) , i. e. the assignment of a certain identity to F does not create duplications

let γ ( F ) be the set of individual fragments that coexist with F in at least one frame of the video. F can be accumulated if at least half of the elements of γ ( F ) have already been accumulated

Conclude the training according to the conditions listed in section D.4.2

case 1: | N | > 1

Field of Study

Journal Information

Name

Venue Information

Name

Type

URL