Iterative SE(3)-Transformers (2021-02-26T00:00:00.000000Z)

TL;DR

An iterative version of the SE(3)-Transformer is implemented, an SE( 3)-equivariant attention-based model for graph data, to address the additional complications which arise when applying the SE (3)- Transformer in an iterative fashion and consider why a iterative model may be beneficial in some problem settings.

Abstract

When manipulating three-dimensional data, it is possible to ensure that rotational and translational symmetries are respected by applying so-called SE(3)-equivariant models. Protein structure prediction is a prominent example of a task which displays these symmetries. Recent work in this area has successfully made use of an SE(3)-equivariant model, applying an iterative SE(3)-equivariant attention mechanism. Motivated by this application, we implement an iterative version of the SE(3)-Transformer, an SE(3)-equivariant attention-based model for graph data. We address the additional complications which arise when applying the SE(3)-Transformer in an iterative fashion, compare the iterative and single-pass versions on a toy problem, and consider why an iterative model may be beneficial in some problem settings. We make the code for our implementation available to the community.

Authors

I. Posner

7 papers

F. Fuchs

2 papers

E. Wagstaff

1 papers

TL;DR

Abstract

Authors

References36 items

LieTransformer: Equivariant self-attention for Lie Groups

SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks

Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data

Improved protein structure prediction using potentials from deep learning

Improved protein structure prediction using predicted interresidue orientations

Advances in protein structure prediction and design

Stand-Alone Self-Attention in Vision Models

Methods for the Refinement of Protein Structure 3D Models

Modeling Point Clouds With Self-Attention and Gumbel Subset Sampling

HH-suite3 for fast remote homology detection and deep protein annotation

Deep learning extends de novo protein modelling coverage of genomes using iteratively predicted structural constraints

UniProt: a worldwide hub of protein knowledge

Protein Data Bank: the single global archive for 3D macromolecular structure data

Set Transformer

3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data

Relational inductive biases, deep learning, and graph networks

Attentional ShapeContextNet for Point Cloud Recognition

Self-Attention with Relative Position Representations

N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials

Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds

Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions

Neural Relational Inference for Interacting Systems

Non-local Neural Networks

Biological Sequence Modeling with Convolutional Kernel Networks

Graph Attention Networks

VAIN: Attentional Multi-agent Predictive Modeling

Attention is All you Need

A Structured Self-attentive Sentence Embedding

Group Equivariant Convolutional Networks

Lupoglaz

End-to-end recurrent multi-object tracking and prediction with relational reasoning

High accuracy protein structure prediction using deep learning. Fourteenth Critical Assessment of Techniques for Protein Structure Prediction

Alphafold2 @ casp14: “it feels like one’s child has left home.”

Alphafold 2 explained: A semi-deep dive

Casp14: what google deepmind's alphafold 2 really achieved, and what it means for protein folding

A Comprehensive Survey on Graph Neural Networks

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names