Exploring robust architectures for deep artificial neural networks (2021-06-30T00:00:00.000000Z)

TL;DR

It is shown that robustness performance of DANNs can be quantified before training using graph structural properties such as topological entropy and Olivier-Ricci curvature, with the greatest reliability for complex tasks and large DANN's.

Abstract

The architectures of deep artificial neural networks (DANNs) are routinely studied to improve their predictive performance. However, the relationship between the architecture of a DANN and its robustness to noise and adversarial attacks is less explored, especially in computer vision applications. Here we investigate the relationship between the robustness of DANNs in a vision task and their underlying graph architectures or structures. First we explored the design space of architectures of DANNs using graph-theoretic robustness measures and transformed the graphs to DANN architectures using various image classification tasks. Then we explored the relationship between the robustness of trained DANNs against noise and adversarial attacks and their underlying architectures. We show that robustness performance of DANNs can be quantified before training using graph structural properties such as topological entropy and Olivier-Ricci curvature, with the greatest reliability for complex tasks and large DANNs. Our results can also be applied for tasks other than computer vision such as natural language processing and recommender systems.

Authors

A. Waqas

2 papers

Hamza Farooq

1 papers

N. Bouaynaya

1 papers

TL;DR

Abstract

Authors

References77 items

Failure Detection in Deep Neural Networks for Medical Imaging

Understanding The Robustness in Vision Transformers

Spanning Network Gels from Nanoparticles and Graph Theoretical Analysis of Their Structure and Properties

Directionality theory and the second law of thermodynamics

PaSca: A Graph Neural Architecture Search System under the Scalable Paradigm

AutoML: state of the art with a focus on anomaly detection, challenges, and research directions

Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions

NAP: Neural architecture search with pruning

Deciphering the generating rules and functionalities of complex networks

Quantifying the Robustness of Complex Networks with Heterogeneous Nodes

Modeling Realistic Adversarial Attacks against Network Intrusion Detection Systems

Graph Vulnerability and Robustness: A Survey

Entropy-Based Modeling for Estimating Adversarial Bit-flip Attack Impact on Binarized Neural Network

Robustness Gym: Unifying the NLP Evaluation Landscape

Robustness of Brain Structural Networks Is Affected in Cognitively Impaired MS Patients

Graph Structure of Neural Networks

Adversarial Attacks on Graph Neural Networks

NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processing

Neural Architecture Transfer

Adversarial Attacks and Defenses on Graphs

NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search

Intriguing Properties of Adversarial ML Attacks in the Problem Space

Ollivier-Ricci Curvature-Based Method to Community Detection in Complex Networks

On Network Design Spaces for Visual Recognition

POPQORN: Quantifying Robustness of Recurrent Neural Networks

Making Convolutional Networks Shift-Invariant Again

Exploring Randomly Wired Neural Networks for Image Recognition

On Evaluating Adversarial Robustness

The Devil of Face Recognition is in the Noise

Communications in Engineering

DARTS: Differentiable Architecture Search

Clustering Coefficients for Correlation Networks

Fractal and multifractal analysis of complex networks: Estonian network of payments

Network curvature as a hallmark of brain structural connectivity

Towards Deep Learning Models Resistant to Adversarial Attacks

A comparison study between MLP and convolutional neural network models for character recognition

Network neuroscience

BRAPH: A graph theory software for the analysis of brain connectivity

Towards Evaluating the Robustness of Neural Networks

Ricci curvature: An economic indicator for market fragility and systemic risk

Robust Transport Over Networks

Deep Residual Learning for Image Recognition

Graph Curvature for Differentiating Cancer Networks

Ricci Curvature and Robustness of Cancer Networks

Explaining and Harnessing Adversarial Examples

Going deeper with convolutions

ImageNet Large Scale Visual Recognition Challenge

Intriguing properties of neural networks

Boltzmann, Darwin and Directionality theory

What is network science?

ImageNet classification with deep convolutional neural networks

On Random Weights and Unsupervised Feature Learning

Ricci curvature of metric spaces

Ricci curvature of Markov chains on metric spaces

Small-World Brain Networks

Complexity and demographic stability in population models.

Statistical mechanics of complex networks

Collective dynamics of ‘small-world’ networks

A Spectral Perspective of DNN Robustness to Label Noise

PremiUm-CNN: Propagating Uncertainty Towards Robust Convolutional Neural Networks

Expressive 1-Lipschitz Neural Networks for Robust Multiple Graph Learning against Adversarial Attacks

Robustness veri ﬁ cation for transformers

Experiment tracking with weights and biases

Adversarial Attacks on Graph Neural Networks

Random Graphs : Theory and Applications from Nature to Society to the Brain

On the Robustness of ConvNets to Training on Noisy Labels

Connected brains: introduction to graph theory

Learning Multiple Layers of Features from Tiny Images

Graph Theory Methods for the Analysis of Neural Connectivity Patterns

Publisher's Note

Gradient-based learning applied to document recognition

Riemannian Geometry

On the evolution of random graphs

Kaggle

Reprints and permission information

Exploring Robust Architectures for Deep Artiﬁcial Neural Networks 29