Home Research Papers Datasets State of the Art Pricing

Discover, visualize, and connect AI research papers. Explore the latest trends and insights in artificial intelligence research.

Product

Home
Research Papers
About

Support

Contact
Terms of Service
Privacy Policy

© 2026 Papersgraph. All rights reserved.

methodology-4

Information Plane

3260 papers • 126 benchmarks • 313 datasets

To obtain the Information Plane (IP) of deep neural networks, which shows the trajectories of the hidden layers during training in a 2D plane using as coordinate axes the mutual information between the input and the hidden layer, and the mutual information between the output and the hidden layer.

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in information-plane-4

Trend

Dataset

Best Model

Actions

No benchmarks available.

Libraries

i

Use these libraries to find information-plane-4 models and implementations

Datasets

No datasets available.

Subtasks

No subtasks available.

Most implemented papers

Opening the Black Box of Deep Neural Networks via Information

Ravid Shwartz-Ziv, Naftali Tishby•Wed Mar 01 2017

This work demonstrates the effectiveness of the Information-Plane visualization of DNNs and shows that the training time is dramatically reduced when adding more hidden layers, and the main advantage of the hidden layers is computational.

1570

Content

Introduction Benchmarks Datasets Subtasks Libraries Papers

0

On the information bottleneck theory of deep learning

Yamini Bansal, David D. Cox, Andrew M. Saxe, Joel Dapello, Madhu S. Advani, Artemy Kolchinsky, Brendan D. Tracey•Wed Feb 14 2018

This work studies the information bottleneck (IB) theory of deep learning, and finds that there is no evident causal connection between compression and generalization: networks that do not compress are still capable of generalization, and vice versa.

624 0

Scalable Mutual Information Estimation Using Dependence Graphs

M. Noshad, A. Hero•Fri Jan 26 2018

EDGE is the first non-parametric MI estimator that can achieve paramet- ric MSE rates with linear time complexity and the utility of EDGE is illustrated for the analysis of the information plane (IP) in deep learning.

80 0

On the Information Plane of Autoencoders

Nicol'as I. Tapia, P. Est'evez•Thu May 14 2020

This work derives a theoretical convergence for the IP of autoencoders based on the information-theoretic concept of mutual information (MI), and proposes a new rule to adjust its parameters that compensates scale and dimensionality effects.

20 0

The Dual Information Bottleneck

Naftali Tishby, Zoe Piran•Tue Sep 24 2019

A theoretical analysis of the dualIB framework is provided, solving for the structure of its solutions, unraveling its superiority in optimizing the mean prediction error exponent and demonstrating its ability to preserve exponential forms of the original distribution.

16 0

Malicious Network Traffic Detection via Deep Learning: An Information Theoretic View

Erick Galinkin•Tue Sep 15 2020

The results show that since mutual information remains invariant under homeomorphism, only feature engineering methods that alter the entropy of the dataset will change the outcome of the neural network, and suggests that neural networks that can exploit the convolution theorem are equally accurate as standard convolutional neural networks, and can be more computationally efficient.

0 0

A Provably Convergent Information Bottleneck Solution via ADMM

A. E. Gamal, Tengfei Huang•Mon Feb 08 2021

With augmented variables, it is shown that the IB objective can be solved with the alternating direction method of multipliers (ADMM), and it is proved that the proposed algorithm is consistently convergent, regardless of the value of β.

7 0

Information Flows of Diverse Autoencoders

Sungyeop Lee, Junghyo Jo•Sun Feb 14 2021

This study examines how the information flows are shaped by the network parameters, such as depth, sparsity, weight constraints, and hidden representations, and adopts autoencoders as models of deep learning.

12 0

HRel: Filter pruning based on High Relevance between activation maps and class labels

S. Dubey, C. Sarvani, Mrinmoy Ghorai, S. H. Shabbeer Basha•Tue Nov 30 2021

The proposed HRel pruning method outperforms recent state-of-the-art filter pruning methods and the Information Plane dynamics of Information Bottleneck theory is analyzed for various Convolutional Neural Network architectures with the effect of pruning.

46 0

End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Training

Keitaro Sakamoto, Issei Sato•Tue Feb 13 2024

The results of the normalized HSIC value analysis reveal the E2E training ability to exhibit different information dynamics across layers, in addition to efficient information propagation, and suggest the need to consider the cooperative interactions between layers, not just the final layer when analyzing the information bottleneck of deep learning.

10 0

Adding a benchmark result helps the community track progress.

Information Plane | State-of-the-Art