Multi-Label Classification Neural Networks with Hard Logical Constraints (2021-03-24T00:00:00.000000Z)

TL;DR

This article proposes C-HMCNN(h), a novel approach for solving HMC problems, which, given a network h for the underlying MC problem, exploits the hierarchy information in order to produce predictions coherent with the constraints and to improve performance.

Abstract

Multi-label classification (MC) is a standard machine learning problem in which a data point can be associated with a set of classes. A more challenging scenario is given by hierarchical multi-label classification (HMC) problems, in which every prediction must satisfy a given set of hard constraints expressing subclass relationships between classes. In this article, we propose C-HMCNN(h), a novel approach for solving HMC problems, which, given a network h for the underlying MC problem, exploits the hierarchy information in order to produce predictions coherent with the constraints and to improve performance. Furthermore, we extend the logic used to express HMC constraints in order to be able to specify more complex relations among the classes and propose a new model CCN(h), which extends C-HMCNN(h) and is again able to satisfy and exploit the constraints to improve performance. We conduct an extensive experimental analysis showing the superior performance of both C-HMCNN(h) and CCN(h) when compared to state-of-the-art models in both the HMC and the general MC setting with hard logical constraints.

Authors

Eleonora Giunchiglia

3 papers

Thomas Lukasiewicz

11 papers

TL;DR

Abstract

Authors

References86 items

Coherent Hierarchical Multi-Label Classification Networks

Asynchronous Distributed Learning From Constraints

Machine learning for discovering missing or wrong protein function annotations

Hierarchical Classification Based on Label Distribution Learning

Structured Bayesian Networks: From Inference to Learning with Routes

Augmenting Neural Networks with First-order Logic

Collaboration based Multi-Label Learning

mlDEEPre: Multi-Functional Enzyme Function Prediction With Hierarchical Multi-Label Deep Learning

A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability

A hierarchical multi-label classification method based on neural networks for gene function prediction

AWX: An Integrated Approach to Hierarchical-Multilabel Classification

Hierarchical Multi-Label Classification Networks

Correlation analysis of performance measures for multi-label classification

Conditional PSDDs: Modeling and Learning With Modular Knowledge

DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems

A Semantic Loss Function for Deep Learning with Symbolic Knowledge

DEEPre: sequence-based enzyme EC number prediction by deep learning

An approach to reachability analysis for feed-forward ReLU neural networks

Logic Tensor Networks for Semantic Image Interpretation

DeepXplore: Automated Whitebox Testing of Deep Learning Systems

DeepGO: predicting protein functions from sequence and interactions using a deep ontology-aware classifier

Semantic-based regularization for learning and inference

A scikit-based Python environment for performing multi-label classification

Deep Neural Networks with Massive Learned Knowledge

Label-Free Supervision of Neural Networks with Physics and Domain Knowledge

Reduction strategies for hierarchical multi-label classification in protein function prediction

Harnessing Deep Neural Networks with Logic Rules

Learning With Mixed Hard/Soft Pointwise Constraints

Bidirectional LSTM-CRF Models for Sequence Tagging

Should We Really Use Post-Hoc Tests Based on Mean-Ranks?

The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets

Adam: A Method for Stochastic Optimization

Cutset Networks: A Simple, Tractable, and Scalable Approach for Improving the Accuracy of Chow-Liu Trees

Large-Scale Object Classification Using Label Relation Graphs

Learning Deep Structured Models

Probabilistic Sentential Decision Diagrams

Hierarchical multi-label classification using local neural networks

A systematic review on experimental multi-label learning

Hierarchical classification using a Competitive Neural Network

Hierarchical multi-label classification for protein function prediction: A local approach based on neural networks

Hierarchical annotation of medical images

Sum-product networks: A new deep architecture

MultiLabel Classification on Tree- and DAG-Structured Hierarchies

True Path Rule Hierarchical Ensembles for Genome-Wide Gene Function Prediction

Scikit-learn: Machine Learning in Python

An Abstraction-Refinement Approach to Verification of Artificial Neural Networks

Predicting gene function using hierarchical multi-label decision tree ensembles

ImageNet: A large-scale hierarchical image database

Multi-label Classification Using Ensembles of Pruned Sets

Decision trees for hierarchical multi-label classification

Semantic Annotation and Retrieval of Music and Sound Effects

Random k -Labelsets: An Ensemble Method for Multilabel Classification

ML-KNN: A lazy learning approach to multi-label learning

A shared task involving multi-label classification of clinical free text

Statistical Comparisons of Classifiers over Multiple Data Sets

Kernel-Based Learning of Hierarchical Multilabel Classification Models

Protein Classification with Multiple Algorithms

Discovering recurring anomalies in text reports regarding complex space systems

Incremental Algorithms for Hierarchical Classification

RCV1: A New Benchmark Collection for Text Categorization Research

Learning multi-label scene classification

A kernel method for multi-labelled classification

Practical Methods of Optimization: Fletcher/Practical Methods of Optimization

Long Short-Term Memory

Introduction to algorithms

Fast Algorithms for Mining Association Rules in Large Databases

Towards a Theory of Declarative Knowledge

Constrained Optimization and Lagrange Multiplier Methods

Probabilistic circuits: Representation and inference

Tractable Operations for Arithmetic Circuits of Probabilistic Models

10 Notice that, the above list does not include AU (P RC): already in 2013 and still today, contrarily to the specialized HMC literature, it is generally not used in the MC literature

Hierarchical classification of diatom images using ensembles of predictive clustering trees

Correlation-Based Pruning of Stacked Binary Relevance Models for Multi-Label Learning

Introduction to Algorithms, 3rd Edition

Technical supplement to \Consistent probabilistic outputs for protein function prediction"

Effective and Efficient Multilabel Classification in Domains with Large Number of Labels