A Survey of Human-in-the-loop for Machine Learning

Published in

Future generations computer systems(2021)

External Links:

Generate Graph DownloadPDF

TL;DR

This paper surveys existing works on human-in-the-loop from a data perspective and classify them into three categories with a progressive relationship: the work of improving model performance from data processing, the work through interventional model training, and the design of the system independent human- in- the-loop.

Authors

Xingjiao Wu

1 papers

Luwei Xiao

1 papers

Yixuan Sun

1 papers

Junhang Zhang

1 papers

Tianlong Ma

1 papers

Liangbo He

1 papers

References242 items

Human-in-the-Loop-Aided Privacy-Preserving Scheme for Smart Healthcare

Guest Editorial: Computational Intelligence for Human-in-the-Loop Cyber Physical Systems

Optimal Energy Operation Strategy for We-Energy of Energy Internet Based on Hybrid Reinforcement Learning With Human-in-the-Loop

Document image layout analysis via explicit edge embedding network

A Human-in-the-Loop Probabilistic CNN-Fuzzy Logic Framework for Accident Prediction in Vehicular Networks

A Survey of Human-in-the-loop for Machine Learning

Published in

Future generations computer systems(2021)

External Links:

Generate Graph DownloadPDF

TL;DR

Authors

Xingjiao Wu

1 papers

Luwei Xiao

1 papers

Yixuan Sun

1 papers

Junhang Zhang

1 papers

Tianlong Ma

1 papers

Liangbo He

1 papers

References242 items

Human-in-the-Loop-Aided Privacy-Preserving Scheme for Smart Healthcare

Guest Editorial: Computational Intelligence for Human-in-the-Loop Cyber Physical Systems

Optimal Energy Operation Strategy for We-Energy of Energy Internet Based on Hybrid Reinforcement Learning With Human-in-the-Loop

Document image layout analysis via explicit edge embedding network

A Human-in-the-Loop Probabilistic CNN-Fuzzy Logic Framework for Accident Prediction in Vehicular Networks

Quality Evaluation of Holographic Images Coded With Standard Codecs

Towards multi-modal causability with Graph Neural Networks enabling information fusion for explainable AI

PACE: Learning Effective Task Decomposition for Human-in-the-loop Healthcare Delivery

When and Why a Model Fails? A Human-in-the-loop Error Detection Framework for Sentiment Analysis

Boosting Unconstrained Face Recognition with Auxiliary Unlabeled Data

Improving Transferability of Adversarial Patches on Face Recognition with Generative Models

Data Cleaning Tools for Token Classification Tasks

Generating Manga from Illustrations via Mimicking Manga Creation Workflow

MARTA: Leveraging Human Rationales for Explainable Text Classification

Transitioning to Human Interaction with AI Systems: New Challenges and Opportunities for HCI Professionals to Enable Human-Centered AI

A survey on deep learning and its applications

A human in the loop in surgery automation

Smart technology–driven aspects for human-in-the-loop smart manufacturing

Adaptive Autonomy in Human-on-the-Loop Vision-Based Robotics Systems

A survey: Deep learning for hyperspectral image classification with few labeled samples

Putting Humans in the Natural Language Processing Loop: A Survey

Domain Generalization: A Survey

Towards a Weakly Supervised Framework for 3D Point Cloud Object Detection and Annotation

Classification by ordinal sums of conjunctive and disjunctive functions for explainable AI and interpretable machine learning solutions

Heterogeneous data fusion for predicting mild cognitive impairment conversion

Joint auto-weighted graph fusion and scalable semi-supervised learning

Transformers in Vision: A Survey

Efficient Human-In-The-Loop Object Detection using Bi-Directional Deep SORT and Annotation-Free Segment Identification

NICER: Aesthetic Image Enhancement with Humans in the Loop

Investigating Typed Syntactic Dependencies for Targeted Sentiment Classification Using Graph Attention Neural Network

Towards Transparent Human-in-the-Loop Classification of Fraudulent Web Shops

Classification of COVID-19 chest X-rays with deep learning: new models or fine tuning?

Offline Reinforcement Learning from Human Feedback in Real-World Sequence-to-Sequence Tasks

Multi-head self-attention based gated graph convolutional networks for aspect-based sentiment classification

Human-in-the-Loop AI for Analysis of Free Response Facial Expression Label Sets

Human-in-the-Loop Error Precursor Detection using Language Translation Modeling of HMI States

Generative Melody Composition with Human-in-the-Loop Bayesian Optimization

Classical and deep learning methods for recognizing human activities and modes of transportation with smartphone sensors

Engineering human-in-the-loop interactions in cyber-physical systems

Improving human-in-the-loop simulation to optimize soldier-systems integration.

Efficient Transformers: A Survey

Learning to summarize from human feedback

Thermal comfort management leveraging deep reinforcement learning and human-in-the-loop

Soliciting Human-in-the-Loop User Feedback for Interactive Machine Learning Reduces User Trust and Impressions of Model Accuracy

Human-in-the-Loop Low-Shot Learning

Deep learning techniques for rating prediction: a survey of the state-of-the-art

A system for human-in-the-loop simulation of industrial collaborative robot applications*

Learning from Few Samples: A Survey

EasierPath: An Open-source Tool for Human-in-the-loop Deep Learning of Renal Pathology

Semi-Automatic Data Annotation guided by Feature Space Projection

Digital Human-in-the-Loop Framework

Deep Hough-Transform Line Priors

How to Trust Unlabeled Data? Instance Credibility Inference for Few-Shot Learning

Iterative Bounding Box Annotation for Object Detection

Online Replanning With Human-in-the-Loop for Non-Prehensile Manipulation in Clutter — A Trajectory Optimization Based Approach

Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning

Contextualized Weak Supervision for Text Classification

From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains

Information Preparation with the Human in the Loop

Differentiable Augmentation for Data-Efficient GAN Training

Objective Measurement Assessment of Departure Advisories for Ramp Controllers from a Human-In-The-Loop Simulation

Skeleton-Based Action Recognition With Shift Graph Convolutional Network

Human-in-the-Loop Conversation Agent for Customer Service

Video Object Segmentation and Tracking

Human-in-the-loop optimization of wearable robots to reduce the human metabolic energy cost in physical movements

Human-Machine Collaboration for Medical Image Segmentation

Explanations for Human-on-the-loop: A Probabilistic Model Checking Approach

Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting

PARTNER: Human-in-the-Loop Entity Name Understanding with Deep Learning

Pre-trained models for natural language processing: A survey

DeepCap: Monocular Human Performance Capture Using Weak Supervision

Draw with me: human-in-the-loop for image restoration

Human-in-the-loop AI in government: a case study

ISABELA - A Socially-Aware Human-in-the-Loop Advisor System

Toward Interactive Self-Annotation For Video Object Bounding Box: Recurrent Self-Learning And Hierarchical Annotation Based Framework

Defect Image Sample Generation With GAN for Improving Defect Recognition

Targeted Sentiment Classification Based on Attentional Encoding and Graph Convolutional Networks

Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension

A Short Survey of Pre-trained Language Models for Conversational AI-A New Age in NLP

Image Segmentation Using Deep Learning: A Survey

LayoutGAN: Synthesizing Graphic Layouts With Vector-Wireframe Adversarial Networks

Weak Supervision for Fake News Detection via Reinforcement Learning

Human-In-The-Loop Automatic Program Repair

Multimodal Self-Supervised Learning for Medical Image Analysis

Large-scale relation extraction from web documents and knowledge graphs with human-in-the-loop

Reinforcement Learning for Data Preparation with Active Reward Learning

Learning-Based Methods with Human-in-the-Loop for Entity Resolution

Unsupervised pre-trained filter learning approach for efficient convolution neural network

Journalist-in-the-Loop: Continuous Learning as a Service for Rumour Analysis

Aspect-Level Sentiment Analysis Via Convolution over Dependency Tree

A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis

Automatic Image Enhancement Taking into Account User Preference

Deep Reinforcement Active Learning for Human-in-the-Loop Person Re-Identification

Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study

Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis

Rethinking deep active learning: Using unlabeled data at model training

Fine-Tuning Language Models from Human Preferences

An interactive visual analytics approach for network anomaly detection through smart labeling

A deep learning algorithm for simulating autonomous driving considering prior knowledge and temporal information

100

Rumour verification through recurring information and an inner-attention mechanism

101

Combining Deep Learning and Prior Knowledge for Crop Mapping in Tropical Regions from Multitemporal SAR Image Sequences

102

Leveraging Human Guidance for Deep Reinforcement Learning Tasks

103

TopicSifter: Interactive Search Space Reduction through Targeted Topic Modeling

104

How to Invest my Time: Lessons from Human-in-the-Loop Entity Extraction

105

Knowledge Graph Programming with a Human-in-the-Loop: Preliminary Results

106

Fast video crowd counting with a Temporal Aware Network

107

Human-in-the-Loop Control Using Euler Angles

108

Review of Human-in-the-Loop Cyber-Physical Systems (HiLCPS): The Current Status from Human Perspective

109

Explore and Exploit. Dictionary Expansion with Human-in-the-Loop

110

Why Didn’t You Listen to Me? Comparing User Control of Human-in-the-Loop Topic Models

111

A “Human-in-the-Loop” Approach for Semi-automated Image Restoration in Electron Microscopy

112

Object Detection in 20 Years: A Survey

113

A Human-in-the-loop Attribute Design Framework for Classification

114

Fast User-Guided Video Object Segmentation by Interaction-And-Propagation Networks

115

Generalizing from a Few Examples

116

Human-in-the-loop HVAC operations: A quantitative review on occupancy, comfort, and energy-efficiency dimensions

117

Enhancing Deep Learning with Visual Interactions

118

Attentional Encoder Network for Targeted Sentiment Classification

119

On Human-in-the-Loop CPS in Healthcare: A Cloud-Enabled Mobility Assistance Service

120

Learning from Dialogue after Deployment: Feed Yourself, Chatbot!

121

EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning

122

Smart computational exploration of stochastic gene regulatory network models using human-in-the-loop semi-supervised learning

123

Interactive machine learning: experimental evidence for the human in the algorithmic loop

124

Deep Reinforcement Learning for Time Optimal Velocity Control using Prior Knowledge

125

Why do Larger Models Generalize Better? A Theoretical Perspective via the XOR Problem

126

Image Aesthetics Assessment Using Composite Features from off-the-Shelf Deep Models

127

Give Me a Hint! Navigating Image Databases Using Human-in-the-Loop Feedback

128

“Weak” Control for Human-in-the-Loop Systems

129

Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering

130

Communication-Human Information Processing (C-HIP) Model

131

Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning

132

A Survey on Deep Transfer Learning

133

Digitized Trust in Human-in-the-Loop Health Research

134

Values and Ethics in Human-Computer Interaction

135

Towards improving diagnosis of skin diseases by combining deep neural network and human knowledge

136

Natural Language Interfaces with Fine-Grained User Interaction: A Case Study on Web APIs

137

A Human-in-the-Loop System for Sound Event Detection and Annotation

138

A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress

139

Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure

140

A Review of User Interface Design for Interactive Machine Learning

141

Human-in-the-Loop Data Analysis: A Personal Perspective

142

Data Integration and Machine Learning: A Natural Synergy

143

Human-in-the-loop development of soft wearable robots

144

Towards understanding and simplifying human-in-the-loop machine learning

145

Image Inpainting for Irregular Holes Using Partial Convolutions

146

Accelerating Human-in-the-loop Machine Learning: Challenges and Opportunities

148

Closing the Loop: User-Centered Design and Evaluation of a Human-in-the-Loop Topic Modeling System

149

The 2018 DAVIS Challenge on Video Object Segmentation

150

Shift-Net: Image Inpainting via Deep Feature Rearrangement

151

Interactive Video Object Segmentation in the Wild

152

Integrating Prior Knowledge into Deep Learning

153

Deep Image Prior

154

Data and Decision Intelligence for Human-in-the-Loop Cyber-Physical Systems: Reference Model, Recent Progresses and Challenges

155

Building Natural Language Interfaces to Web APIs

156

Hike: A Hybrid Human-Machine Method for Entity Alignment in Large-Scale Knowledge Bases

157

Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning

158

NIMA: Neural Image Assessment

159

The human touch: How non-expert users perceive, interpret, and fix topic models

160

Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning

161

A Review on Image Enhancement Techniques

162

Stacked Deconvolutional Network for Semantic Segmentation

163

Rise of the HaCRS: Augmenting Autonomous Cyber Reasoning Systems with Human Assistance

164

Human-in-the-loop Data Integration

165

A Survey on Deep Learning in Big Data

166

Human-in-the-loop optimization of exoskeleton assistance during walking

167

Attention is All you Need

168

Improving Human-in-the-Loop Adaptive Systems Using Brain-Computer Interaction

169

Human-in-the-Loop Challenges for Entity Matching: A Midterm Report

170

Learning Video Object Segmentation with Visual Memory

171

Almond: The Architecture of an Open, Crowdsourced, Privacy-Preserving, Programmable Virtual Assistant

172

Simple online and realtime tracking with a deep association metric

173

Challenges and opportunities: from big data to knowledge in AI 2.0

174

Computer Vision and Natural Language Processing

175

Learning Video Object Segmentation from Static Images

176

One-Shot Video Object Segmentation

177

Dialogue Learning With Human-In-The-Loop

178

Human-in-the-Loop Parsing

179

Face Recognition via Active Annotation and Learning

180

Bridging the gap between user intention and model parameters for human-in-the-loop data analytics

181

Abstractive Sentence Summarization with Attentive Recurrent Neural Networks

182

Rationale-Augmented Convolutional Neural Networks for Text Classification

183

Designing for Transient Use: A Human-in-the-loop Translation Platform for Refugees

184

Deep Interactive Object Selection

185

Interactive machine learning for health informatics: when do we need the human-in-the-loop?

186

“Why Should I Trust You?”: Explaining the Predictions of Any Classifier

187

Artificial Swarm Intelligence, a Human-in-the-Loop Approach to A.I

188

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

189

LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

190

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

191

U-Net: Convolutional Networks for Biomedical Image Segmentation

192

Learning Deconvolution Network for Semantic Segmentation

194

WhittleSearch: Interactive Image Search with Relative Attribute Feedback

195

A Survey on Human-in-the-Loop Applications Towards an Internet of All

196

Automatic Photo Adjustment Using Deep Neural Networks

197

Learning Deep Features for Scene Recognition using Places Database

198

Reducing Energy Waste for Computers by Human-in-the-Loop Control

199

A shared control architecture for human-in-the-loop robotics applications

200

Towards the safety of human-in-the-loop robotics: Challenges and opportunities for safety assurance of robotic co-workers'

201

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

202

Dynodroid: an input generation system for Android apps

203