Temporal stability in predictive process monitoring (2017-12-12T00:00:00.000000Z)

TL;DR

This paper defines a notion of temporal stability for binary classification tasks in predictive process monitoring and evaluates existing methods with respect to both temporal stability and accuracy and finds that methods based on XGBoost and LSTM neural networks exhibit the highest temporal stability.

Abstract

Predictive process monitoring is concerned with the analysis of events produced during the execution of a business process in order to predict as early as possible the final outcome of an ongoing case. Traditionally, predictive process monitoring methods are optimized with respect to accuracy. However, in environments where users make decisions and take actions in response to the predictions they receive, it is equally important to optimize the stability of the successive predictions made for each case. To this end, this paper defines a notion of temporal stability for binary classification tasks in predictive process monitoring and evaluates existing methods with respect to both temporal stability and accuracy. We find that methods based on XGBoost and LSTM neural networks exhibit the highest temporal stability. We then show that temporal stability can be enhanced by hyperparameter-optimizing random forests and XGBoost classifiers with respect to inter-run stability. Finally, we show that time series smoothing techniques can further enhance temporal stability at the expense of slightly lower accuracy.

Authors

M. Dumas

4 papers

Irene Teinemaa

3 papers

F. Maggi

3 papers

TL;DR

Abstract

Authors

References42 items

Predictive Monitoring of Business Processes: A Survey

Intra and Inter-case Features in Predictive Process Monitoring: A Tale of Two Dimensions

Data-driven advice for applying machine learning to bioinformatics problems

Generalising Random Forest Parameter Optimisation to Include Stability and Cost

On Calibration of Modern Neural Networks

Predicting process behaviour using deep learning

Predictive Business Process Monitoring with LSTM Neural Networks

Reliable early classification of time series based on discriminating the classes over time

XGBoost: A Scalable Tree Boosting System

A general process mining framework for correlating, predicting and clustering dynamic behavior based on event logs

Complex Symbolic Sequence Encodings for Predictive Monitoring of Business Processes

Clustering-Based Predictive Process Monitoring

Reliable Early Classification on Multivariate Time Series with Numerical and Categorical Attributes

Comparing and Combining Predictive Business Process Monitoring Techniques

Data-aware remaining time prediction of business process instances

Predictive Monitoring of Business Processes

Prediction of Remaining Service Execution Time Using Stochastic Petri Nets with Arbitrary Firing Delays

Random Search for Hyper-Parameter Optimization

Best Practices in Data Cleaning: A Complete Guide to Everything You Need to Do Before and After Collecting Your Data

Early classification on time series

Predictive Analytics for Semi-structured Case Oriented Business Processes

Cycle Time Prediction: When Will This Case Finally Be Finished?

Mining Sequence Classifiers for Early Prediction

Stability of Randomized Learning Algorithms

Predicting good probabilities with supervised learning

Missing data: our view of the state of the art.

Bagging Predictors

Fundamentals of Business Process Management

Outcome-oriented predictive process monitoring: Review and benchmark

A Literature Survey of Early Time Series Classification and Deep Learning

Do we need hundreds of classifiers to solve real world classification problems?

Predictivemonitoring of business processes. In: International conference on advanced information systems engineering

Six: Dealing with Missing or Incomplete Data: Debunking the Myth of Emptiness

Classifying with confidence from incomplete information

Process Mining

Stability and Generalization

Random Forests

Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods

RQ1 What is the relative performance of diﬀerent predictive process monitoring methods in terms of temporal stability (in addition to accuracy)?

We conducted an empirical evaluation to address the following questions

RQ2 How does maximizing the inter-run stability in combination with prediction accuracy aﬀect the temporal stability?

perform an evaluation of several existing predictive process monitoring meth-ods with respect to both prediction accuracy and temporal stability

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL

Alternate Names