PC-Gym: Benchmark Environments For Process Control Problems (2024-10-29T00:00:00.000000Z)

TL;DR

Case studies demonstrate the framework's effectiveness in evaluating RL approaches for systems like continuously stirred tank reactors, multistage extraction processes, and crystallization reactors, and reveal performance gaps between RL algorithms and NMPC oracles.

Abstract

PC-Gym is an open-source tool for developing and evaluating reinforcement learning (RL) algorithms in chemical process control. It features environments that simulate various chemical processes, incorporating nonlinear dynamics, disturbances, and constraints. The tool includes customizable constraint handling, disturbance generation, reward function design, and enables comparison of RL algorithms against Nonlinear Model Predictive Control (NMPC) across different scenarios. Case studies demonstrate the framework's effectiveness in evaluating RL approaches for systems like continuously stirred tank reactors, multistage extraction processes, and crystallization reactors. The results reveal performance gaps between RL algorithms and NMPC oracles, highlighting areas for improvement and enabling benchmarking. By providing a standardized platform, PC-Gym aims to accelerate research at the intersection of machine learning, control, and process systems engineering. By connecting theoretical RL advances with practical industrial process control applications, offering researchers a tool for exploring data-driven control solutions.

Authors

Maximilian Bloor

1 papers

Jos'e Torraca

1 papers

I. O. Sandoval

1 papers

TL;DR

Abstract

Authors

References51 items

Control-Informed Reinforcement Learning for Chemical Processes

A data-driven tracking control framework using physics-informed neural networks and deep reinforcement learning for dynamical systems

Beyond Expected Return: Accounting for Policy Reproducibility when Evaluating Reinforcement Learning Algorithms

do-mpc: Towards FAIR nonlinear and robust model predictive control

Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX

Modeling and Predictive Control of Cooling Crystallization of Potassium Sulfate by Dynamic Image Analysis: Exploring Phenomenological and Machine Learning Approaches

A Tutorial on Derivative-Free Policy Learning Methods for Interpretable Controller Representations

Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learning

Where Reinforcement Learning Meets Process Control: Review and Guidelines

Meta-reinforcement learning for the tuning of PI controllers: An offline approach

Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning

Reinforcement learning for batch process control: Review and perspectives

Brax - A Differentiable Physics Engine for Large Scale Rigid Body Simulation

Using process data to generate an optimal control policy via apprenticeship and reinforcement learning

Safe Chance Constrained Reinforcement Learning for Batch Process Control

DiffLoop: Tuning PID Controllers by Differentiating Through the Feedback Loop

Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning

Reinforcement learning based optimal control of batch processes using Monte-Carlo deep deterministic policy gradient with phase segmentation

Constrained Model-Free Reinforcement Learning for Process Optimization

First Order Constrained Optimization in Policy Space

OR-Gym: A Reinforcement Learning Library for Operations Research Problem

A review On reinforcement learning: Introduction and applications in industrial process control

Chance Constrained Policy Optimization for Process Control and Optimization

Scalable reinforcement learning for plant-wide control of vinyl acetate monomer process

Reinforcement Learning - Overview of recent progress and implications for process control

Understanding the impact of entropy on policy optimization

CasADi: a software framework for nonlinear optimization and optimal control

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Proximal Policy Optimization Algorithms

Constrained Policy Optimization

Continuous control with deep reinforcement learning

Stochastic model predictive control with joint chance constraints

Human-level control through deep reinforcement learning

Adaptive PID Controller based on Reinforcement Learning for Wind Turbine Control

Approximate dynamic programming based approach to process control and scheduling

On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming

Optimal control of a fed-batch bioreactor using simulation-based approximate dynamic programming

An actor-critic algorithm for constrained Markov decision processes

Simulation based strategy for nonlinear optimal control: application to a microbial cell reactor

The quadruple-tank process: a multivariable laboratory process with an adjustable zero

Neuro-fuzzy modeling and control of a batch process involving simultaneous reaction and distillation

Q-learning

ChemGymRL: A Customizable Interactive Framework for Reinforcement Learning for Digital Chemistry

Mark Towers et al. Gymnasium

Stable-Baselines3: Reliable Reinforcement Learning Implementations

JAX: composable transformations of Python+NumPy programs . Version 0.3.13

Process Dynamics and Control . 4th

Reinforcement Learning: An Introduction

gymnax: A JAX-based Reinforcement Learning Environment Library . Version 0.0.4. 2022

Disturbance generation: The suite includes functionality for creating and incorporating disturbances, allowing researchers to test the robustness of their algorithms under various perturbations

Constraint handling: PC-Gym provides mechanisms for incorporating process constraints, encouraging the development of algorithms that respect real-world limitations

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names