Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning

Published in

Knowledge Discovery and Data Mining(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

A paradigm for automatic pair trading as a unified task rather than a two-step pipeline is proposed and a hierarchical reinforcement learning framework is designed to jointly learn and optimize two sub-tasks.

Abstract

Pair trading is one of the most effective statistical arbitrage strategies which seeks a neutral profit by hedging a pair of selected assets. Existing methods generally decompose the task into two separate steps: pair selection and trading. However, the decoupling of two closely related sub-tasks can block information propagation and lead to limited overall performance. For pair selection, ignoring the trading performance results in the wrong assets being selected with irrelevant price movements, while the agent trained for trading can overfit to the selected assets without any historical information of other assets. To address it, in this paper, we propose a paradigm for automatic pair trading as a unified task rather than a two-step pipeline. We design a hierarchical reinforcement learning framework to jointly learn and optimize two sub-tasks. A high-level policy would select two assets from all possible combinations and a low-level policy would then perform a series of trading actions. Experimental results on real-world stock data demonstrate the effectiveness of our method on pair trading compared with both existing pair selection and trading methods.

Authors

Jimin Huang

4 papers

Weiguang Han

3 papers

Yanzhao Lai

3 papers

References57 items

The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges

DGR: Decomposition Graph Reconstruction for Question Understanding

Hybrid Deep Reinforcement Learning for Pairs Trading

Stability effects of arbitrage in exchange traded funds: an agent-based model

Structural break-aware pairs trading strategy using deep reinforcement learning

Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning

Published in

Knowledge Discovery and Data Mining(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Jimin Huang

4 papers

Weiguang Han

3 papers

Yanzhao Lai

3 papers

References57 items

The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges

DGR: Decomposition Graph Reconstruction for Question Understanding

Hybrid Deep Reinforcement Learning for Pairs Trading

Stability effects of arbitrage in exchange traded funds: an agent-based model

Structural break-aware pairs trading strategy using deep reinforcement learning

Qianqian Xie

3 papers

Improving Pairs Trading Strategies via Reinforcement Learning

Hierarchical Reinforcement Learning for Integrated Recommendation

Hierarchical Reinforcement Learning for Air-to-Air Combat

On the Autocorrelation of the Stock Market*

The Use of Deep Reinforcement Learning in Tactical Asset Allocation

Dynamic Portfolio Management Based on Pair Trading and Deep Reinforcement Learning

Deep Reinforcement Learning Pairs Trading with a Double Deep Q-Network

Optimizing the Pairs-Trading Strategy Using Deep Reinforcement Learning with Trading and Stop-Loss Boundaries

Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?

Hierarchical Reinforcement Learning for Open-Domain Dialog

A Deep Reinforcement Learning Approach for Automated Cryptocurrency Trading

Overview and History of Statistics for Equity Markets

Attention is All you Need

Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500

Empirical Investigation of an Equity Pairs Trading Strategy

Statistical Arbitrage Pairs Trading Strategies: Review and Outlook

FeUdal Networks for Hierarchical Reinforcement Learning

The Option-Critic Architecture

Pairs trading strategy optimization using the reinforcement learning method: a cointegration approach

Layer Normalization

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation

Asynchronous Methods for Deep Reinforcement Learning

Deep Recurrent Q-Learning for Partially Observable MDPs

The Profitability of Pairs Trading Strategies: Distance, Cointegration, and Copula Methods

Human-level control through deep reinforcement learning

Pairs trading and selection methods: is cointegration superior?

Neural Machine Translation by Jointly Learning to Align and Translate

Trading in the Presence of Cointegration

Are Pairs Trading Profits Robust to Trading Costs

Finding the Optimal Pre-set Boundaries for Pairs Trading Strategy Based on Cointegration Technique

Analytic Solutions for Optimal Statistical Arbitrage Trading

The Epoch-Greedy algorithm for contextual multi-armed bandits

M of a Kind: A Multivariate Approach at Pairs Trading

Statistical Arbitrage: Algorithmic Trading Insights and Techniques

Loss protection in pairs trading through minimum profit bounds: A cointegration approach

Pairs Trading: Quantitative Methods and Analysis

The Illusory Nature of Momentum Profits

Pairs Trading: Performance of a Relative Value Arbitrage Rule

Hierarchical Reinforcement Learning

Feudal Reinforcement Learning

Portfolio Selection

Reinforcement learning in financial markets - a survey

Taking the Human Out of the Loop: A Review of Bayesian Optimization

Pairs Trading via Three-Regime Threshold Autoregressive GARCH Models

Rmsprop: Divide the gradient by a running average of its recent magnitude. coursera: Neural networks for machine learning

Longshort-termmemory

Expert Systems With Applications

on machine learning

KDD ’23, August 6–10, 2023,

Field of Study

EconomicsComputer Science

Journal Information

Name

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Venue Information

Name

Knowledge Discovery and Data Mining

Type

conference

URL

http://www.acm.org/sigkdd/

Alternate Names

KDD
Knowl Discov Data Min

TL;DR

Abstract

Authors

References57 items

The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges

DGR: Decomposition Graph Reconstruction for Question Understanding

Hybrid Deep Reinforcement Learning for Pairs Trading

Stability effects of arbitrage in exchange traded funds: an agent-based model

Structural break-aware pairs trading strategy using deep reinforcement learning

TL;DR

Abstract

Authors

References57 items

The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges

DGR: Decomposition Graph Reconstruction for Question Understanding

Hybrid Deep Reinforcement Learning for Pairs Trading

Stability effects of arbitrage in exchange traded funds: an agent-based model

Structural break-aware pairs trading strategy using deep reinforcement learning

Improving Pairs Trading Strategies via Reinforcement Learning

Hierarchical Reinforcement Learning for Integrated Recommendation

Hierarchical Reinforcement Learning for Air-to-Air Combat

On the Autocorrelation of the Stock Market*

The Use of Deep Reinforcement Learning in Tactical Asset Allocation

Dynamic Portfolio Management Based on Pair Trading and Deep Reinforcement Learning

Deep Reinforcement Learning Pairs Trading with a Double Deep Q-Network

Optimizing the Pairs-Trading Strategy Using Deep Reinforcement Learning with Trading and Stop-Loss Boundaries

Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?

Hierarchical Reinforcement Learning for Open-Domain Dialog

A Deep Reinforcement Learning Approach for Automated Cryptocurrency Trading

Overview and History of Statistics for Equity Markets

Attention is All you Need

Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500

Empirical Investigation of an Equity Pairs Trading Strategy

Statistical Arbitrage Pairs Trading Strategies: Review and Outlook

FeUdal Networks for Hierarchical Reinforcement Learning

The Option-Critic Architecture

Pairs trading strategy optimization using the reinforcement learning method: a cointegration approach

Layer Normalization

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation

Asynchronous Methods for Deep Reinforcement Learning

Deep Recurrent Q-Learning for Partially Observable MDPs

The Profitability of Pairs Trading Strategies: Distance, Cointegration, and Copula Methods

Human-level control through deep reinforcement learning

Pairs trading and selection methods: is cointegration superior?

Neural Machine Translation by Jointly Learning to Align and Translate

Trading in the Presence of Cointegration

Are Pairs Trading Profits Robust to Trading Costs

Finding the Optimal Pre-set Boundaries for Pairs Trading Strategy Based on Cointegration Technique

Analytic Solutions for Optimal Statistical Arbitrage Trading

The Epoch-Greedy algorithm for contextual multi-armed bandits

M of a Kind: A Multivariate Approach at Pairs Trading

Statistical Arbitrage: Algorithmic Trading Insights and Techniques

Loss protection in pairs trading through minimum profit bounds: A cointegration approach

Pairs trading

Pairs Trading: Quantitative Methods and Analysis

The Illusory Nature of Momentum Profits

Pairs Trading: Performance of a Relative Value Arbitrage Rule

The Sharpe Ratio

Hierarchical Reinforcement Learning

Feudal Reinforcement Learning

Q-learning

Portfolio Selection

Reinforcement learning in financial markets - a survey

Taking the Human Out of the Loop: A Review of Bayesian Optimization

Pairs Trading via Three-Regime Threshold Autoregressive GARCH Models

Rmsprop: Divide the gradient by a running average of its recent magnitude. coursera: Neural networks for machine learning

Longshort-termmemory

Expert Systems With Applications

on machine learning

KDD ’23, August 6–10, 2023,

Field of Study

Journal Information

Name

Venue Information

Name

Type

URL

Alternate Names