Mixed-Integer Convex Nonlinear Optimization with Gradient-Boosted Trees Embedded

Abstract

Decision trees usefully represent sparse, high-dimensional, and noisy data. Having learned a function from these data, we may want to thereafter integrate the function into a larger decision-making problem, for example, for picking the best chemical process catalyst. We study a large-scale, industrially relevant mixed-integer nonlinear nonconvex optimization problem involving both gradient-boosted trees and penalty functions mitigating risk. This mixed-integer optimization problem with convex penalty terms broadly applies to optimizing pretrained regression tree models. Decision makers may wish to optimize discrete models to repurpose legacy predictive models or they may wish to optimize a discrete model that accurately represents a data set. We develop several heuristic methods to find feasible solutions and an exact branch-and-bound algorithm leveraging structural properties of the gradient-boosted trees and penalty functions. We computationally test our methods on a concrete mixture design instance and a chemical catalysis industrial instance.