Original scientific paper
https://doi.org/10.17535/crorr.2018.0001
An analysis of covariance parameters in Gaussian process-based optimization
Hossein Mohammadi
; College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK
Rodolphe Le Riche
; Mines School of Saint Etienne, H. Fayol Institute, Saint-Etienne, France
Xavier Bay
; Mines Scfool of Saint Etienne, H. Fayol Institute, Saint-Etienne, France
Eric Touboul
; Mines Scfool of Saint Etienne, H. Fayol Institute, Saint-Etienne, France
Abstract
The need for globally optimizing expensive-to-evaluate functions frequently occurs in many real-world applications. Among the methods developed for solving such problems, the Efficient Global Optimization (EGO) is regarded as one of the state-of-the-art unconstrained continuous optimization algorithms. The surrogate model used in EGO is a Gaussian process (GP) conditional on data points. The most important control on the efficiency of the EGO algorithm is the GP covariance function (or kernel), which is taken as a parameterized function. In this paper, we theoretically and empirically analyze the effect of the covariance parameters, the so-called "characteristic length scale" and "nugget", on EGO performance. More precisely, we analyze the EGO algorithm with fixed covariance parameters and compare them to the standard setting where they are statistically estimated. The limit behavior of EGO with very small or very large characteristic length scales is identified. Experiments show that a "small" nugget should be preferred to its maximum likelihood estimate. Overall, this study contributes to a better theoretical and practical understanding of a key optimization algorithm.
Keywords
Covariance kernel; EGO; Gaussian process; global optimization
Hrčak ID:
203889
URI
Publication date:
24.7.2018.
Visits: 1.406 *