|Résumé :||Continuous optimization problems are optimization problems where all variables
have a domain that typically is a subset of the real numbers; mixed discrete-continuous
optimization problems have additionally other types of variables, so
that some variables are continuous and others are on an ordinal or categorical
scale. Continuous and mixed discrete-continuous problems have a wide range
of applications in disciplines such as computer science, mechanical or electrical
engineering, economics and bioinformatics. These problems are also often hard to
solve due to their inherent difficulties such as a large number of variables, many
local optima or other factors making problems hard. Therefore, in this thesis our
focus is on the design, engineering and configuration of high-performing heuristic
We tackle continuous and mixed discrete-continuous optimization problems
with two classes of population-based heuristic algorithms, ant colony optimization
(ACO) algorithms and evolution strategies. In a nutshell, the main contributions
of this thesis are that (i) we advance the design and engineering of ACO algorithms to algorithms that are competitive or superior to recent state-of-the-art
algorithms for continuous and mixed discrete-continuous optimization problems,
(ii) we improve upon a specific state-of-the-art evolution strategy, the covariance
matrix adaptation evolution strategy (CMA-ES), and (iii) we extend CMA-ES to
tackle mixed discrete-continuous optimization problems.
More in detail, we propose a unified ant colony optimization (ACO) framework
for continuous optimization (UACOR). This framework synthesizes algorithmic
components of two ACO algorithms that have been proposed in the literature
and an incremental ACO algorithm with local search for continuous optimization,
which we have proposed during my doctoral research. The design of UACOR
allows the usage of automatic algorithm configuration techniques to automatically
derive new, high-performing ACO algorithms for continuous optimization. We also
propose iCMAES-ILS, a hybrid algorithm that loosely couples IPOP-CMA-ES, a
CMA-ES variant that uses a restart schema coupled with an increasing population
size, and a new iterated local search (ILS) algorithm for continuous optimization.
The hybrid algorithm consists of an initial competition phase, in which IPOP-CMA-ES and the ILS algorithm compete for further deployment during a second
phase. A cooperative aspect of the hybrid algorithm is implemented in the form
of some limited information exchange from IPOP-CMA-ES to the ILS algorithm
during the initial phase. Experimental studies on recent benchmark functions
suites show that UACOR and iCMAES-ILS are competitive or superior to other
To tackle mixed discrete-continuous optimization problems, we extend ACOMV
and propose CESMV, an ant colony optimization algorithm and a covariance matrix adaptation evolution strategy, respectively. In ACOMV and CESMV , the decision variables of an optimization problem can be declared as continuous, ordinal, or categorical, which allows the algorithm to treat them adequately. ACOMV and
CESMV include three solution generation mechanisms: a continuous optimization
mechanism, a continuous relaxation mechanism for ordinal variables, and a categorical optimization mechanism for categorical variables. Together, these mechanisms allow ACOMV and CESMV to tackle mixed variable optimization problems.
We also propose a set of artificial, mixed-variable benchmark functions, which can
simulate discrete variables as ordered or categorical. We use them to automatically tune ACOMV and CESMV's parameters and benchmark their performance.
Finally we test ACOMV and CESMV on various real-world continuous and mixed-variable engineering optimization problems. Comparisons with results from the
literature demonstrate the effectiveness and robustness of ACOMV and CESMV
on mixed-variable optimization problems.
Apart from these main contributions, during my doctoral research I have accomplished a number of additional contributions, which concern (i) a note on the
bound constraints handling for the CEC'05 benchmark set, (ii) computational results for an automatically tuned IPOP-CMA-ES on the CEC'05 benchmark set and
(iii) a study of artificial bee colonies for continuous optimization. These additional
contributions are to be found in the appendix to this thesis.