Population-based heuristic algorithms for continuous and mixed discrete-continuous optimization problems

have a domain that typically is a subset of the real numbers; mixed discrete-continuous

optimization problems have additionally other types of variables, so

that some variables are continuous and others are on an ordinal or categorical

scale. Continuous and mixed discrete-continuous problems have a wide range

of applications in disciplines such as computer science, mechanical or electrical

engineering, economics and bioinformatics. These problems are also often hard to

solve due to their inherent difficulties such as a large number of variables, many

local optima or other factors making problems hard. Therefore, in this thesis our

focus is on the design, engineering and configuration of high-performing heuristic

optimization algorithms.

We tackle continuous and mixed discrete-continuous optimization problems

with two classes of population-based heuristic algorithms, ant colony optimization

(ACO) algorithms and evolution strategies. In a nutshell, the main contributions

of this thesis are that (i) we advance the design and engineering of ACO algorithms to algorithms that are competitive or superior to recent state-of-the-art

algorithms for continuous and mixed discrete-continuous optimization problems,

(ii) we improve upon a specific state-of-the-art evolution strategy, the covariance

matrix adaptation evolution strategy (CMA-ES), and (iii) we extend CMA-ES to

tackle mixed discrete-continuous optimization problems.

More in detail, we propose a unified ant colony optimization (ACO) framework

for continuous optimization (UACOR). This framework synthesizes algorithmic

components of two ACO algorithms that have been proposed in the literature

and an incremental ACO algorithm with local search for continuous optimization,

which we have proposed during my doctoral research. The design of UACOR

allows the usage of automatic algorithm configuration techniques to automatically

derive new, high-performing ACO algorithms for continuous optimization. We also

propose iCMAES-ILS, a hybrid algorithm that loosely couples IPOP-CMA-ES, a

CMA-ES variant that uses a restart schema coupled with an increasing population

size, and a new iterated local search (ILS) algorithm for continuous optimization.

The hybrid algorithm consists of an initial competition phase, in which IPOP-CMA-ES and the ILS algorithm compete for further deployment during a second

phase. A cooperative aspect of the hybrid algorithm is implemented in the form

of some limited information exchange from IPOP-CMA-ES to the ILS algorithm

during the initial phase. Experimental studies on recent benchmark functions

suites show that UACOR and iCMAES-ILS are competitive or superior to other

state-of-the-art algorithms.

To tackle mixed discrete-continuous optimization problems, we extend ACOMV

and propose CESMV, an ant colony optimization algorithm and a covariance matrix adaptation evolution strategy, respectively. In ACOMV and CESMV , the decision variables of an optimization problem can be declared as continuous, ordinal, or categorical, which allows the algorithm to treat them adequately. ACOMV and

CESMV include three solution generation mechanisms: a continuous optimization

mechanism, a continuous relaxation mechanism for ordinal variables, and a categorical optimization mechanism for categorical variables. Together, these mechanisms allow ACOMV and CESMV to tackle mixed variable optimization problems.

We also propose a set of artificial, mixed-variable benchmark functions, which can

simulate discrete variables as ordered or categorical. We use them to automatically tune ACOMV and CESMV's parameters and benchmark their performance.

Finally we test ACOMV and CESMV on various real-world continuous and mixed-variable engineering optimization problems. Comparisons with results from the

literature demonstrate the effectiveness and robustness of ACOMV and CESMV

on mixed-variable optimization problems.

Apart from these main contributions, during my doctoral research I have accomplished a number of additional contributions, which concern (i) a note on the

bound constraints handling for the CEC'05 benchmark set, (ii) computational results for an automatically tuned IPOP-CMA-ES on the CEC'05 benchmark set and

(iii) a study of artificial bee colonies for continuous optimization. These additional

contributions are to be found in the appendix to this thesis.