Introduction

The original contributions of this work are:

• The free, stand-alone implementation in C++ of the whole algorithm. Most optimizer are expensive piece of software. Usually ``free'' optimizers are using expensive, external, unavailable libraries (such libraries are usually the Harwell Libraries, the NAG tools, the NPSOL non-linear solver,...). In CONDOR, there is no call to expensive, external libraries.
• The algorithm used for the parallelization of the optimization procedure.
• The algorithm used for the constrained case.
• The assembly of different well-known algorithms in one code (In particular, the code of the Chapter 4 is a completely new implementation of the Moré and Sorensen algorithm which has never been used in conjunction with quadratical model build by interpolation).
• The bibliographical work needed to:
  • Understand and implement all the parts of the algorithm.
  • Acquire a huge amount of background knowledge in constrained and unconstrained continuous optimization. This is necessary to be able to review all the available techniques and choose the best promising one. However, I will not present here a full review of state-of-the-art techniques (for a starting point, see the excellent book ``Trust-region Methods'' by Andrew R. Conn, Nicholas I.M.Gould, and Philippe L.Toint [CGT00a]). A very short review for the constrained case is given in Chapter 8. In particular, this Section contains:
    • A short, concise and original introduction to Primal-Dual methods for non-linear optimization. The link between Primal-Dual methods and Barrier methods is motivated and intuitively explained.
    • A short, complete, intuitive and original description and justification of the Second Order Step (SOC) used inside SQP algorithms.
• The redaction of the complete description of the CONDOR algorithm (``COnstrained, Non-linear, Direct, parallel Optimization using trust Region method for high-computing load function''). This description is intended for graduate school level and requires no a priori knowledge in the field of continuous optimization. Most optimization books are very formal and give no insight to the reader. This thesis is written in a informal way, trying to give an intuition to the reader of what is going on, rather than giving pure formal demonstration which are usually hiding the intuition behind many formalism. The thesis is a complete start from scratch (or nearly) and is thus a good introduction to the optimization theory for the beginner.
• The comparison of the new algorithm CONDOR with other famous algorithms: CFSQP, DFO, UOBYQA, PDS,etc. The experimental results of this comparison shows very good performance of CONDOR compared to the other algorithms. On nearly all the unconstrained problems, CONDOR finds the optimal point of the objective function in substantially fewer evaluation of the objective function than its competitor, especially when more than one CPU is used. In the constrained case, the performances of CONDOR are comparable to the best algorithms. Preliminary results indicates that on box and linear constraints only, CONDOR also outperforms its competitors. However, more numerical results are needed to assert definitively this last statement.
• The application of a gradient-based optimizer on a objective function based on CFD code is unusual. This approach is usually rejected and is considered by many researcher as a ``dead-end''. The validation of the usefulness of the gradient-based approach is a primordial result.
• The algorithm to solve the secondary Trust-Region subproblem described in Section 5 is slightly different that the algorithm proposed by Powell. Numerical results exhibit better performances for the CONDOR algorithm.

The other ideas used inside CONDOR are mostly coming from recent work of M.J.D.Powell [Pow00].