Airoyoung dissertation award 2025

This thesis advances the field of Derivative-Free Optimization (DFO) by addressing two critical challenges: the rigorous theoretical understanding of line-search algorithms and the effective handling of general nonlinear constraints in black-box settings. The first part of this work presents the first worst-case complexity analysis for line-search DFO methods. We establish iteration and function evaluation bounds that match those of established direct search methods, filling a significant theoretical gap. A novel result is also provided, bounding the total number of iterations where the stationarity measure is above a given tolerance. This analysis is extended to bound-constrained problems, where we further prove a finite active-set identification property under standard assumptions, a result not generally available for direct search. The second contribution is a novel mixed penalty-barrier framework for nonlinearly constrained problems. For the first time in a derivative-free context, this work introduces a logarithmic barrier for inequality constraints, combined with an exterior penalty for equalities. A key theoretical breakthrough was proving convergence to stationary points without requiring convexity assumptions on the constrained functions, overcoming the challenge of a non-Lipschitz merit function near the feasible boundary. This framework led to the development of two new algorithms, LOG-DFL (line-search) and LOG-DS (direct search), both equipped with adaptive, problem-driven rules for updating the penalty parameter. The practical efficacy of this framework is demonstrated through extensive numerical experiments, where the proposed methods show competitive and often superior performance against state-of-the-art solvers. Overall, this work delivers theoretically sound and computationally robust tools that expand the capabilities of DFO for solving complex, real-world constrained optimization problems.