Optimization

Optimization is the discipline of selecting the best element from a set of feasible alternatives according to a defined criterion. The standard formulation of a continuous optimization problem is:

where x = (x_1, x_2, \ldots, x_n)^T is the vector of decision variables, f: \mathbb{R}^n \to \mathbb{R} is the objective function to be minimized, g_i are inequality constraints, and h_j are equality constraints. A maximization problem is converted to minimization by negating the objective: \max f(x) \equiv \min -f(x). The feasible set \mathcal{F} is the set of all x satisfying all constraints simultaneously.

Local and global optima

A point x^* is a local minimum if f(x^*) \leq f(x) for all x in some neighbourhood of x^*. It is a global minimum if this inequality holds for all x \in \mathcal{F}. For unconstrained smooth problems, necessary conditions for a local minimum require the gradient to vanish: \nabla f(x^*) = 0. The nature of the stationary point — minimum, maximum, or saddle — is determined by the Hessian \nabla^2 f(x^*). For constrained problems, the optimality conditions generalise to the Karush–Kuhn–Tucker (KKT) conditions, treated in a dedicated entry.

Classification of optimization problems

The structure of the objective function and constraints determines the appropriate solution method. The fundamental distinction is between convex and non-convex problems: in a convex problem, the feasible set is convex and the objective is a convex function, guaranteeing that any local minimum is also the global minimum. Linear programming and quadratic programming are the most structured convex classes. Non-convex problems — including most structural topology problems, combinatorial scheduling, and neural network training — may have many local optima, and global optimality cannot be guaranteed by gradient-based methods.

When decision variables are required to be integers or binary, the problem becomes a mixed-integer program. When multiple competing objectives must be optimised simultaneously, the problem has no single optimal solution but a Pareto front of trade-off solutions.

Engineering applications

Optimization pervades every engineering discipline: structural topology optimization, economic dispatch in power grids, process optimization in chemical plants, optimal control in aerospace and robotics, parameter fitting in data-driven models, and reliability-based design. The specific methods used, including linear programming, gradient descent, Newton-type methods, sequential quadratic programming, surrogate optimization, or metaheuristics, depend on the mathematical structure and the cost of evaluating candidate solutions.

Engineering interpretation

An optimization result is only as meaningful as the model used to define it. The objective must represent the actual decision criterion, constraints must represent real limits, and decision variables must correspond to choices that can be manufactured, operated, controlled, or purchased. A design that is optimal for an incomplete model can be fragile when uncertainty, tolerance variation, degradation, human operation, or off-design conditions are introduced.

For this reason, engineering optimization is often followed by sensitivity analysis, uncertainty analysis, robustness checks, validation against higher-fidelity models, and review of active constraints. The final selected design may not be the mathematical optimum if a nearby point offers better manufacturability, maintainability, safety margin, or stakeholder acceptance.

Common mistakes

A common mistake is to optimize a proxy objective because it is easy to compute, then treat the result as if it optimizes the real business or engineering outcome. Another is to ignore scaling: poorly scaled variables and constraints can make solvers unreliable or make one term dominate a composite objective. A credible optimization study documents variables, bounds, objective definition, constraints, solver settings, convergence criteria, local versus global claims, and sensitivity of the result to modelling assumptions.