Formula sheet

Mathematical Modeling and Optimization Formula Sheet

Modeling formulas for residuals, discretization, conditioning, Newton updates, KKT conditions, uncertainty, calibration, sensitivity, and Pareto trade-offs.

Branch: Mathematical Engineering
Content: Formula sheet
Updated: May 29, 2026
Revision: v1.0.8 · reviewed

This formula sheet collects common relationships used in mathematical modeling, numerical simulation, uncertainty analysis, and engineering optimization. Use it as a compact reference, not as a substitute for defining model assumptions, units, boundary conditions, data quality, and validation evidence.

General model form

Implicit model:

F(x,p)=0

where $x$ is the vector of unknowns and $p$ is the vector of parameters.

Output model:

y=M(x,p)

Residual:

r=F(x,p)

Residual norm:

\|r\|_2=\sqrt{\sum_i r_i^2}

A small residual indicates consistency with the equations, not necessarily correctness of the physical model.

Scaling and nondimensionalization

Scaled variable:

\displaystyle \tilde{x}_i=\frac{x_i-x_{ref,i}}{s_i}

Normalized residual:

\displaystyle \tilde{r}_i=\frac{r_i}{r_{scale,i}}

Scaled objective:

\displaystyle \tilde{f}=\frac{f-f_{ref}}{f_{scale}}

Dimensionless input ratio:

\displaystyle \Pi_i=\frac{x_i}{x_{ref,i}}

Scaling should preserve the engineering meaning of variables while improving numerical conditioning, solver tolerance selection, and comparison between objectives with different units.

Error measures

Absolute error:

e_{abs}=|x_{approx}-x_{ref}|

Relative error:

\displaystyle e_{rel}=\frac{|x_{approx}-x_{ref}|}{|x_{ref}|}

Percent error:

\displaystyle e_{\%}=100\frac{|x_{approx}-x_{ref}|}{|x_{ref}|}

Root mean square error:

\displaystyle RMSE=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2}

Use relative error carefully when the reference value is near zero.

Discretization

Forward finite difference:

\displaystyle \frac{du}{dx}\approx \frac{u_{i+1}-u_i}{\Delta x}

Central finite difference:

\displaystyle \frac{du}{dx}\approx \frac{u_{i+1}-u_{i-1}}{2\Delta x}

Second derivative:

\displaystyle \frac{d^2u}{dx^2}\approx \frac{u_{i+1}-2u_i+u_{i-1}}{\Delta x^2}

Time-step update form:

u^{n+1}=u^n+\Delta t\,\Phi(u^n,t^n)

Discretization error often decreases as mesh spacing or time step decreases, but stability and cost must also be checked.

Mesh convergence

Solution change under refinement:

\displaystyle \Delta Q=\frac{|Q_{fine}-Q_{coarse}|}{|Q_{fine}|}

Observed convergence ratio:

\displaystyle R=\frac{|Q_{h}-Q_{h/2}|}{|Q_{h/2}-Q_{h/4}|}

If the method is in the asymptotic range:

R\approx 2^p

where $p$ is the observed order of accuracy.

Mesh convergence should be evaluated on the engineering quantity of interest, not only on global solver residuals.

Finite element system

Linear static form:

K u=f

where $K$ is the stiffness matrix, $u$ is the unknown vector, and $f$ is the load vector.

Residual form:

r=K u-f

General nonlinear form:

R(u)=0

Tangent linearization:

K_T \Delta u=-R(u_k)

Update:

u_{k+1}=u_k+\Delta u

Boundary conditions, load representation, element type, and material model determine whether the finite element result is meaningful.

Linear systems and conditioning

Linear system:

A x=b

Residual:

r=b-Ax

Condition number:

\kappa(A)=\|A\|\|A^{-1}\|

Large condition numbers can amplify input errors and roundoff errors.

Iterative convergence criterion:

\displaystyle \frac{\|r_k\|}{\|r_0\|}<\epsilon

Choose $\epsilon$ based on the engineering decision, not only on a software default.

Newton method

Scalar Newton update:

\displaystyle x_{k+1}=x_k-\frac{f(x_k)}{f'(x_k)}

Vector Newton update:

x_{k+1}=x_k-J(x_k)^{-1}F(x_k)

Linearized step:

J(x_k)\Delta x=-F(x_k)

x_{k+1}=x_k+\Delta x

The Jacobian matrix is:

\displaystyle J_{ij}=\frac{\partial F_i}{\partial x_j}

In large systems, solve the linearized system directly or iteratively instead of forming an explicit inverse.

Optimization problem

Unconstrained optimization:

\min_x f(x)

Constrained optimization:

\min_x f(x)

subject to:

g_i(x)\le 0,\quad h_j(x)=0

Feasible set:

\mathcal{F}=\{x:g_i(x)\le 0,\ h_j(x)=0\}

An optimum outside the feasible set is not an engineering solution.

Gradient descent and Newton optimization

Gradient descent:

x_{k+1}=x_k-\alpha_k \nabla f(x_k)

where $\alpha_k$ is the step size.

Newton optimization step:

x_{k+1}=x_k-H(x_k)^{-1}\nabla f(x_k)

where $H$ is the Hessian matrix.

First-order necessary condition for an unconstrained differentiable optimum:

\nabla f(x^*)=0

Second-order sufficient condition for a local minimum:

H(x^*) \succ 0

Scaling, step-size control, and starting point strongly affect practical convergence.

Linear programming

Standard minimization form:

\min_x c^T x

subject to:

A x\le b

x\ge 0

Linear programming is convex when the objective and feasible region are linear. If a feasible optimum exists, at least one optimum occurs at a vertex of the feasible polytope.

KKT conditions

Lagrangian:

\mathcal{L}(x,\lambda,\mu)=f(x)+\sum_i \lambda_i g_i(x)+\sum_j \mu_j h_j(x)

Stationarity:

\nabla_x \mathcal{L}(x^*,\lambda^*,\mu^*)=0

Primal feasibility:

g_i(x^*)\le 0,\quad h_j(x^*)=0

Dual feasibility:

\lambda_i^*\ge 0

Complementary slackness:

\lambda_i^*g_i(x^*)=0

KKT conditions require suitable regularity assumptions. In convex problems, they can provide strong optimality conditions.

Probability and uncertainty

Expected value:

E[X]=\int_{-\infty}^{\infty}x f_X(x)\,dx

Variance:

Var(X)=E[(X-E[X])^2]

Standard deviation:

\sigma=\sqrt{Var(X)}

Propagation by first-order linearization:

\displaystyle \sigma_y^2\approx \sum_i \left(\frac{\partial y}{\partial x_i}\right)^2\sigma_{x_i}^2

For correlated inputs:

\sigma_y^2\approx \nabla y^T \Sigma_x \nabla y

where $\Sigma_x$ is the input covariance matrix.

Sensitivity and calibration

Local sensitivity to parameter $p_i$ :

\displaystyle S_i=\frac{\partial y}{\partial p_i}

Normalized sensitivity:

\displaystyle \hat{S}_i=\frac{p_i}{y}\frac{\partial y}{\partial p_i}

Least-squares calibration objective:

\displaystyle J(p)=\frac{1}{2}\sum_{i=1}^{n}\left(y_i-M_i(p)\right)^2

Weighted least squares:

\displaystyle J_w(p)=\frac{1}{2}(y-M(p))^T W (y-M(p))

Regularized calibration:

J_\lambda(p)=J(p)+\lambda\|Lp\|_2^2

Calibration should report fitted parameters, bounds, data sources, residual structure, validation data, and whether the parameter set is identifiable.

Monte Carlo estimates

Sample mean:

\displaystyle \bar{y}=\frac{1}{N}\sum_{i=1}^{N}y_i

Sample variance:

\displaystyle s_y^2=\frac{1}{N-1}\sum_{i=1}^{N}(y_i-\bar{y})^2

Estimated failure probability:

\displaystyle \hat{P}_f=\frac{N_f}{N}

Standard error of a sample mean:

\displaystyle SE=\frac{s_y}{\sqrt{N}}

Monte Carlo accuracy improves slowly with sample size, so rare-event estimates may require specialized methods.

Kalman prediction and update

State prediction:

\hat{x}_{k|k-1}=A\hat{x}_{k-1|k-1}+B u_k

Covariance prediction:

P_{k|k-1}=A P_{k-1|k-1}A^T+Q

Innovation:

v_k=z_k-H\hat{x}_{k|k-1}

Innovation covariance:

S_k=H P_{k|k-1}H^T+R

Kalman gain:

K_k=P_{k|k-1}H^T S_k^{-1}

State update:

\hat{x}_{k|k}=\hat{x}_{k|k-1}+K_k v_k

Covariance update:

P_{k|k}=(I-K_kH)P_{k|k-1}

The filter is only as credible as its dynamic model, measurement model, and noise assumptions.

Transforms

Laplace transform:

F(s)=\int_0^\infty f(t)e^{-st}\,dt

Discrete-time z-transform:

X(z)=\sum_{k=-\infty}^{\infty}x[k]z^{-k}

Discrete Fourier transform:

X[k]=\sum_{n=0}^{N-1}x[n]e^{-j2\pi kn/N}

The Fast Fourier Transform is an efficient algorithm for computing the discrete Fourier transform.

Sampling relation:

\displaystyle f_s=\frac{1}{T_s}

where $f_s$ is sampling frequency and $T_s$ is sampling period.

Multi-objective optimization

Vector objective:

\min_x \left[f_1(x), f_2(x), \ldots, f_m(x)\right]

Pareto dominance:

x_a \prec x_b

if $x_a$ is no worse in all objectives and strictly better in at least one objective.

Pareto-optimal condition:

\nexists x\in\mathcal{F}: x\prec x^*

Weighted-sum scalarization:

\min_x \sum_{i=1}^{m} w_i f_i(x)

where $w_i\ge 0$ and $\sum_i w_i=1$ .

Weights encode priorities. They do not remove the need to review the engineering trade-off.

REF

Disciplines