Glossary term
Iterative Solver
A numerical method that approximates the solution of equations through repeated updates until a convergence criterion is met.
Definition
methodAn iterative solver is a numerical algorithm that repeatedly improves an approximate solution until residuals, updates, or objective changes satisfy a stopping criterion.
Iterative solvers are used when direct solution is too expensive, impossible, or unnecessary. They appear in linear systems, nonlinear equations, optimization, finite-element analysis, computational fluid dynamics, circuit simulation, inverse problems, and machine learning. Their performance depends on conditioning, initial guess, scaling, preconditioning, convergence tolerance, stopping rule, discretization error, and numerical stability.
An iterative solver starts from an approximate solution and repeatedly updates it. The process stops when a convergence criterion is satisfied or when the algorithm reaches an iteration, time, or error limit. Iterative methods are essential when the problem is too large for direct solution, when the equations are nonlinear, or when an approximate solution is sufficient.
For a linear system:
an iterative solver generates approximations x_0, x_1, x_2,\ldots and monitors the residual:
If the residual becomes small enough relative to the problem scale, the solution is accepted. For nonlinear equations or optimization problems, the solver may monitor residual norm, gradient norm, update size, objective reduction, constraint violation, or physical balance error.
Types
Stationary methods such as Jacobi, Gauss-Seidel, and successive over-relaxation are simple but can be slow. Krylov subspace methods such as conjugate gradient, GMRES, and BiCGSTAB are widely used for large sparse linear systems. Newton methods solve nonlinear problems by repeatedly linearizing them with a Jacobian matrix. Gradient-based methods update variables using first-order derivative information. Fixed-point iterations repeatedly apply a mapping until the solution stops changing.
Preconditioning is often the difference between practical and impractical computation. A preconditioner transforms the problem so the iterative method converges faster, ideally without changing the true solution. In finite-element and CFD models, poor mesh quality, high contrast material properties, incompressibility, contact, or multiphysics coupling can make preconditioning essential.
Convergence and accuracy
Convergence does not automatically mean the engineering answer is correct. A solver can converge to the wrong solution if the model is wrong, boundary conditions are wrong, discretization error is large, or the tolerance is too loose. Conversely, a solver may fail to converge because the physical problem is unstable, the initial guess is poor, the mesh is poor, the time step is too large, or the formulation is ill-conditioned.
Stopping criteria must be scaled to the problem. An absolute residual tolerance that is strict for one unit system may be meaningless in another. Relative residuals, normalized balances, and engineering quantity checks are usually more informative.
Common mistakes
A common mistake is treating non-convergence as only a software problem. It can reveal modelling errors, incompatible constraints, unstable physics, bad material data, or discontinuous loads. Another mistake is over-tightening solver tolerances while ignoring mesh convergence or measurement uncertainty. Good simulation practice reports solver method, preconditioner, tolerances, residual history, iteration count, discretization checks, and any quantities used to judge engineering convergence.