Uncertainty Quantification and Sensitivity Analysis Exercises

These exercises practise uncertainty quantification and sensitivity analysis as engineering decision tools. Each problem converts uncertainty, variability, sampling evidence or sensitivity into a release, hold, redesign or validation decision.

The examples are simplified, but the habit is realistic: state the limit, quantify uncertainty on the decision metric, identify what drives the conclusion, and decide whether the evidence is strong enough for the consequence.

How to use these exercises

For each problem, compute the requested uncertainty metric and then decide what it means for the engineering action. A result is not complete until the pass/fail boundary, assumptions and residual risk are explicit.

Use local propagation for transparent first-pass checks, Monte Carlo for nonlinear or threshold-driven cases, and validation residuals when the model itself may be wrong.

Release Evidence Notes

Uncertainty and sensitivity evidence should state the model output, decision limit, input distributions, correlation assumptions, sample size, convergence criterion, validation data and consequence of a wrong decision. A percentile or failure probability is not useful unless the input evidence and model boundary are defensible.

Sensitivity evidence should identify which inputs drive the decision and which ones are merely convenient to vary. A high sensitivity can call for better measurement, tighter supplier control, more conservative design margin or a model update; a low sensitivity should not be used to ignore an input outside the validated range.

Engineering Boundary Notes

These exercises use simplified propagation, Monte Carlo and sensitivity calculations. They do not replace a full uncertainty management plan, reliability analysis, model validation, probability elicitation, calibration of input distributions or risk acceptance review. Simulation precision does not compensate for weak input assumptions or an invalid model.

Common Release Mistakes

• reporting many Monte Carlo samples without checking convergence or input evidence;
• treating independent inputs as independent when they share a cause or source;
• quoting a percentile without stating the confidence, sample size and model limit;
• using local linear propagation outside a nonlinear or threshold region;
• focusing on nominal sensitivity while the release decision is controlled by tail risk;
• accepting a model because uncertainty is small even though validation residuals are large.

Scenario Map

Scenario	Main calculation	Engineering decision
Guarded margin	nominal value plus uncertainty guard	Decide whether a nominal pass remains a release pass.
Propagation	derivatives, RSS and covariance	Identify uncertainty contributors and correlation effects.
Monte Carlo	exceedance, percentile and convergence	Decide whether simulated evidence is stable enough.
Sensitivity	finite differences, normalized effects and variance indices	Prioritize which input needs better evidence.
Reliability	limit-state margin and failure probability	Compare risk with an acceptance threshold.
Validation	residuals, coverage and evidence gates	Hold a model when uncertainty claims are not supported.

Validation Package Checklist

• output metric, limit state and decision consequence are defined;
• input distributions, correlations and evidence sources are recorded;
• propagation method or Monte Carlo model is appropriate for the nonlinearity;
• sample size, convergence and tail-estimate stability are checked;
• dominant sensitivities and weak assumptions are named;
• validation residuals, calibration data or independent checks support the model;
• final decision states accept, hold, redesign, collect data or reduce uncertainty.

Exercise 1: Guarded Margin for a Temperature Limit

A component has predicted maximum temperature:

with standard uncertainty:

The release limit is:

Use a guard factor:

Find the nominal margin, guarded temperature and guarded margin.

Solution

The nominal margin is:

The guarded temperature is:

The guarded margin is:

The guarded check passes, but with limited remaining margin.

Engineering Comment

The nominal value alone looked comfortable. The uncertainty guard consumes most of the margin, so the release should require evidence that u_T is credible for the actual board, enclosure and operating load.

Plausibility Check

The guard adds 4.2\ \text{deg C} to the nominal prediction. The original 6.5\ \text{deg C} margin should shrink to 2.3\ \text{deg C}, which it does.

Exercise 2: Local Propagation for a Product Model

A heat loss estimate uses:

Nominal values are:

Independent standard uncertainties are:

Find Q and the standard uncertainty u_Q using relative uncertainty propagation.

Solution

The nominal heat loss is:

For a product:

Substitute:

Calculate:

So:

The output standard uncertainty is:

Engineering Comment

The heat-transfer coefficient dominates because it has the largest relative uncertainty. Improving area measurement would not materially reduce the uncertainty unless U is also better characterized.

Plausibility Check

The largest relative uncertainty is about 8.6\% from U, so a combined result near 10\% is plausible. Ten percent of 1260\ \text{W} is about 126\ \text{W}.

Exercise 3: Correlated Inputs in a Clearance Calculation

A clearance is calculated as:

The standard uncertainties are:

The correlation coefficient is:

Find the standard uncertainty of C.

Solution

For C=X-Y, the sensitivities are:

The covariance is:

The propagated variance is:

Substitute:

Therefore:

Engineering Comment

Positive correlation reduces uncertainty in a difference. If both dimensions move together because they share a process setup, their difference can be more stable than either individual measurement.

Plausibility Check

If the variables were independent, the uncertainty would be \sqrt{0.08^2+0.06^2}=0.10\ \text{mm}. Positive correlation in a subtraction should reduce it, and 0.0529\ \text{mm} does.

Exercise 4: Interval Bound for a Monotonic Stress Model

A simplified stress model is:

The force and area are bounded by:

and:

Find the worst-case maximum stress.

Solution

Stress increases with force and decreases with area. The maximum occurs at maximum force and minimum area:

Convert:

Since:

the stress is:

Engineering Comment

Interval analysis is defensible when only bounds are known. It is conservative for monotonic models, but non-monotonic models may need optimization over the interval rather than endpoint checking.

Plausibility Check

A 10\ \text{kN} load over about 80\ \text{mm}^2 gives roughly 125\ \text{MPa}. The worst-case value should be somewhat higher, so 138.5\ \text{MPa} is plausible.

Exercise 5: Monte Carlo Exceedance Probability

A Monte Carlo study evaluates N=20{,}000 samples of a design metric. The limit is exceeded in:

samples. Estimate the exceedance probability and its approximate sampling standard error.

Solution

The exceedance probability estimate is:

The approximate standard error is:

Substitute:

The estimate is:

Engineering Comment

If the release threshold is 0.5\%, this looks acceptable. If the threshold is 0.3\%, it is not. The decision depends on the risk criterion, not only on the existence of a Monte Carlo run.

Plausibility Check

Seventy-four failures out of twenty thousand is a few tenths of a percent. The standard error is much smaller than the estimate but not negligible near tight limits.

Exercise 6: Sample Count for Mean Convergence

A preliminary simulation gives output standard deviation:

You want the approximate 95\% confidence half-width for the mean to be no more than:

Use:

with:

Find the required sample count.

Solution

Substitute:

Calculate:

Then:

Use at least:

samples for this mean estimate.

Engineering Comment

This sample count is for the mean, not for tail probability or rare failures. A release decision based on the 99th percentile or P_f may require many more samples.

Plausibility Check

The desired half-width is about one ninth of the standard deviation, so a few hundred samples are reasonable.

Exercise 7: Percentile Release Check

A simulated load distribution has:

for the 95th percentile. The allowable load is:

The simulation percentile uncertainty is estimated as:

Use a guarded percentile:

Does the 95th percentile check pass?

Solution

The guarded percentile is:

Compare:

The guarded 95th percentile passes.

The margin is:

Engineering Comment

The margin is not large. If the percentile came from a small sample or an unvalidated distribution, this should be treated as a conditional pass pending stronger tail evidence.

Plausibility Check

The unguarded percentile is 2.8\ \text{kN} below the limit. Adding 1.6\ \text{kN} of uncertainty should leave 1.2\ \text{kN}, which matches the result.

Exercise 8: One-at-a-Time Sensitivity

A baseline model output is:

An input x has baseline value:

When x is increased by 5, the output becomes 258. When x is decreased by 5, the output becomes 223.

Estimate the central finite-difference sensitivity and normalized sensitivity.

Solution

The central sensitivity is:

The normalized sensitivity is:

Engineering Comment

A normalized sensitivity of 0.875 means a 1 percent increase in x produces roughly a 0.875 percent increase in y near the baseline. This is a strong local effect.

Plausibility Check

The output changes by 35 units over a 10 unit input span, so a slope of 3.5 is direct. Since x/y=0.25, the normalized sensitivity should be about one quarter of 3.5, or 0.875.

Exercise 9: Variance Contribution Ranking

Three independent inputs contribute to output uncertainty:

Input	Sensitivity	Standard uncertainty
x_1	4.0	0.20
x_2	1.5	0.60
x_3	8.0	0.05

Compute each variance contribution and identify the dominant input.

Solution

Contribution from x_1:

Contribution from x_2:

Contribution from x_3:

Total variance:

Fractions:

The dominant contributor is x_2.

Engineering Comment

The largest sensitivity does not automatically dominate. Input x_3 has the largest sensitivity, but its uncertainty is small. Input x_2 deserves the first evidence-improvement effort.

Plausibility Check

The products are 0.8, 0.9 and 0.4. Squaring them gives the ranking x_2, x_1, x_3, as calculated.

Exercise 10: First-Order and Total Sensitivity Indices

A global sensitivity study reports:

Input	S_i	S_{T_i}
A	0.42	0.47
B	0.18	0.41
C	0.09	0.12

Which input has the strongest direct effect, and which input shows the strongest interaction effect?

Solution

The strongest direct effect is the largest first-order index:

So input A has the strongest direct effect.

Interaction involvement can be screened by:

For A:

For B:

For C:

Input B shows the strongest interaction effect.

Engineering Comment

Input B would be underestimated by a one-at-a-time study. It matters mainly through interactions, so experiments or simulations should vary it jointly with other inputs.

Plausibility Check

A has the highest direct index, but B has the largest gap between total and first-order effect. The two conclusions are different and both are useful.

Exercise 11: Failure Probability from a Normal Safety Margin

A safety margin is modeled as approximately normal:

with:

Failure occurs when:

Find the reliability index:

and estimate the failure probability using:

Use \Phi(-3.0)=0.00135.

Solution

The reliability index is:

Therefore:

As a percentage:

Engineering Comment

This result depends strongly on the normal-margin assumption. If the margin distribution is skewed, bounded, mixed-mode or driven by rare operational states, the normal approximation may be misleading.

Plausibility Check

A mean margin three standard deviations above zero should correspond to a small lower-tail probability, around one tenth of a percent. The value is plausible.

Exercise 12: Chance Constraint Release Screen

A controller design must satisfy:

A Monte Carlo study with N=10{,}000 samples gives:

samples with Y>50.

Does the chance constraint pass?

Solution

The estimated exceedance probability is:

The estimated satisfaction probability is:

Compare with the requirement:

The nominal Monte Carlo estimate passes.

Engineering Comment

The pass is narrow because the exceedance threshold is 1\% and the estimate is 0.86\%. A release package should include sampling uncertainty and evidence that the input distributions are credible in the tail.

Plausibility Check

Eighty-six exceedances in ten thousand samples is less than one percent. The satisfaction probability should therefore be just above 99\%.

Exercise 13: Robust Objective Comparison

Two design candidates have simulated cost metric J:

Candidate	\operatorname{E}[J]	\sigma_J
A	100	18
B	108	6

For a lower-is-better objective, use:

with:

Which candidate is preferred?

Solution

For candidate A:

For candidate B:

Since:

candidate B is preferred under the robust objective.

Engineering Comment

Candidate A has better nominal expected cost, but its uncertainty is much larger. The robust objective prefers the design with less spread because the decision penalizes risk.

Plausibility Check

The uncertainty penalty for A is 27, while for B it is 9. That difference is large enough to reverse the nominal ranking.

Exercise 14: Validation Residual Bias and RMSE

A model is compared with five independent validation measurements. Residuals are:

Compute the mean bias and RMSE.

Solution

The mean residual is:

The RMSE is:

Calculate:

Engineering Comment

The model has a positive average bias and a larger RMS residual. A release decision should compare both with the uncertainty claimed by the model and with the engineering margin.

Plausibility Check

Residuals are mostly a few units, so an RMSE around 2.6 is reasonable. The positive residuals are larger than the negative ones, so positive bias is expected.

Exercise 15: Prediction Interval Coverage

A model reports 95 percent prediction intervals for n=40 validation cases. The measured value falls inside the reported interval in:

cases. Estimate empirical coverage and decide whether the interval model looks overconfident.

Solution

The empirical coverage is:

So:

The reported intervals claim 95 percent coverage but achieved only 85 percent on validation cases.

The interval model looks overconfident.

Engineering Comment

Undercoverage means the uncertainty bands are too narrow, the validation data are outside the intended envelope, or the model is missing important variability. This is a model-authority problem, not just a plotting issue.

Plausibility Check

Missing 6 cases out of 40 means 15 percent are outside the interval. A 95 percent interval should miss about 5 percent, so the observed coverage is clearly low.

Exercise 16: Evidence Value from Sensitivity Ranking

A release metric has total uncertainty:

The largest variance contributor is input x_1 with contribution fraction:

If new calibration halves the standard uncertainty of x_1, estimate the new total output uncertainty. Assume all other contributions stay the same.

Solution

The total variance is:

The variance from x_1 is:

The remaining variance is:

Halving standard uncertainty quarters the variance contribution:

The new total variance is:

Therefore:

Engineering Comment

Sensitivity ranking becomes valuable when it tells where evidence will reduce decision risk. Here, calibrating x_1 reduces output uncertainty from 5.0 to 3.61, a meaningful improvement.

Plausibility Check

Because x_1 dominates but does not account for all variance, halving its standard uncertainty should reduce total uncertainty substantially but not by half. The result matches that expectation.

Exercise 17: Decision Margin After Improved Evidence

A nominal design metric is:

The upper limit is:

Before improved evidence, output uncertainty is:

After improved evidence, it is:

Use guard factor:

Compare the guarded margins before and after evidence improvement.

Solution

Initial guarded value:

Initial guarded margin:

The initial guarded check fails.

Improved guarded value:

Improved guarded margin:

After improved evidence, the guarded check passes with margin 0.8.

Engineering Comment

This shows why reducing uncertainty can be equivalent to design improvement. The nominal design did not change, but the evidence became strong enough to support a guarded release.

Plausibility Check

The original nominal margin is 8. A two-sigma guard of 10 fails by 2; a two-sigma guard of 7.2 passes by 0.8. The arithmetic is consistent.

Exercise 18: UQ Release Decision Gate

A model-supported release package gives:

Check	Requirement	Result
Guarded margin	\ge 0	1.4
Monte Carlo exceedance	\le 0.5\%	0.42\%
Dominant input evidence	required	calibrated
Validation RMSE	\le 3.0	3.4
Prediction interval coverage	\ge 90\%	86\%

Should the model-supported decision be released?

Solution

The guarded margin passes:

The exceedance probability passes:

The dominant input evidence is present.

The validation RMSE fails:

Prediction interval coverage also fails:

The model-supported decision should not be released.

Engineering Comment

The UQ calculation appears acceptable, but validation does not support the model’s claimed accuracy. Release should be held for recalibration, model-form review, wider uncertainty bands or a restricted operating envelope.

Plausibility Check

Two validation gates fail. Even if uncertainty propagation and Monte Carlo checks pass, unsupported model accuracy is enough to block release.

Review Checklist

Before accepting an uncertainty or sensitivity analysis, check:

• the decision metric, acceptance threshold and action are explicit;
• input evidence and units are traceable;
• variability, lack of knowledge and model-form uncertainty are separated;
• independence or correlation assumptions are justified;
• local propagation is valid near the operating point;
• Monte Carlo convergence is checked on the decision metric;
• sensitivity ranks the input that controls the action, not a convenient output;
• failure probability or percentile checks include sampling uncertainty;
• validation residuals and interval coverage support the claimed uncertainty;
• the report states what new evidence would change the decision.

Common Mistakes

• Treating a nominal pass as release evidence without a guarded margin.
• Assuming all inputs are independent because covariance data are inconvenient.
• Using local linear propagation through threshold, saturation or discontinuity.
• Reporting a Monte Carlo histogram without exceedance or percentile evidence.
• Ranking sensitivities on mean performance when the decision depends on tail risk.
• Ignoring interaction effects when total sensitivity is much larger than first-order sensitivity.
• Reporting failure probability without defining failure.
• Validating a model with calibration data and calling it independent.
• Treating low RMSE as sufficient when interval coverage is poor.
• Reducing uncertainty in an input that barely contributes to the decision metric.

The central habit is to make uncertainty actionable. A good UQ result tells which decision is stable, which input can overturn it, and which evidence would most efficiently reduce risk.