Sobol
Overview
Sobol sensitivity indices quantify parameter importance through variance decomposition of the objective function. They work for linear and nonlinear functions alike and can measure interaction effects between parameters. Values are in [0, 1].
In v0.1, the reported values estimate sensitivity of a fitted quadratic Ridge surrogate under independent uniform inputs, not of the empirical joint Trial distribution or the constrained design domain. The input measure and surrogate quality are essential to their interpretation.
Tunny Dashboard provides two indices:
| Index | Symbol | Meaning |
|---|---|---|
| First-order | Fraction of variance explained by alone | |
| Total-effect | alone plus all interactions involving |
Theory
For independent inputs and positive output variance, the model output's variance is decomposed as:
First-order index:
Total-effect index:
where means "all parameters except ."
The theoretical indices satisfy . A large gap indicates interactions in the evaluated model under the chosen input measure.
Note on finite-sample estimates: the Saltelli first-order estimator and the Jansen total-effect estimator are computed independently, so raw estimates can violate in finite samples. Tunny Dashboard enforces the property after estimation by taking before clamping both values to .
How Tunny Dashboard Computes This
A quadratic Ridge surrogate is fitted using only rows where all parameter and objective values are finite. At least two retained rows, one parameter, and one objective are required. Parameter ranges are computed from those retained rows.
For each nondegenerate parameter range, the input measure is , independently across parameters. A and B contain separate independent uniform draws; each replaces column of A with column of B. These are observed ranges, not declared search ranges, and sampling does not reproduce empirical Trial frequencies or dependencies.
If , the range is expanded to , rather than keeping the parameter fixed. The range helper also has an empty-column fallback of , but empty columns are not normal accepted input: at least two valid rows are required.
All A, B, and outputs are surrogate predictions, with no simulation rerun, domain feasibility validation, or snapping to discrete values. First-order indices use Saltelli (2010); total-effect indices use Jansen (1999). Near-zero surrogate output variance ($\mathrm{Var}_Y < \texttt{f64::EPSILON}$) returns zeros for both indices; otherwise the max-then-clamp post-processing above is applied.
Parameters
| Setting | Value |
|---|---|
| Saltelli sample count | 1,024 |
| Ridge regularization strength | 1.0 |
| Random number generator | ChaCha8 (deterministic, seed 0xDEAD_BEEF_1234_5678) |
Interpreting Results
Interpret these patterns as effects in the fitted surrogate under the independent uniform input measure, not as guarantees about the actual objective function:
- high, → parameter has a large independent effect, few interactions.
- small, high → parameter mainly acts through interactions with others.
- Both near zero → parameter has little influence on surrogate output variance under this measure.
Strengths and Limitations
Strengths:
- Captures linear, quadratic, and pairwise interaction effects in the surrogate.
- Values in [0, 1] allow easy comparison across parameters.
- indicates interactions in the surrogate under this measure.
Limitations:
- Results depend on surrogate model quality (quadratic Ridge).
- Accuracy decreases when the true function is highly nonlinear.
- Correlations and conditional parameter activation in the Trials are not preserved. Independent sampling over the rectangular product of ranges can produce unobserved combinations or points outside constrained/nonrectangular domains. Predictions there can be extrapolations.
- String categories are label-encoded into IDs 0.0, 1.0, … in order of first appearance; numeric category labels are treated as numeric values. Continuous uniform sampling can draw fractional IDs that are not real categories. The surrogate imposes artificial ordering and distances on these IDs, so this is not categorical-aware sensitivity analysis and results can depend on encoding.
When to Use
- When interactions between parameters are important.
- When global sensitivity of a quadratic Ridge surrogate under independent uniform inputs is an appropriate approximation.
- After Spearman / Ridge screening identifies top candidates.
References
- Saltelli, A. et al. (2010). Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Computer Physics Communications, 181(2), 259–270. https://doi.org/10.1016/j.cpc.2009.09.018
- Jansen, M. J. W. (1999). Analysis of variance designs for model output. Computer Physics Communications, 117(1), 35–43. https://doi.org/10.1016/S0010-4655(98)00154-4
- Sobol, I. M. (1993). Sensitivity estimates for nonlinear mathematical models. Mathematical Modelling and Computational Experiments, 1(4), 407–414.