Compare Surrogates
The Compare Surrogates widget fits every surrogate model kind on one objective in a single run — Ridge, GP-FITC, GP-VFE, LightGBM, and optionally GP-MOE — and ranks them by cross-validated accuracy. It answers the question that comes before every model-based widget: which surrogate should I trust for this objective? Run it once, read off the winner, then use that model in the Surrogate Optimizer, Response Surface 3D, and Robustness widgets.
For what each model is and when it fits well, see Surrogate Models.
Workflow
- Choose the objective to model and the slice parameter for the overlay plot (below).
- Optionally tick include MoE to add GP-MOE. It is off by default because it is the slowest model; add it only when you specifically want to compare it.
- Press Compare. All models are fit in the background (the same ≥10-trial minimum as the other surrogate widgets applies).
The CV metrics table
Models are listed best-first by cross-validated R²; any model that fails to fit drops to the bottom with its error. The best model is marked with a ✓.
| Column | Meaning |
|---|---|
| CV R² (±std) | Cross-validated R² and its spread — the primary, overfitting-resistant score |
| Holdout R² | R² on a held-out split |
| Holdout RMSE | Root-mean-square error on the held-out split, in the objective's units |
| Train R² | R² on the training data |
Read CV R² first. A high Train R² next to a low CV R² is the signature of overfitting — the model memorizes the trials but does not generalize. Holdout RMSE is the one figure in the objective's own units, useful for judging whether the error is small enough to matter in practice.
The prediction overlay
Below the table, an overlay plots the observed data plus each model's 1D prediction slice along the chosen slice parameter, all anchored at the best observed trial (every other parameter held at that trial's values). Because the anchor is fixed rather than user-selectable, the models are directly comparable: any difference between the lines is a difference between the models, not between anchor points.
- Lines that agree in data-dense regions and diverge where trials are sparse reveal where the models extrapolate differently — exactly where model choice matters most.
- A Ridge slice is always a straight line; visible curvature means the objective is nonlinear along that parameter and a GP or LightGBM model is warranted.
- Observed points need not lie on any line: they generally have different values of the frozen parameters, so they sit off this particular slice.