External Tool Optimization

This section walks through optimizing an external command-line tool from Tunny Dashboard.

Any tool you can run from the command line can be plugged in as an evaluator, not just Grasshopper — a structural analysis solver, a hand-written calculation script, anything. Sampling is handled by the Dashboard's built-in Rust implementation, so you do not need to install Python or Optuna to run an optimization. All you need is the runtime environment of the evaluation tool itself — if the tool you want to evaluate is a Python script, for example, setting up a Python environment for that script is all it takes. Results are recorded in an Optuna-compatible journal file, so live updates and every analysis widget work exactly as they do for any other Study.

Integration with the tool uses a "template substitution → run → output extraction" file-interface approach. Rather than building a dedicated adapter for each tool, you only need to declare the input and output formats, so you can integrate any tool without vendor-specific knowledge.

The overall flow looks like this:

Build a tool definition (process definition) in the definition builder and save it as JSON
Configure variable search ranges, objective directions, and the sampler in the run setup modal
Run the optimization and watch results arrive via live updates

The per-trial evaluation pipeline

Every time the sampler proposes a set of variable values, the Dashboard evaluates the external tool once, following these steps:

Convert the parameter values into the form the tool expects (generating an input file, building environment variables, etc.)
Run the pre-command, if one is configured
Run the main command (with timeout and retries)
Run the post-command, if one is configured
Extract the objective and constraint values from standard output or an output file

The pre-command and post-command are auxiliary commands that run before and after the evaluation. For example, you can prepare input data or convert a solver's raw output into a CSV that is easier to extract from, all without modifying the main tool itself.

How parameters are passed in

You can choose from four ways to pass parameters, matching whatever input convention the tool already has.

Method	What the tool receives
Command-line args	Command-line arguments, expanding a template such as `--{name}={value}` per parameter
Environment variables	Environment variables named after each parameter
JSON on stdin	A JSON object such as `{"length": 12.5}` sent to standard input
Input file template	An input file with `{name}` placeholders substituted, written out to a specified path

With Input file template, a placeholder such as {length} in the template is replaced with that trial's value. To write a literal curly brace into the input file, escape it as {{ and }}.

Integer values are written out as 3, not 3.0, so they can be passed straight through to a tool that parses them as integers.

Extracting output

For each objective and constraint, you specify where the value comes from (standard output or an output file) and how to extract it.

Extraction method	Specification
Regex	Reads the first capture group of a regular expression (or the whole match if there is no group) as a number
JSON path	Walks a JSON document via a dot-separated path such as `results.weight`; arrays are indexed like `values.0`
CSV	Specifies a single cell by row (index or last row) and column (index or header name)

The extracted value must be a finite number. If the output is not a number, or is NaN or infinite, the trial is recorded as a failure (FAIL) rather than quietly succeeding with the wrong value. This is by design, so that a diverged solver's result does not contaminate the optimization.

Also, a string with a unit attached, such as 12.5 kg, is not partially parsed and results in an extraction error. Write a regular expression that captures only the numeric portion in its capture group.

Creating a tool definition

Clicking New Tool… on the toolbar opens the definition builder (the Tool Definition modal). The builder is a form that maps one-to-one to the definition JSON, and you fill in the following sections from top to bottom:

Parameters: names of the optimization variables, referenced as {name} from templates and arguments
Input: how parameters are passed in (the four methods above) and its settings
Command: the program to run, fixed arguments, working directory, timeout, and retry count
Objectives / Constraints: name, where the value comes from, and how it is extracted
Hooks (optional): enabling the pre-command and post-command

Save to File… at the bottom saves the definition to a JSON file, and Load… loads an existing JSON file for editing. Pressing Optimize → goes straight to the run setup without saving.

Because the definition is plain JSON like the following, you can also share it as a file with your team or edit it directly in a text editor.

{
  "param_names": ["length", "thickness"],
  "input": { "kind": "args", "arg_template": "--{name}={value}" },
  "command": {
    "program": "python3",
    "args": ["solve.py"],
    "timeout_secs": 60,
    "retries": 1
  },
  "objectives": [
    {
      "name": "mass",
      "source": { "kind": "stdout" },
      "extractor": { "kind": "regex", "pattern": "mass\\s*=\\s*([0-9.eE+-]+)" }
    }
  ],
  "constraints": [
    {
      "name": "stress",
      "source": { "kind": "file", "path": "out.json" },
      "extractor": { "kind": "json_path", "path": "results.stress_ratio" }
    }
  ]
}

Fixed args are not template-expanded and are passed through as-is. So you can pass arguments containing curly braces, such as an awk or shell one-liner, without them being substituted by mistake.

Run setup and execution

Opening a saved definition JSON, or pressing Optimize → in the builder, opens the run setup modal (Tool Optimization). The command is shown read-only, and you configure:

Variables: the search range (Low / High), decimal precision (Digits), and integer flag (Integer) for each parameter. Ranges start out as [0, 1] for editing
Objectives: choose Minimize / Maximize for each objective
Sampler: NSGA-II (Population, default 16 / Generations, default 10) or Random (Trials, default 50), plus the random seed (default 42)
Output: where to save the journal file and the Study name

Any row where Low is greater than or equal to High is highlighted in red, and the Run button stays disabled until it is fixed. Constraints are part of the definition and cannot be edited here; a trial is considered feasible when every constraint value is 0 or less. Trials that violate a constraint are still recorded, and the amount of violation steers NSGA-II's search toward the feasible region.

Pressing Run creates the Study in the journal immediately, and you can follow its progress with the progress overlay in the bottom-right corner and live updates. Trials are evaluated in parallel, and a command failure or timeout only causes that trial to be recorded as FAIL — the optimization as a whole continues. When the run finishes, the overlay shows the number of successful and failed trials.

A worked example

Here is an example that minimizes the two-variable function $f(x, y) = (x - 3)^2 + y$ using a tiny tool that just prints the result to standard output. The tool can be written in any language, but this example uses awk.

{
  "param_names": ["x", "y"],
  "input": { "kind": "args", "arg_template": "{value}" },
  "command": {
    "program": "sh",
    "args": [
      "-c",
      "awk \"BEGIN{print \\\"f=\\\" (($1-3)*($1-3) + $2)}\"",
      "sh"
    ],
    "timeout_secs": 10,
    "retries": 1
  },
  "objectives": [
    {
      "name": "f",
      "source": { "kind": "stdout" },
      "extractor": { "kind": "regex", "pattern": "f=([-0-9.]+)" }
    }
  ]
}

Set $x \in [0, 6]$ and $y \in [0, 5]$ to be minimized in the run setup and press Run: the Dashboard's sampled points are evaluated by the command, and results are written out as a regular Optuna Study. Running the optimization and analyzing it are both done without Python or Optuna as the optimization engine.

Caveats

No shell is involved: the command is launched directly as an OS process. For Windows batch files, set Program to cmd and pass /C and the script path as arguments; for shell scripts, specify the interpreter explicitly
Working directory baseline: leaving Working dir blank uses the Dashboard's current directory as the baseline. This also affects how relative paths in output files are resolved, so specifying an absolute path is more reliable
Parallel evaluation and file conflicts: trials are evaluated in parallel. A tool that reads and writes a fixed path, as with Input file template, may have multiple trials contending for the same file, so make sure it is safe to run in parallel
Adaptive (surrogate) sampler is not supported: only NSGA-II and Random can be selected currently. Adaptive sampling is supported first for Grasshopper (.ghx) optimization
Only objectives and constraints can be extracted: extracting per-trial user attributes is not supported yet

Summary

This section covered optimizing an external tool through the file interface.

Integrates any tool: declaring template substitution → run → output extraction is enough, with no dedicated adapter needed
The definition is shareable JSON: build it in the GUI definition builder and reuse it as a file across your team
No runtime dependencies: no Python or Optuna needed as the optimization engine — only the runtime environment of your evaluation tool
Resilient to failure: non-numeric output and timeouts are recorded as FAIL, and the optimization does not stop

In-house analysis tools and calculation scripts are often already built to run from the command line. The biggest value of this approach is that you can bring such a tool into the optimization loop almost as-is. Start with a small definition like the one in this section's worked example, and try it with a tool of your own.