External Tool Optimization
Tunny Dashboard drives optimization with its built-in Rust samplers and evaluates your command-line tool through a process definition. Python and Optuna are not required by the Dashboard; install only the runtime your evaluation tool needs.
Tool definitions and run setup
Use New Tool… to build a definition, or open a saved definition JSON through Optimize Tool…. The definition specifies parameters, command, input format, objective/constraint extraction, and optional pre/post commands. See the process integration guide for the unchanged definition formats, worked examples, retries, timeouts, and detailed execution caveats.
In Tool Optimization, set variable ranges/precision, ordered objective directions, sampler settings and seed, then the output Journal path and Study name. Choose Random with a trial budget or NSGA-II with population/generations. Press Run to start; the overlay reports progress and offers Cancel. Evaluation failures are recorded as FAIL and do not abort the whole optimization; Journal write errors abort the run.
Execution caveats
- No automatic shell: commands launch directly as OS processes. For Windows
batch files, use Program
cmd.exewith/Cand the script path as arguments. For scripts, explicitly specify the interpreter, such aspythonorsh. - Working dir: leaving it blank uses the Dashboard's current directory, not the definition JSON's directory. Relative Working dir values are also relative to the Dashboard's current directory. Relative input-template and extracted output-file paths use the main command's Working dir. Prefer absolute paths when you need an unambiguous location.
- Output values: extraction must produce finite numbers. NaN, infinity,
missing values, or text containing units fail the trial; use an extractor that
selects only the numeric part. Constraints use
c <= 0for feasibility; violated constraints are still recorded rather than treated as evaluation failures. - Process integration limits: only Random and NSGA-II are available here, and extraction supports objectives and constraints, not per-trial user attributes. For BO methods, see Grasshopper optimization.
Parallel evaluation and file races: trials evaluate in parallel. Fixed shared input/output paths, including Input file template paths, can race between trials. Make the tool's file handling safe for concurrent evaluations; the Dashboard does not automatically isolate trials into separate directories.
Adding new results to an existing Study
Study-name collisions within the same Journal error by default. Enable Add results to existing Study (if present) in Output only when you intend to add results to that Study. This option defaults to OFF. If no Study has that name, a new one is created.
The requested objective count and ordered Minimize/Maximize directions must match the stored Study; otherwise preparation errors before any evaluation. Correct the request rather than expecting stored directions to be adopted silently. Existing objective names and trial records remain unchanged, even if the current definition uses different labels. Trial IDs and Study-local numbering continue.
This is recording reuse, not history-aware optimization resume. Each invocation starts its optimizer fresh. Existing trials are not used for bootstrap, model fitting, populations, or candidate deduplication. Budget, progress and success/fail counts cover only the new invocation, not historical totals.
Serialize writers to the same Journal, including preparation. Append preparation rejects detectable invalid references, malformed/incomplete records and unsupported operations such as Study deletion; it does not repair or renumber the file. Legitimate rejected duplicate Optuna creation attempts remain accepted and do not consume Study IDs. Valid-looking wrong membership cannot always be detected; no filesystem transaction or parallel-writer guarantee is implied.