Deepmark

ThreadTuner

class vigil::ThreadTuner

Benchmarks different intra_op_threads / inter_op_threads combinations for both the embedder and detector, and recommends the fastest configuration for your hardware. The recommended values can then be applied to EmbedderConfig and DetectorConfig.

TunerConfig

struct vigil::TunerConfig

Configuration structure for ThreadTuner.

Fields:

  • embedder_model_path — std::string (default: "")

    Path to the embedder model file used for benchmarking.

  • detector_model_path — std::string (default: "")

    Path to the detector model file used for benchmarking.

  • max_trials — int (default: 100)

    Maximum number of thread combinations to test. The tuner searches across different intra_op_threads x inter_op_threads combinations up to this limit.

  • num_inferences — int (default: 10)

    Number of inference runs per trial. More runs produce more stable latency measurements but increase total benchmarking time.

ThreadTuner constructor

vigil::ThreadTuner::ThreadTuner(const TunerConfig& config)

Constructs a thread tuner instance.

Parameters:
  • config — const TunerConfig&

    Tuner configuration specifying model paths and benchmark parameters. See TunerConfig for details.

ThreadTuner.set_progress_callback

void vigil::ThreadTuner::set_progress_callback(TunerProgressCallback cb)

Registers a callback function that will be invoked to report progress during the tuning run. Useful for displaying progress bars or status updates in your application.

Parameters:
  • cb — TunerProgressCallback

    Callback function to invoke with progress updates.

ThreadTuner.Run

vigil::Status vigil::ThreadTuner::Run(TuneResult& result)

Executes the full benchmark suite, testing each planned thread configuration and measuring inference latency. After completion, the result contains latency statistics for each trial and the recommended thread configuration.

Parameters:
  • result — TuneResult&

    Output structure that will contain benchmark results and the recommended configuration.

Returns:

Status — StatusCode::kOk on success. May fail if models cannot be loaded.

Example:
vigil::TunerConfig tcfg;
tcfg.embedder_model_path = "models/embedder.enc";
tcfg.detector_model_path = "models/detector.enc";

vigil::ThreadTuner tuner(tcfg);
vigil::TuneResult result;
tuner.Run(result);

std::cout << "Recommended: intra=" << result.recommended_intra_op_threads
          << " inter=" << result.recommended_inter_op_threads << "\n";

// Apply recommended settings to your configs
vigil::EmbedderConfig ecfg;
ecfg.model_path = "models/embedder.enc";
ecfg.intra_op_threads = result.recommended_intra_op_threads;
ecfg.inter_op_threads = result.recommended_inter_op_threads;

vigil::DetectorConfig dcfg;
dcfg.model_path = "models/detector.enc";
dcfg.intra_op_threads = result.recommended_intra_op_threads;
dcfg.inter_op_threads = result.recommended_inter_op_threads;

TuneResult

struct vigil::TuneResult

Result structure returned by ThreadTuner.Run, containing benchmark results and the recommended configuration.

Fields:

  • recommended_intra_op_threads — int

    The intra_op_threads value from the trial with the lowest overall median latency.

  • recommended_inter_op_threads — int

    The inter_op_threads value from the trial with the lowest overall median latency.

  • trials — std::vector<TrialResult>

    Full list of benchmark trials. Each TrialResult contains intra_op_threads, inter_op_threads, and LatencyStats with mean, median, p95, p99 in milliseconds for embed, detect, and overall.