Build the Fugu pattern in the open: fan out, assign roles, verify

The Stack

Steps

1. 1

   Author the orchestration config and validate it

   Write orchestrate.yaml: a panel of models with roles (thinker/worker/verifier) and an arbiter strategy, plus a verifier step. CI parses it and asserts the panel, the roles or arbiter strategy, and a verifier are present. Running the panel needs keys and is fenced. See also the panel+judge and tests_pass recipes.

   cat > orchestrate.yaml <<'YAML'
   version: 1
   orchestrator:
     panel:
       - { model: "anthropic/claude-opus-4.8", role: thinker }
       - { model: "openai/gpt-5.5", role: worker }
       - { model: "google/gemini-3.1-pro", role: verifier }
     arbiter:
       strategy: tests_pass
     verify:
       required: true
   YAML
   ruby -ryaml -e '
   c = YAML.load_file("orchestrate.yaml")
   abort "BAD: version must be 1" unless c["version"] == 1
   o = c["orchestrator"] || {}
   panel = o["panel"]
   abort "BAD: panel needs at least 2 models" unless panel.is_a?(Array) && panel.length >= 2
   roles = panel.map { |p| p["role"] }.compact
   allowed = ["best_of_n", "synthesize", "majority", "first", "tests_pass"]
   arb = o["arbiter"] || {}
   has_strategy = allowed.include?(arb["strategy"])
   abort "BAD: set roles or an arbiter strategy" unless roles.include?("verifier") || has_strategy
   verify = o["verify"] || {}
   has_verifier = verify["required"] == true || roles.include?("verifier")
   abort "BAD: a verifier step must exist" unless has_verifier
   puts "config OK: orchestrator fans out to a " + panel.length.to_s + "-model panel, arbiter strategy " + arb["strategy"].to_s + ", verifier step present"
   '

2. 2

   Run it on your keys (the model step, not checked by CI)

   Point the config at your router (OrcaRouter or similar) and run it with your provider keys. You see every hop: which model took which role, what the verifier rejected, and the real per-call cost. The multi-model run is fenced.