Documentation

Any repository, one command

You have a GitHub URL (or a local checkout) and want a verified pull request.

hotpath doctor                                   # git, Docker, keys, accelerator
hotpath check https://github.com/you/your-repo   # seconds, read-only: blockers, cautions, cost
hotpath go    https://github.com/you/your-repo   # the nine stages, ending in a draft PR

• Nine stages: setup, fetch, assess, baseline, benchmark, configure, optimize, publish, verify. Each one finishes or stops with the reason and a next step.
• The baseline must be green on 3 of 3 runs; a flaky or failing suite stops the run before anything is measured.
• No benchmark in the repo? Hotpath profiles the test suite, writes one over the hottest functions, and validates it by running it before trusting it.
• Nothing is pushed without your confirmation (or --yes), never to the default branch, and the PR opens as a draft.
• After the PR opens, Hotpath waits for the repository's own CI and only touches failures the PR introduced.

Inside your own repository

You maintain the code and want Hotpath as a repeatable tool with a config you control.

cd my-repo
hotpath init                 # writes .hotpath.yaml + .github/workflows/hotpath-verify.yml
git add .hotpath.yaml .gitignore .github && git commit -m "Set up Hotpath" && git push
hotpath run --pr             # optimize, then push hotpath/<run id> and open a PR
hotpath serve                # the dashboard for this repo's runs
hotpath pr --prune           # later: ship only the changes that pull their weight

• init detects your correctness check and benchmark, locks them, and asks what Hotpath may edit.
• Every command then finds the nearest .hotpath.yaml, so no config path is needed.
• The generated workflow re-runs the locked check on GitHub and fails any hotpath/* PR that touches a locked or non-editable file.
• Publishing refuses if your base branch moved since the run measured it; re-publishing the same run refreshes the same PR.

Offline demo, no API keys

You want to see the whole loop, including every rejection path, before spending anything.

git clone https://github.com/Nijjea1/hotpath && cd hotpath
pip install -e .
hotpath run configs/demo_repo.yaml            # full loop with recorded patches
hotpath serve configs/demo_repo.yaml          # http://127.0.0.1:8765
hotpath ablate configs/demo_repo.yaml         # what each kept change contributed
hotpath export configs/demo_repo.yaml out/    # PR bundle: tree, changes.patch, REPORT.md
hotpath run configs/demo_repo_beam.yaml       # beam search + a retry chain

• The mock provider replaces the model, not the verification: every recorded patch still goes through a real worktree, the locked tests, and the benchmark.
• Several recorded patches are deliberately wrong (a test edit, a behaviour change, a no-op), so you see locked_file, rejected_correctness, rejected_speed and patch_failed for real.

Bring your own benchmark

You already know what 'faster' means for your code and want Hotpath to measure exactly that.

# bench.py — lock this file in .hotpath.yaml
from hotpath.benchlib import run
from mylib import workload

run(lambda: workload(), warmup=2, trials=15, seed=0)
# prints one JSON line: {"hotpath_benchmark": 1, "samples": [...], "metric": "seconds"}

• Any command works as bench_cmd if it prints one JSON line with at least two positive samples.
• Add "higher_is_better": true for throughput metrics such as tokens/sec.
• GPU targets can declare a fixed workload matrix (benchmark.required_workloads); a candidate must keep ≥98% of its parent's throughput on every shape, not just win on aggregate.

Where does the time actually go?

Runs the target under a profiler and summarises the widest bars. Amdahl's law: a function that takes 5% of runtime can only ever save 5%.

What should we try next?

The planner reads the profile, the hottest source, and every past result, and returns structured experiments. It stops proposing what already failed.

Can the idea be written as a patch?

Fast worker models write each idea as a patch in its own isolated worktree, so many experiments run in parallel without touching each other.

Is the output still the same?

Generated tokens must match the frozen reference exactly and logits must stay within tolerance. Tests are locked: a patch that touches them is rejected before a byte is written.

Is it faster than the noise?

Warmup runs, GPU sync, many trials, the median. The speedup must clear the machine-measured noise floor, and its 95% interval must exclude 1.0.

Does it still help on top of the rest?

Accepted patches stack, and each new one is measured against the stack it builds on. The bottleneck moves, so Hotpath re-profiles. An ablation pass re-measures what each shipped change contributes.

Use it when: First thing on a new machine, and whenever a run fails for an environmental reason.

hotpath doctor
hotpath doctor --verify-keys      # ask each endpoint whether the key still works
hotpath doctor --require-gpu      # exit non-zero unless PyTorch sees an accelerator

Use it when: Setting up a repository you own so that run, pr and serve need no config path.

hotpath init
hotpath init --yes --test-cmd "python -m pytest -q" --bench-cmd "python bench.py" --editable "src/*.py"

Use it when: One command from a URL to a verified draft pull request.

hotpath go https://github.com/you/your-repo
hotpath go owner/repo --iterations 4 --beam 2 --max-minutes 30
hotpath go . --provider mock --mock-patches DIR --sandbox local --no-pr   # offline

Use it when: Before spending anything: what would stop a run, and what it would cost. Exit 0 go, 1 caution, 2 stop.

hotpath check https://github.com/you/your-repo
hotpath check . --json

Use it when: A read-only look at what Hotpath detects: ecosystem, tier, test and benchmark commands, editable and locked files.

hotpath assess path/to/repo
hotpath assess . --json

Use it when: One optimization loop from a config you control.

hotpath run                                   # nearest .hotpath.yaml
hotpath run configs/demo_repo.yaml --iterations 5 --beam 2
hotpath run --pr                              # publish if something was proved
hotpath run --resume run_ab12cd34             # continue a stopped run

Use it when: Publishing a finished run as a pull request, one verified commit per change.

hotpath pr
hotpath pr --run-id run_ab12cd34 --prune --draft
hotpath pr --no-push                          # build the branch locally only

Use it when: Reading a run: the experiment tree, the metric chart with its noise band, and every rejection reason.

hotpath serve                                  # http://127.0.0.1:8765
hotpath serve --db path/to/hotpath.db          # read-only

Use it when: Finding out what each accepted change actually contributed.

hotpath ablate
hotpath ablate --prune --json ablation.json

Use it when: A reviewable bundle without touching git remotes: optimized tree, changes.patch, REPORT.md.

hotpath export .hotpath.yaml out/
hotpath export .hotpath.yaml out/ --ablate --prune

Set the key in your shell, or add it to ~/.hotpath/.env. To try Hotpath with no key at all: --provider mock.

The key is revoked or mistyped. hotpath doctor --verify-keys checks every configured key.

Start Docker Desktop (or the daemon) until docker info succeeds. For a repository you trust, --sandbox local runs it on the host.

Docker was not available and nobody could answer the prompt. Start Docker, or pass --sandbox local / --yes for trusted code.

Hotpath refuses to measure against a suite that is not green 3/3. Narrow it with --test-cmd (e.g. deselect network or timing-dependent tests).

Property tests with a deadline fail under load. Deselect them with --test-cmd, or add a conftest profile with deadline=None.

The worker could not write exact search/replace text. Try a stronger worker_model; hotpath go already sizes the source budget to the repository.

That is a result, not an error. A noisy benchmark raises the bar; a narrower benchmark over the hot function (or rebenchmark_parent: true) measures smaller wins.

Add a remote, or use --no-pr / hotpath export for a local bundle.

Rerun with hotpath -v go … for the traceback, add --resume to continue, and please open an issue.

Contributors: run python -m pytest from the interpreter Hotpath is installed into (activate the venv first).