Project

Editor responsiveness

Reproduce full-document LSP latency measurements for large bodies, recovery, imports, and projects.

On this page

The LSP checks synchronously after each change and shares the resulting symbol index between open files in a checked package. These measurements cover checking, diagnostic delivery, and editor requests, and guide decisions about caching.

Reproduce the measurements

Configure LLVM as described in Building from source, then build and run from the repository root:

cargo build --locked --release
python3 scripts/bench_lsp.py target/release/dodo --features --samples 100 \
  --workload functions large-function errors generic-imports project \
  --compiler-revision "$(git rev-parse HEAD)" --build-profile release \
  --report target/lsp-performance/suite.json

--size selects one or more sizes; --functions remains an alias. Without it, each workload uses the sizes below. --documents defaults to one open buffer, or five for project. --samples defaults to 30.

Workload Size means Default sizes Fixture
functions Small functions 20, 200, 1,000 Typed parameter, arithmetic return, and a caller
large-function Local declaration/assignment pairs in one body 200, 1,000, 2,000 One function with twice that many statements, plus initialization and return
errors Intended diagnostics 20, 100, 200 Alternating syntax errors and unknown bindings in independent functions, plus a valid helper and caller
generic-imports Distinct nominal element types 10, 50, 100 Record types instantiate fixed_vector.Vector and its methods; also imports collections, shared_vector, and shared_hash_map and their transitive dependencies
project Functions per file 20, 200 Required dodo.toml, explicit named build target, 20 sibling files on disk, five open overlays, and package checking

The project manifest has two targets without a default, so successful initialization requires selecting the named bench target. The manifest supplies compiler configuration; --check-mode package selects sibling checking. A manifest alone does not add siblings to the import graph. --project-files controls the total number of project files, including closed files on disk, and must be at least --documents. Other workloads use unsaved file-mode buffers.

Useful controls for repeated work:

# Repeat the original comment-only fixtures, including std/math.
python3 scripts/bench_lsp.py target/release/dodo --features --samples 100 --edit comment
python3 scripts/bench_lsp.py target/release/dodo --features --samples 100 \
  --size 20 --stdlib --edit comment
# A larger individual-body stress case uses 30 timed samples.
python3 scripts/bench_lsp.py target/release/dodo --features --samples 30 \
  --workload large-function --size 5000
# Same statement count while reusing two locals.
python3 scripts/bench_lsp.py target/release/dodo --features --samples 100 \
  --workload large-function --size 5000 --body-shape assignments
# Edit one of many independent open buffers.
python3 scripts/bench_lsp.py target/release/dodo --features --samples 100 \
  --size 200 --documents 5  --report target/lsp-performance/five-files.json
python3 scripts/bench_lsp.py target/release/dodo --features --samples 100 \
  --size 200 --documents 20 --report target/lsp-performance/twenty-files.json
# Same manifest workspace, different checking scope or overlay count.
python3 scripts/bench_lsp.py target/release/dodo --features --samples 100 \
  --workload project --size 200 --project-files 20 --documents 5 --check-mode file
python3 scripts/bench_lsp.py target/release/dodo --features --samples 100 \
  --workload project --size 200 --project-files 20 --documents 1
python3 scripts/bench_lsp.py target/release/dodo --features --samples 100 \
  --workload project --size 200 --project-files 20 --documents 20
# Refresh selected target settings and recheck unchanged buffers.
python3 scripts/bench_lsp.py target/release/dodo --features --samples 100 \
  --workload project --size 200 --edit manifest

The script starts a real dodo lsp process. After opening the fixture, it warms up with five changes, then measures full-text body edits alternating a return constant between zero and one. --edit comment appends a changing comment to the original source instead. Only the first buffer changes; other open overlays and on-disk dependencies stay fixed. Requests run after every change, including warmup. Initialization and initial document opens are outside the measured samples. --body-shape assignments reuses two locals instead of declaring a new local for every pair, isolating growing checker state from statement count. For project, --edit manifest alternates the selected target between 32-bit WebAssembly and 64-bit Linux, writes the manifest, and sends a watched-file change. Its manifest_to_diagnostics interval includes that disk write and configuration resolution; buffer versions stay fixed. These arithmetic-only projects are valid for both targets. Ordinary body edits reuse the settings selected at initialization.

Each change ends with an unknown-method request barrier. Because the server processes messages synchronously, its MethodNotFound response follows diagnostic publication and completion of analysis for all open roots. The measured interval includes JSON serialization, pipe transport, checking, indexing, and diagnostic delivery plus that barrier round trip. It ends before benchmark validation.

Validation requires current diagnostic versions for every open buffer, the exact intended error counts, locations, and message fragments, and no errors in valid fixtures or imported files. Hover, completion, definition, and signature-help responses must resolve the valid helper at positions derived from the source. This avoids timing an empty result after recovery fails. Run without --features for older servers that only support hover. The harness’s smoke test covers all workloads in both edit modes; it skips real-server checks if the binary is absent. It also covers the assignment-only body and manifest refresh.

JSON lines contain p50, nearest-rank p95, and maximum milliseconds. --report also saves toolchain, platform, command, source revision, binary and benchmark hashes, and fixture dimensions after each completed run. --compiler-revision and --build-profile record build provenance supplied by the caller; the checkout revision and binary hash are recorded independently. Compare identical hardware, build profile, fixtures, and samples; these are observations, not CI latency thresholds.

Measurements

Measured on 2026-10-03, macOS 27.0.1 arm64, Apple M5 (10 cores), 16 GiB RAM, Rust/Cargo 1.99.0, and LLVM 23.1.2. Both binaries used Cargo’s ordinary release profile, locked dependencies, and the default LLVM feature. Builds and tests finished before the sequential measurements. Python 3.9.6 ran the harness.

The matched comparison is 326003d545364e209faf2515702f2ef2682e9bb4 (Dodo 0.1.4, before the latest flow-analysis refinements) against 388f463f6c96adcb3f8f72bd274c121b21dc10d7. The older build already includes the earlier flow checker. Compiler sources were unchanged during the runs; the benchmark and documentation were working-tree changes. The September 14 Linux/Core Ultra 5 measurements are historical and are not a same-machine baseline for this series.

The published JSON report contains every run, command, binary/benchmark hash, fixture byte count, and all four request percentiles and maxima. Repository paths are normalized for publication. Both binaries passed the harness’s correctness smoke tests.

Unless indicated below, each fixture has five warmups and 100 timed changes without inter-sample delays. All values below are milliseconds. Each row uses one fresh server process; these percentiles do not measure variation across many server launches. Sizes exclude the additional valid caller.

Body-edit fixture 0.1.4 p50 / p95 Current p50 / p95 Current completion p95
20 small functions 0.20 / 0.22 0.21 / 0.24 0.07
200 small functions 1.90 / 2.03 2.10 / 2.25 0.54
1,000 small functions 23.27 / 23.90 26.69 / 30.93 3.05
One body, 200 local pairs 4.78 / 4.90 5.47 / 5.73 0.06
One body, 1,000 local pairs 104.30 / 115.24 112.92 / 119.46 0.11
One body, 2,000 local pairs 443.46 / 481.95 463.32 / 488.26 0.15
20 mixed syntax/semantic errors 0.19 / 0.21 0.20 / 0.23 0.04
100 mixed syntax/semantic errors 0.72 / 0.77 0.73 / 0.75 0.06
200 mixed syntax/semantic errors 1.61 / 1.68 1.62 / 1.67 0.08
Imports, 10 distinct container types 9.05 / 9.39 9.50 / 9.75 0.07
Imports, 50 distinct container types 40.27 / 42.62 41.25 / 43.59 0.12
Imports, 100 distinct container types 103.16 / 110.71 113.27 / 121.75 0.17
Manifest package: 20 files × 20 functions, 5 open 5.76 / 5.98 5.92 / 6.17 0.26
Manifest package: 20 files × 200 functions, 5 open 266.21 / 381.00 274.59 / 275.60 2.35

The other controls isolate open-buffer count, checking scope, edit kind, and local-state size. File mode checks only the open roots here; package mode checks all 20 project files, including closed siblings. The 5,000-local stress and 2,000-local comment controls use 30 timed changes; all other controls use 100.

Control 0.1.4 p50 / p95 Current p50 / p95 Current completion p95
5 independent open files, 200 functions each 10.37 / 12.95 9.81 / 10.00 0.49
20 independent open files, 200 functions each 38.66 / 40.39 38.78 / 39.61 0.52
Manifest, 20 files × 200 functions, 5 open, file mode (5 checked) 9.86 / 10.66 10.11 / 10.57 0.54
Manifest, 20 files × 200 functions, 20 open, file mode (20 checked) 39.81 / 45.75 39.96 / 40.71 0.51
Manifest, 20 files × 200 functions, 1 open, package mode 201.00 / 351.69 209.35 / 210.89 2.30
Manifest, 20 files × 200 functions, 20 open, package mode 576.28 / 835.48 513.89 / 1309.45 7.48
Manifest target change, 20 files × 200 functions, 5 open 242.13 / 249.15 649.93 / 908.44 8.78
One body, 5,000 local pairs (30 samples) 2694.92 / 2921.84 3049.68 / 3153.57 0.31
One body, 5,000 assignment pairs reusing two locals 273.14 / 308.04 273.32 / 294.12 0.27
Comment edit, one body with 2,000 local pairs (30 samples) 485.74 / 752.81 468.20 / 510.15 0.15
Comment edit, imports with 100 container types 108.04 / 127.35 107.33 / 114.17 0.17

The original small-function comment fixtures now have current p95 values of 0.22, 2.07, and 26.14 ms for 20, 200, and 1,000 functions; 20 functions plus std/math reach 2.77 ms. They still represent only a small part of the workload range. The errors fixture verifies all intended errors while retaining navigation to the valid helper; its 200-error p95 remains 1.67 ms.

Hover, definition, and signature help are generally below 0.2 ms after analysis; the initial wide-package and manifest-refresh runs have tails above that. Requests arriving while the synchronous server is checking wait for that check to finish. The post-barrier request measurements describe querying already refreshed analysis.

Repeated wide-project runs

The first current 20-overlay package run has a 513.89 ms median, 1,309.45 ms p95, and 1,416.64 ms maximum. Its target-refresh control also has a long tail. The host had substantial swap allocated during the series, and a later query reported 44% memory free; neither observation establishes the cause of the tails. Repeats after both main series are reported alongside the first runs:

Repeat (100 samples) Change p50 / p95 Change max Completion p95
Current, 20 files / 20 open, package mode 520.69 / 542.03 600.25 2.58
0.1.4, 20 files / 20 open, package mode 482.60 / 531.79 598.84 2.53
Current, manifest target change, 5 open 274.79 / 303.69 394.62 2.67

The repeated current target-refresh median (274.79 ms) is close to the ordinary five-overlay body-edit median (274.59 ms). Configuration parsing is not shown to dominate refresh, and ordinary buffer edits already reuse the selected settings.

The median scaling across bodies and package sizes is sufficient to identify expensive work. Individual tail differences between these sequential runs should not be attributed solely to the recent compiler changes.

Caching decision

A cache of unchanged independent root analyses is worth prototyping. Editing one of twenty independent buffers repeats about 39 ms of work, compared with about 2 ms for one buffer. Only one root changes in this fixture. The server already retains each document’s analysis, so a prototype can reuse those results and diagnostics when the root and all its dependencies remain valid. The roughly 37 ms difference is an opportunity estimate, not a demonstrated cache speedup.

Reduce work inside the changed root before building a general incremental checker. The 5,000-local body reaches a 3.15-second p95; the same statement count with two reused locals is 294 ms. Checker::block clones recovery state before every statement, including successful statements, and HoverIndex::name_span scans the document’s token stream. The latter also runs while visiting every function for each open document in a checked package. These code paths are specific candidates for smaller recovery checkpoints, bounded token lookups, and skipping declarations outside the document. Caching unchanged imports or independent files cannot eliminate the work in the edited large function.

Investigate reuse within import graphs and packages after those fixes. The 100-type import case reaches 122 ms p95, and checking a 4,000-function package reaches hundreds of milliseconds even with one open overlay. Comment edits remain similarly expensive. This justifies profiling reusable parsing, specialization, body checking, and indexing separately. The round trips alone do not establish how much an imported-parse-tree cache would save, and a whole-root cache misses when any member of the checked package changes.

Any prototype must invalidate for imported buffer edits, disk dependency and package-membership changes, and manifest target/pointer-width changes. Re-run these fixtures and correctness guards against the prototype, including recovery and manifest refresh, before retaining it. Full synchronous checking remains the current implementation; the broader fixtures identify costs that the original small-function measurements could not expose.

Search every guide, reference page, and API declaration.