Developer Tool · Interactive
Routing Playground
Simulate how MeshInfer routes inference across local, mesh, and cloud paths. Includes speculative cloud dispatch with AbortController and synchronous privacy gating.
Illustrative simulation — not measured benchmarks
The throughput and latency figures below are hardcoded simulation parameters designed to demonstrate routing logic — how the router chooses between local, mesh, and cloud under different policies. They are not performance measurements and will not match real-world results. Measured data — p50/p95 latency by route, route mix by device class, and cost per 1k requests — lives on the Benchmarks page. The two pages use different methodologies and should not be compared numerically.
Simulation Parameters
256 tok
165122048
Speculative Dispatch
t=0ms · Privacy gate
PASS — speculative allowed
t=0ms · Cloud fired
AbortController created · stream open · TTFT=65ms
t=77ms · Gate-check
warm=true → abort() · cloud cancelled · local TTFT=12ms
Speculative cloud fired at t=0 · gate-check returned warm after 77ms · AbortController.abort() called · cloud cancelled · switched to local · TTFT=12ms
Route Decision
Phi-3.5-mini (Q4) · 256 output tokens
Simulated
Chosen
Local
Total latency9866ms
Throughput26 tok/s
Generation9846ms
Mesh
Total latency9748ms
Throughput26 tok/s
Generation9697ms
Cloud
Total latency4009ms
Throughput65 tok/s
Generation3938ms
Chosen path:
local(9866ms · 26 tok/s)
Latency Comparison
Latency Breakdown — local path
Where time is spent on the chosen route.
Token generation
9846ms100%
Overhead / queue
12ms0%
Total TTFT9866ms
Speculative dispatch
Cloud stream fires at t=0 concurrently with the gate-check. If the gate-check returns warm, AbortController.abort() cancels the cloud stream before meaningful tokens are billed. Privacy policy is checked synchronously before any I/O — local-only policy structurally prevents speculative dispatch.