A Pi extension that measures how much of each task is spent in tools, and flags patterns that look like avoidable work.
Pi 0.83.0+, Node.js 22.19.0+, installed from Git, not npm
Extensions run with your full user permissions. Tooltime inspects prompts, tool inputs, and results in memory, so review the source and ref before installing. What it stores ↓
No prompts, tool inputs, or results are ever persisted.
Remote Export is opt-in, and project config can never enable it.
Coach mode adds guidance text. It never blocks or rewrites a call.
Four steps from install to your first session report.
/reload after installing, then start Pi in a project./tooltime status/tooltime report session/tooltime status.
Pi Tooltime mode: analytics configured mode: analytics mode source: config or session cached traces: 0 storage: ~/.pi/agent/state/pi-tooltime/... config: defaults promotion: 3 tasks and 30000ms suspected remote: off
Tooltime report · session tasks: 3 task wall: 4m 18s tool wall: 2m 6s (49% of task wall) tool call sum: 2m 41s calls: 18 (1 failed) suspected avoidable wall: 28s Top tools: - bash: 1m 28s across 6 calls, 1 failed - read: 42s across 7 calls - grep: 31s across 5 calls Efficiency findings: - repeat.expensive-identical (high): 2 tasks, 20s suspected - search.slow-empty (medium): 1 task, 8.0s suspected
Illustrative sample. A finding is evidence, not proof that work was avoidable.
Parallel tool calls make "total tool time" ambiguous, so every report carries two numbers. Findings are printed as fixed codes.
toolWallMs
The union of all Tool Spans. Parallel calls count once. Use it for user-visible tool occupancy.
toolCallMs
The sum of all Tool Spans. Parallel calls can push this past Task Wall Time. Use it to inspect aggregate tool work.
| Finding code | Trigger | Confidence |
|---|---|---|
repeat.expensive-identical | A repeated nonmutating fingerprint passes the duration threshold, with no mutation between calls. | High |
test.full-suite-loop | A later full-suite call repeats the prior suite or follows a mutation. | High |
search.broad-scope | A search or analysis targets root or outside the project. | High or medium |
search.slow-empty | A search reaches slowCallMs and returns an empty or error Result Kind. | Medium |
route.explicit-web-local-search | Repository search before a web source, for explicit web intent. | High |
route.explicit-database-local-search | Repository search before a database source, for explicit database intent. | High |
route.explicit-repository-database-query | A database query before a repository source, for explicit repository intent. | High |
Only high-confidence findings can become observed guidance, after 3 affected tasks and 30 seconds of suspected time by default. The three route.* codes all map to the single source-routing guidance rule. Exact report and API names are defined in Terminology ↗
Analytics is the default. Coach is the opt-in. Off pauses capture and keeps your existing data.
| Mode | Records traces | Changes system prompt | Use when |
|---|---|---|---|
off | No | No | Pausing capture. Existing traces are kept until they expire or you delete them. |
analytics Default | Yes | No | Measuring without touching the prompt. |
coach Opt-in | Yes | Yes | After findings repeat. Appends at most the configured number of fixed rules, never blocks a call. |
With Coach Mode enabled, Tooltime appends this fixed suffix to the existing Pi system prompt before each task. This initial example applies when no observed guidance qualifies.
Repeated high-confidence findings can replace baseline lines with other fixed registry lines. The default bound is three rules and 500 characters.
/tooltime prompt previews the exact current suffix. Captured prompts, tool inputs, results, paths, fingerprints, and timing metrics never enter it.
Tool economy:
- Match the tool to the source: repository facts → find/grep/read; current public docs → browser_read; live state → database tools.
- Use targeted searches and tests while iterating. Broaden only after evidence, and run full suites near handoff.
- After two empty or repeated calls, stop and change the source, scope, or hypothesis.
/tooltime mode saves a session mode that replaces the
configured fallback and applies from the next task. A valid PI_TOOLTIME_MODE is
authoritative; the command cannot override it. Invalid values warn instead of failing the
agent, and /tooltime status shows the active sources and latest warning.
| Command | Result | Kind |
|---|---|---|
/tooltime or /tooltime status | Show mode, config sources, storage, promotion thresholds, and Remote Export status. | notification |
/tooltime mode off|analytics|coach | Select the mode for the current session. | notification |
/tooltime report [session|project|Nd] | Show measured totals. The default scope is project. | notification |
/tooltime analyze [session|project|Nd] | Send a model-safe aggregate report to the current model as a user message. Pi can persist it, and the provider sees it. | Model-visible |
/tooltime findings | Show finding candidates and active observed guidance. | notification |
/tooltime prompt | Preview the current coach prompt. | notification |
/tooltime reload | Reload Tooltime config and local traces after a JSON edit. | lifecycle |
Nd selects the last N days, such as 7d, from 1 to 3,650. Environment changes need a Pi restart; use /reload after installing or updating the extension.
Everything Tooltime touches falls into one of four boundaries, each with its own rules.
Prompt text, tool names and inputs, up to 4,096 characters of result text, the stop reason, and the working directory are inspected in process memory to classify calls. They are never persisted.
Allowlisted Task Trace fields only: local IDs, timestamps, fixed classifications,
timing, and findings. JSONL under <agent-dir>/state/pi-tooltime/ with
0700 directories, 0600 files, and a 30-day default retention.
Nothing by default. Remote Export is opt-in, project config can never enable it, and Remote Traces omit fingerprints and all raw content.
Coach mode adds fixed rule text only. /tooltime analyze sends an aggregate
report as a user message that Pi can persist and the model provider sees.
pi remove git:github.com/Jeecabs/pi-tooltime@v1.0.0
(add -l for a project-local install). Remove does not delete analytics or
config; the README
has the full deletion steps.
The package root is a Pi-independent Node.js core for eval harnesses. It registers no extension, writes no files, and makes no network requests.
| Export | Target |
|---|---|
pi-tooltime | Pi-independent Core v1 interface |
pi-tooltime/extension | Pi extension default export |
Node.js 22.19.0+, ESM only, installed as a pinned Git dependency. Core v1 remains compatible throughout the 1.x package line.
import { analyzeTraces, createTaskRecorder,
formatAnalysis } from "pi-tooltime";
const task = createTaskRecorder({
sessionId: "eval-run-42",
taskId: "case-7",
cwd: process.cwd(),
prompt: "Inspect the repository",
});
task.record({ type: "start", callId: "call-1",
toolName: "read", input: { path: "src/index.ts" } });
task.record({ type: "finish", callId: "call-1",
result: "file text", isError: false });
const analysis = analyzeTraces([task.finish("completed")]);
console.log(formatAnalysis(analysis,
{ audience: "human", title: "Eval run 42" }));
Global config lives at <agent-dir>/tooltime.json, and a trusted project can add .pi/tooltime.json.
Use it
Every key, range, environment variable, and precedence rule.
Read ↗ TerminologyExact report and API names used across docs and output.
Read ↗Trust it
Transient, local, remote, and model-visible data, field by field.
Read ↗ Remote ExportThe HTTP contract and receiver duties for the opt-in export.
Read ↗ Security policySupported versions and how to report a vulnerability.
Read ↗Build on it