MCP and Service API

KeepGPU ships a local service that powers four local surfaces:

• MCP over stdio (keep-gpu-mcp-server)
• JSON-RPC over HTTP (/rpc)
• REST API (/api/*)
• Dashboard UI (/)

This is the same backend used by keep-gpu start/status/stop/list-gpus. Exact /api and unknown /api/* paths return structured JSON 404 errors instead of dashboard HTML. /api/sessions/{job_id} uses one raw path component; extra raw path segments are unknown endpoints. Noncanonical raw aliases such as //api/sessions and encoded leading-slash aliases such as /%2Fapi/gpus or /%252Fapi/gpus, including deeper repeated encodings, are also unknown endpoints and do not start or stop sessions. Dot-segment aliases such as /./api/gpus and /foo/../api/gpus follow the same structured JSON 404 path. Exact API collection routes such as /api/gpus and /api/sessions do not accept query strings or path parameters unless documented; query-shaped collection URLs return JSON 404 responses. Missing packaged asset URLs, including raw, encoded, or repeatedly encoded /assets/* aliases, dot-segment aliases that normalize into the packaged asset directory, and paths with recognized static-asset extension components, also return JSON 404 responses instead of asset bytes or the dashboard shell.

Start service

Preferred

keep-gpu serve --host 127.0.0.1 --port 8765

MCP executable

keep-gpu-mcp-server --mode http --host 127.0.0.1 --port 8765

HTTP endpoint inputs are validated before socket bind. --host must be a DNS hostname or IPv4 address, and --port must be a plain ASCII decimal integer in 1..65535.

MCP protocol over stdio

MCP clients should start with initialize, then discover KeepGPU actions with tools/list, then invoke an action with tools/call. KeepGPU exposes these tool names:

• start_keep
• stop_keep
• status
• list_gpus

tools/list responses are snapshots. Local embedding callers can inspect or modify the returned tool metadata without mutating the server's shared MCP tool registry.

Use MCP ping for cheap liveness checks instead of polling GPU or session endpoints when you only need to know whether the service is responsive.

Minimal client config:

servers:
  keepgpu:
    command: ["keep-gpu-mcp-server"]
    adapter: stdio

The stdio transport writes only JSON protocol messages to stdout. KeepGPU logs and diagnostics go to stderr so MCP clients can parse stdout safely.

JSON-RPC request messages that include a jsonrpc version must use "2.0"; valid id-less notifications/* messages remain silent and do not produce response envelopes. Id-less notifications with an explicit non-"2.0" jsonrpc version return -32600 Invalid Request with id: null.

HTTP JSON-RPC quick example

HTTP mode is KeepGPU's local JSON-RPC/REST/dashboard service. It accepts the same JSON-RPC message shapes at the exact /rpc endpoint, but it is not a Streamable HTTP MCP endpoint. Noncanonical /rpc URLs, including trailing slashes, query strings, leading double slashes such as //rpc, or encoded aliases such as /rp%63, /%2Frpc, and /%252Frpc, including deeper repeated encodings, return structured 404 Unknown endpoint errors. Dot-segment aliases such as /./rpc and /foo/../rpc are rejected the same way.

Malformed HTTP JSON-RPC bodies, including non-standard constants such as NaN or Infinity, return a JSON-RPC -32700 Parse error envelope with id: null; REST routes keep REST-shaped JSON errors.

curl -X POST http://127.0.0.1:8765/rpc \
  -H "content-type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"status","arguments":{}}}'

Legacy JSON-RPC quick example

Direct method calls remain available for scripts and older local integrations.

curl -X POST http://127.0.0.1:8765/rpc \
  -H "content-type: application/json" \
  -d '{"id":1,"method":"start_keep","params":{"gpu_ids":[0],"vram":"512MB","interval":60,"busy_threshold":20}}'

Methods:

• start_keep(gpu_ids?, vram?, interval?, busy_threshold?, job_id?)
• stop_keep(job_id?)
• status(job_id?)
• list_gpus()

Successful direct-method responses are KeepGPU JSON-RPC envelopes with jsonrpc: "2.0", the matching request id, and an object result.

For direct JSON-RPC calls, public validation failures and unknown parameters return JSON-RPC -32602 Invalid params. Omitted params are treated as an empty object for legacy direct calls; explicit params: null or any other non-object params value is invalid and does not call the target method. Expected startup-unavailable conditions, such as an unsupported controller platform, no usable visible GPUs, failed CUDA/ROCm visible-device enumeration during start_keep or list_gpus, or an unavailable PyTorch MPS backend during start_keep, return -32000 with the startup message. Unexpected server failures use -32603 Internal error.

Explicit gpu_ids that are validly shaped but outside the service process's current visible GPU count are public validation failures. Direct JSON-RPC returns -32602, while MCP tools/call returns a tool result with isError=true. When list_gpus() returns validated visible records, explicit gpu_ids must also match those listed IDs; unlisted IDs fail before controller startup with the same public validation shape.

MCP tools/call responses keep protocol envelopes successful for normal tool results, public tool-input validation failures, and expected hardware/platform startup-unavailable failures, including failed CUDA/ROCm device enumeration during start_keep or list_gpus and unavailable PyTorch MPS backends during start_keep. Those tool-level failures return result.isError=true with the message in tool content. Protocol shape errors, such as unknown tools, still return JSON-RPC errors such as -32602, and unexpected internal controller/runtime failures return JSON-RPC -32603 Internal error.

For supported REST route/method calls, service errors stay parseable. Public validation failures return JSON 400 responses, unknown API routes return JSON 404, expected startup-unavailable session creation returns JSON 503, and unexpected runtime failures return JSON 500 responses with an error object.

REST session creation accepts a JSON object body, not arrays or scalar values. Omitting gpu_ids means all GPUs visible to the service process. Omitting busy_threshold uses the eco-safe default 25. Explicit GPU values are visible device ordinals in that same process environment. Duplicate or out-of-range lists are invalid, an explicit empty gpu_ids list is invalid, and startup is unavailable if no visible devices exist even when gpu_ids is explicit. Startup also returns an error if the resolved selection contains zero devices. interval must be a finite positive number of seconds, including fractional seconds, within the Python runtime wait limit; NaN, infinities, and oversized values are rejected before session creation. vram accepts human-readable sizes or bytes. Byte-equivalent requests below 4 bytes or above 1 PiB are rejected as public validation errors, and internal tensor element counts round up to cover the requested byte-equivalent amount. Dashboard session refreshes preserve both string and integer-byte vram values from valid service status records. Cheap local fields (vram, interval, busy_threshold, job_id, duplicate custom job_id, and gpu_ids shape) are rejected before /api/sessions asks the service to list visible GPUs, so bad requests do not spend telemetry work. When gpu_ids is explicit, /api/sessions validates the full list_gpus() response before deriving the allowed visible IDs. An empty validated GPU listing is 503 startup-unavailable; malformed listing payloads or records are unexpected internal 500 errors and no session is created. CUDA telemetry resolves CUDA_VISIBLE_DEVICES for the service process and treats malformed, duplicate/equivalent, ambiguous, or out-of-range masks as unavailable utilization rather than guessing a physical GPU. Unique CUDA UUID prefixes resolve like CUDA-visible devices, and parsing stops at -1 after any valid preceding tokens.

Custom job_id values are unique across active and starting sessions. If a duplicate arrives while the original start is still creating controller work, the duplicate is rejected before another controller begins keep-alive work. Only null/omitted means "generate an ID" for start_keep or "all sessions" for status and stop_keep. Custom IDs must be non-empty strings containing only letters, digits, ., _, -, or ~, except standalone . or ..; invalid IDs return an error before session state changes. The MCP tool schemas advertise the same job_id pattern, so clients can reject invalid custom IDs before making a tool call. For start_keep, the gpu_ids schema also advertises the runtime list limit of at most 64 unique visible ordinals.

Status calls show reserved jobs as state="starting" while controller startup is still in progress. That includes both status(job_id) and the all-session status() list, so agents do not mistake an in-progress start for no session. Returned session params are snapshots; local embedding callers can inspect or modify the returned object without mutating service state. Known fields in params are always present and follow the same public session contract as start_keep; omitted gpu_ids is represented as null, not by omitting the key. If an already-started worker later reports a terminal runtime or allocation failure, the retained session is refreshed to state="runtime_failed" with last_error. It remains visible and stoppable. This is distinct from normal busy-GPU or unavailable-telemetry backoff, where the controller defers allocation and the session stays active. Runtime-health probes are observational: status does not hold the session lifecycle lock while running them, and a late probe result is ignored if the inspected session has already stopped or been replaced. If a runtime-health probe itself fails, the inspected active session is retained as runtime_failed with last_error instead of making the status request fail.

stop_keep returns additive outcome fields:

{"stopped": ["job-a"], "timed_out": [], "failed": [], "errors": {}}

If a release times out, the session remains visible in status with state="stopping" until the background release finishes. If that background release later succeeds, the session is removed; if it fails, the session remains visible with state="stop_failed" and last_error describing what happened. A job id only appears in stopped after cleanup has completed within the stop request timeout.

The same retention rule applies when startup fails after controller work may have begun and cleanup cannot be proven complete; callers still receive the original startup error.

If stop_keep arrives while a matching session is still starting, the service waits for startup to settle before deciding whether the job exists. This wait is bounded: if startup does not settle within the stop wait budget, the response uses the normal timed_out field, status shows state="stopping" with the timeout message, and the service remembers the cancellation so the session is released in the background if startup later succeeds. Stop-all requests also wait for in-progress starts from their initial snapshot, with the same bounded timeout behavior. For stop-all, starts that begin after that request's initial snapshot are not stopped by that request. Stop-all releases the sessions in its snapshot concurrently and aggregates results in deterministic snapshot order using the same additive fields.

REST quick examples

REST endpoint index

Endpoint	Method	Purpose
/health	GET	Service liveness probe.
/api/gpus	GET	GPU telemetry with start-compatible visible ordinals.
/api/sessions	GET	All tracked keep sessions.
/api/sessions/{job_id}	GET	One tracked keep session.
/api/sessions	POST	Start a keep session from a JSON object body.
/api/sessions	DELETE	Stop all sessions.
/api/sessions/{job_id}	DELETE	Stop one session.
/rpc	POST	Exact JSON-RPC compatibility endpoint.
/	GET	Dashboard UI.

curl http://127.0.0.1:8765/health
curl http://127.0.0.1:8765/api/gpus
curl http://127.0.0.1:8765/api/sessions

/api/gpus returns start-compatible, non-negative, unique visible ordinals as id/visible_id. Optional physical_id or uuid fields describe the underlying vendor device only; clients should not send those metadata values as gpu_ids. If visible-device enumeration is temporarily unavailable, the route returns a structured JSON 503 response instead of a generic internal error. On CUDA, NVML records are exposed only when Torch CUDA can start the same visible ordinal set, so NVML-only devices are omitted instead of advertised as usable session targets. On ROCm, records are emitted only for visible ordinals that Torch can select; nullable memory fields mean memory telemetry is unavailable after successful selection. On ROCm, physical_id is included only when KeepGPU can safely resolve ROCR_VISIBLE_DEVICES and one matching HIP/CUDA overlay to a ROCm SMI index; otherwise utilization is reported as unavailable rather than guessed.

Start and stop by way of REST:

curl -X POST http://127.0.0.1:8765/api/sessions \
  -H "content-type: application/json" \
  -d '{"gpu_ids":[0],"vram":"1GiB","interval":120,"busy_threshold":25}'

curl -X DELETE http://127.0.0.1:8765/api/sessions/<job_id>
curl -X DELETE http://127.0.0.1:8765/api/sessions

Dashboard

Open:

http://127.0.0.1:8765/

The dashboard provides live telemetry, tracked session state, and start/stop controls. Its packaged static assets are self-contained and do not fetch remote fonts, CDN scripts, or other runtime network assets. Stop controls use a longer client timeout than ordinary refresh/start requests so the backend can return structured startup-wait plus release-timeout payloads. Telemetry refresh is manual by default. The Auto refresh toggle enables 10-second polling while the tab is visible and pauses when the tab is hidden. Telemetry cards show the visible ordinal to type into the start form before any physical/vendor metadata. The start form rejects duplicate GPU IDs and selections with more than 64 entries before submitting to REST. CUDA and ROCm devices include memory and utilization when the platform APIs are available. Mac M series devices report best-effort MPS memory counters and use null for unsupported or invalid fields such as unavailable utilization, negative memory counters, or impossible used > total memory pairs. Dashboard summary cards average only finite utilization readings and show n/a when every visible reading is unavailable, so unknown telemetry is not presented as idle. Per-GPU cards also omit the utilization fill for unavailable readings. Session cards may show retained runtime failures when a started worker reaches a terminal allocation/runtime error; those sessions can still be stopped. Valid busy_threshold values are -1 or 0..100, and omitted API values default to 25. When utilization is unavailable and busy_threshold is non-negative, controllers sleep instead of allocating keep tensors or running keepalive compute; busy_threshold=-1 is the explicit unconditional mode. Stop controls show timed-out or failed releases instead of claiming success when the backend keeps a session visible for follow-up cleanup. Retained session cards show Releasing or Release failed with the backend error detail when present. When no stop outcome list is populated, the dashboard preserves the backend message, for example a missing targeted job_id. When REST calls fail, the footer uses the structured backend error.message instead of showing the raw JSON error payload. Malformed successful refresh payloads, such as missing /api/gpus.gpus, a non-array /api/sessions.active_jobs, or malformed records inside those lists, are shown as refresh warnings without clearing the last good telemetry or session list.

Remote and security notes

• Bind to loopback (127.0.0.1) by default.
• For remote access, tunnel over SSH instead of exposing public ports.

ssh -L 8765:localhost:8765 gpu-box.example.com

• If you must expose externally, front with your own auth and reverse proxy.