Training observability survey, 2026-05-17

NuClide Research, 2026-05-17 Survey #18 in the AI infrastructure series.

Summary

We surveyed self-hosted training-observability platforms: Weights & Biases (self-hosted), ClearML, Aim, Ray Dashboard, MLflow. The aim was to map the population of public-facing experiment trackers and characterize the auth posture per platform class.

DCWF KSAT coverage

Auto-derived from DCWF AI work-role rule files (ksat-tag).

• 672 (AI Test & Evaluation Specialist): K7003, K7004, K7044, S7068, S7075, S7076, T5904
• 733 (AI Risk & Ethics Specialist): K7052, S7056, S7067, T5868, T5893
• overlap (Common AI KSATs (all 5 roles)): K1158, K22, K6311, K6935, K7003, K7041

Three findings carry the survey.

One. Adya AI’s vanijmcp.adya.ai runs a custom FastAPI service called “WandB Service” on port 5005 with a Weights & Biases API key embedded inside it. The schema documents /runs/full with parameters entity, project, run_path, wandb_url, returning “full history (no limit) + logged artifacts metadata”. Any caller who knows the operator’s entity name gets the full WandB workspace through the embedded credential. This is a new exposure class. The full case study is at adya-ai-vanijmcp-2026-05-17.md.

Two. ClearML signup-open at population scale. Of 72 confirmed ClearML hosts in the 286-host corpus, 23 (32%) have basic.enabled=true in the response to POST /api/v2.30/login.supported_modes. Anyone on the internet can register an account against those servers and read the operator’s experiments, datasets, and logged artifacts. The endpoint is reachable without authentication and reveals the auth posture without sending credentials, so this is a fingerprintable property at scale, not a per-host investigation.

Three. Six WandB self-hosted UI hosts. Smaller population than ClearML but each is a single-org workspace with its own experiment history. We did not enumerate beyond the title-bearing landing page.

Tooling shipped

• aimap v1.9.10 ships fingerprints for “Weights & Biases” (the SaaS/self-hosted UI), “WandB Service (custom FastAPI proxy)” (the Adya pattern), “ClearML”, and “Aim”. The Adya custom-proxy fingerprint matches on the FastAPI /openapi.json title plus the service: wandb_service field on the root.
• A follow-up enumerator can be added to ClearML that issues the login.supported_modes POST and classifies basic.enabled automatically, instead of the post-survey one-off probe we did here.

Population overview

ClearML signup-open hosts (23)

Cluster of 1.10.1-359 builds across most of the AWS-hosted set indicates a common deployment template or container image. ClearML 1.10 is two years old; the current open-server stream is 2.x.

Method

• Shodan harvest: html:"Weights & Biases" (28 hits), html:"ClearML" (102), "ray-dashboard" (1,113 with mixed-quality matches), html:"aim-ui" (2), port:43800 http (71). Union: 286 unique IPs.
• aimap port scan: 15 ports per host covering training-obs defaults (5000, 5005, 5001, 8000, 8008, 8080, 8081, 8085, 8090, 8888, 28080, 43800, 11434) plus 80/443.
• aimap fingerprint: new entries for WandB SaaS/self-hosted UI, WandB Service (custom proxy), ClearML, Aim. Already present: MLflow, Ray Dashboard.
• Auth-mode probe for ClearML: POST /api/v2.30/login.supported_modes (unauthenticated, returns server config; no credentials sent, no signup attempted).

Restraint

We did not attempt user-account creation against any ClearML server. The login.supported_modes endpoint is intended to be read by the browser UI to decide which auth modes to render; reading it is consistent with the restraint ethic. We did not query /runs/full against vanijmcp.adya.ai with any real entity or project value. The OpenAPI schema is the finding.

Disclosure routing

The Adya AI WandB-proxy disclosure is drafted and queued at disclosures/IN-adya-ai-vanijmcp-wandb-proxy-2026-05-17.md. Primary recipient: atma@adya.ai (sourced from the JS bundle on adya.ai). CC: adya.omnichannel@gmail.com (WHOIS-registered registrant address) + abuse@microsoft.com (Azure abuse).

ClearML signup-open hosts: 23 separate disclosures pending. Per-IP recipient resolution required; many appear to be AWS / GCP / Azure customer hosts and will need WHOIS lookups.

See also

• adya-ai-vanijmcp-2026-05-17.md
• ../../methodology/insight-04-whois-over-slug-heuristics-for-disclosure-routing.md