VisorBishop

VisorBishop is a single-binary Go fingerprinter for 12 AI/LLM observability and gateway platforms. Built from the population-scale 2026-05 cross-survey work: when you discover that Phoenix is 25% unauth at population scale and LiteLLM is 10.4%, you need a probe that confirms the platform AND its auth posture in one pass without relying on the operator-set HTTP title alone.

What it does

• 12 platform probes. Phoenix (Arize AI), Langfuse, Helicone, LangSmith, Lunary, OpenLIT, Pezzo, Opik (Comet ML), AgentOps, Argilla, Promptfoo, LiteLLM Proxy. Each probe verifies the platform via a marker-specific endpoint, NOT title-string alone.
• IP-direct-shadow probe. 26-port concurrent sweep of common AI-stack debug ports (Redis, Memcached, NATS, ClickHouse, MailHog, Prometheus, etc.) per host. Surfaces the “hostname-routed SSO does not protect IP-direct shadow” pattern.
• Auth-state classification. Every confirmed finding gets a posture call: open (unauth), protected (auth-fronted), or unknown.
• Cross-platform attribution. When one operator hosts multiple platforms (e.g. Phoenix + Qdrant on the same IP), VisorBishop surfaces the correlation in a single output row.

Why this exists

Phase 1+2 of the 2026-05 cross-survey used per-platform Python probes, which created N parallel toolchains for N platforms and let fingerprint quality drift between them. VisorBishop unifies the fingerprint definition in one place. When we discovered that LangSmith and ZenML both serve /api/v1/info, we tightened the LangSmith probe once in VisorBishop and the population-scale recount produced correct numbers.

How we use it

# Probe a single target
visorbishop -t https://app.langfuse.example.com

# Sweep a corpus
visorbishop -i targets.txt -c 32 -timeout 4s \
  -json out.json -csv out.csv

# Include IP-direct-shadow port sweep on every confirmed host
visorbishop -i targets.txt -ip-shadow -json out.json

The 6-iteration Phase 3 loop below validates VisorBishop against the cumulative population data. Each iteration refined either the prober coverage or the methodology:

• iter-1. Extended IP-direct-shadow port set (Redis, MailHog, node_exporter; +8 unauth surfaces)
• iter-2. MinIO + ClickHouse object/datastore tier (32 hits)
• iter-3. AI-stack pipeline ports (3 unauth Qdrant + Rogers Communications NetOps double-exposure)
• iter-4. Adjacent observability platforms Opik + AgentOps + Phospho
• iter-5. LiteLLM gateway + Argilla annotation + Promptfoo eval (35 new unauth instances; introduced the LLMjacking class to the tool)
• iter-6. Full LiteLLM 5,391-host population sweep: 283 confirmed unauth LLMjacking primitives globally

The cumulative live findings are visible on the VisorBishop dashboard.

The 12 platforms, with detection markers