What it is
When you train models for a living you need to track every experiment: hyperparameters, metrics, artefacts, the model file itself. MLflow (Databricks) is the open default; Weights & Biases is the polished SaaS incumbent; Kubeflow Pipelines, Metaflow (Netflix), and Comet sit in the same niche. The tracking server is the operator’s training history, their model registry, and increasingly the artefact store from which production deployments pull.
What goes wrong
MLflow has no native authentication. The maintainers’ guidance is to deploy behind a reverse proxy. Most operators don’t, and the consequences are richer than they look. CVE-2023-1177 turns the artefact-fetch endpoint into a path-traversal RCE: an attacker creates an experiment with a crafted artefact URI and reads any file the MLflow process can read. We have found instances where this CVE was actively being exploited by other parties: the attacker’s experiment runs were sitting alongside the operator’s, with names like “recon”, “shell”, “check”. The operator’s model files, training data paths, and cloud-credential filenames all leak in the same way.
How we test
We probe /api/2.0/mlflow/experiments/search for the experiment inventory and
/api/2.0/mlflow/runs/search for run-level metadata. We do not invoke the
artefact-fetch endpoint. Operator attribution comes from the experiment names,
the model architecture (visible in the params), and frequently the dataset
URIs which point at named S3 buckets. Where we see signs of third-party
exploitation we escalate the disclosure with the threat-actor evidence
included.