Monitor production traffic

Log production traffic, catch input drift on a schedule and explain the predictions that look wrong.

Log what the model does in production, then let analyses compare that traffic with the reference population. The pattern is three calls.

Log predictions as they happen with POST /v1/predictions/batch (up to 1,000 per call) or one at a time. Pass your own prediction_id so a complaint can be traced to the exact record, and timestamp for backfills.
Run a drift analysis on a schedule: type = "drift" compares the latest window logged inputs (default 500) with the reference rows, feature by feature, using the population stability index. PSI under 0.10 is stable, under 0.25 moderate, otherwise significant. The insight's severity follows (low, medium, high), and output_drift reports the shift in the predictions themselves.
Explain the outliers: POST /v1/explanations with the logged input and its prediction_id links the explanation to the record. The project overview and GET /v1/organization/overview surface medium and high insights as alerts.

A shadow model is the same pattern with two models: log every request to both and compare their insights before switching traffic. Drift needs at least 20 reference rows and 20 logged predictions; until then the insight says insufficient_data.

# in the serving path, a batch at a time
client.predictions.create_batch(model="credit-risk-v3", items=[
    {"prediction_id": req.id, "input": req.features, "output": {"label": label, "probability": p},
     "latency_ms": ms, "metadata": {"region": "eu-west"}}
    for req, label, p, ms in served
])

# nightly
analysis = client.analyses.wait(client.analyses.create(model="credit-risk-v3", type="drift").id)
drift = client.insights.list(model="credit-risk-v3", type="drift").data[0]
print(drift.severity, drift.summary)
# medium Recent traffic has moderate drift (PSI 0.15). Shifted inputs: `loan_amount`, `debt_to_income`, `income`.
for row in drift.payload["features"][:3]:
    print(row["feature"], row["psi"], row["status"], row["reference_mean"], "->", row["current_mean"])

# a suspicious prediction, explained
pred = client.predictions.retrieve("pred_8f2a19")
client.explanations.create(model=pred.model_id, input=pred.input, prediction_id=pred.id, method="shap")

curl https://api.kaimei.ai/v1/predictions/batch \
  -H "Authorization: Bearer $KAIMEI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "credit-risk-v3", "items": [
        {"prediction_id": "pred_8f2a1a", "input": {"income": 54000, "credit_score": 681, "utilization": 0.41}, "output": 0.63},
        {"prediction_id": "pred_8f2a1b", "input": {"income": 72000, "credit_score": 790, "utilization": 0.08},
         "output": {"label": "approve", "probability": 0.91}}
      ]}'

curl https://api.kaimei.ai/v1/analyses -H "Authorization: Bearer $KAIMEI_API_KEY" \
  -H "Content-Type: application/json" -d '{"model": "credit-risk-v3", "type": "drift"}'

curl "https://api.kaimei.ai/v1/insights?model=credit-risk-v3&type=drift" -H "Authorization: Bearer $KAIMEI_API_KEY"

200 OKthe drift insight

{
  "object": "insight",
  "id": "ins_a0a2b5c4a5ddb552758a",
  "model_id": "mdl_390bd6d9568f626851ef",
  "analysis_id": "anl_b8d673126dcf3cdbe79a",
  "type": "drift",
  "title": "Input drift",
  "summary": "Recent traffic has moderate drift (PSI 0.15). Shifted inputs: `loan_amount`, `debt_to_income`, `income`.",
  "payload": {
    "features": [
      {
        "feature": "loan_amount",
        "psi": 0.148039,
        "status": "moderate",
        "reference_mean": 23589.0,
        "current_mean": 27721.72
      },
      …
    ],
    "overall_psi": 0.148039,
    "status": "moderate",
    "reference_rows": 600,
    "current_rows": 198,
    "drifted_features": [
      "loan_amount", "debt_to_income", "income"
    ],
    "output_drift": {"psi": 0.071375, "status": "stable"}
  },
  "score": 0.148039,
  "severity": "medium",
  "created_at": "2026-10-31T22:40:25Z"
}