Sign in Start free
Reference data

Reference data

The population explanations compare an input against and analyses study, and how to give a model one.

Attribution methods need a population to compare an input against: KernelSHAP and permutation Shapley average the model over background rows, LIME perturbs around them, and every model-level analysis studies them. The reference data you attach to a model is that population.

  • Upload rows with PUT /v1/models/{model}/reference-data, or pass X (and y) to client.models.upload. Rows that do not match the schema are skipped and counted in rejected.
  • At most 5,000 rows are kept, the newest first. Image models keep fewer, because the stored set stays under 8 million pixel values.
  • Labels are optional. When present, permutation importance also reports the drop in accuracy or R² when a feature is shuffled.
  • Each explanation draws background_size rows (default 64) from the population; the response says where they came from in options.background_source: reference_data, predictions (recent logged inputs when no reference rows exist) or baselines (one row built from the feature baselines).
  • Drift compares the reference population with recent logged predictions, so for monitoring the reference set should be the data the model was trained or validated on.

A model with no reference rows, no logged predictions and features without baselines cannot be explained by a model-querying method; the API returns 422 no_reference_population. Upload a few hundred representative rows when you register the model.

client.models.upload_reference_data(
    "credit-risk-v3",
    rows=X_valid.to_dict(orient="records"),   # list of {feature: value}
    labels=y_valid.tolist(),                   # optional
    mode="replace",                            # or "append"
)
summary = client.models.reference_data("credit-risk-v3", sample=2)
print(summary.rows, summary.has_labels, summary.columns)
curl -X PUT https://api.kaimei.ai/v1/models/credit-risk-v3/reference-data \
  -H "Authorization: Bearer $KAIMEI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rows": [
      {"income": 72000, "credit_score": 681, "utilization": 0.41, "home_ownership": "rent"},
      {"income": 125000, "credit_score": 760, "utilization": 0.12, "home_ownership": "own"}
    ],
    "labels": ["decline", "approve"],
    "mode": "replace"
  }'
200 OKResponse
{
  "model_id": "mdl_390bd6d9568f626851ef",
  "rows": 600,
  "has_labels": true,
  "sample": [
    {
      "income": 72000,
      "credit_score": 681,
      "utilization": 0.41,
      "home_ownership": "rent"
    },
    …
  ],
  "columns": [
    "income", "credit_score", "utilization", "debt_to_income",
    "loan_amount", "employment_years", "num_delinquencies",
    "age", "home_ownership"
  ],
  "rejected": 0,
  "limit": 5000
}