How Kaimei runs your model
The three ways Kaimei gets a prediction from your model, and the contract a remote scoring endpoint follows.
The inference object on a model tells Kaimei how to get a prediction. Explanations by any model-querying method, predict and analyses need one of the first three kinds; logging predictions, storing pre-computed attributions and drawing an uploaded architecture never need one.
| inference.type | How predictions happen | Typical use |
|---|---|---|
remote | Kaimei POSTs batches of rows to your HTTPS endpoint and reads the predictions back. Headers you set (for example an API key) are stored and never echoed. | Any model, any framework, weights stay with you. |
builtin | A reference model that ships with Kaimei runs in the API. inference.name picks it. | Trying the platform without deploying anything. See built-in models. |
none + structure | You uploaded the layers and weights (kaimei.structure.v1). Kaimei executes the graph itself. | scikit-learn, Keras and PyTorch models uploaded from a notebook. |
none | Nothing can run the model. Only logging and pre-computed attributions work. | Audit trails for models you cannot expose; architecture visualisation of a summary upload. |
The remote contract. Kaimei sends {"instances": [row, …]}, rows being JSON objects keyed by feature name, and expects {"predictions": [...]} with one entry per row: a probability of the positive class for binary models (or logits when outputs.kind is logits), a probability vector for multiclass models, or a number for regressions. input_key and output_key rename those keys; the response may also use outputs, scores, probabilities or data, or be a bare list. Requests are split into batches of batch_size rows and under max_request_bytes.
Remote inference options
HTTPS endpoint Kaimei calls.
Extra request headers, for example an authorization header. Stored with the model and returned as ••••••; send the redacted value back to keep it on updates.
Per-request timeout, at most 300.
Rows per request, 1 to 4096.
Keys of the request body and the response.
Image models: list (nested pixel lists), png ({"b64": …}, 8-bit, the TF-Serving convention) or float32 ({"b64", "dtype", "shape"}).
hwc (channels last) or chw (PyTorch order). Not with png.{"mean": [...], "std": [...]} applied to [0, 1] pixels before sending, like torchvision's Normalize. Not with png.Upper bound on one request body.
A failing endpoint surfaces as 502 upstream_error with a code such as inference_unreachable, inference_http_error (the first 500 bytes of the body are in details), inference_bad_json or inference_bad_shape.
{
"instances": [
{
"income": 72000,
"credit_score": 681,
"utilization": 0.41,
"debt_to_income": 0.38,
"loan_amount": 18000,
"employment_years": 4,
"num_delinquencies": 1,
"age": 34,
"home_ownership": "rent"
},
{
"income": 54000,
"credit_score": 702,
"utilization": 0.22,
"debt_to_income": 0.31,
"loan_amount": 9000,
"employment_years": 11,
"num_delinquencies": 0,
"age": 41,
"home_ownership": "own"
}
]
}{
"predictions": [0.81, 0.07]
}{
"predictions": [[0.12, 0.83, 0.05], [0.6, 0.3, 0.1]]
}{
"predictions": [3072.6, 2890.1]
}