Sign in Start free
Large models

Large models

Upload networks with thousands of layers and hundreds of millions of parameters, and work with them block by block.

Model size is not a constraint on the structure endpoints. A transformer with thousands of layers and hundreds of millions of parameters uploads with the same client.models.upload call.

  • Binary tensors. Weights above externalize_above values are not JSON. The client uploads each as a zlib-compressed little-endian buffer with PUT …/structure/tensors/{key} and references it from the spec as {"$tensor": "<key>"}. Tensors load lazily when a graph needs them; up to 250 million values per tensor, 512 MB compressed per upload.
  • Blocks and repeats. Layers carry a dotted path (encoder.layers.17.attn.q_proj). The structure document groups them into a block tree and tags runs of identical blocks, so the dashboard draws 48 transformer blocks as one ×48 box until opened. GET …/structure?root=encoder.layers.17&depth=2 cuts the scene to one block; detail=summary omits tensor previews.
  • Tiers. scale.tier is small (every tensor ships and the browser replays the forward pass), large or huge; scale.browser_replay says which. For large models POST …/structure/trace with detail: "summary" and nodes: [...] returns per-node statistics instead of narrating every node.
  • Reading weights back. GET …/structure/parameters/{id}?format=f32 streams any window of a tensor as raw float32; client.models.structure_parameter_values wraps it in a NumPy array.
  • Speed. Compiled graphs are cached per structure version and uploaded networks are scored in whole batches, so a 2,000-layer network traces in well under a second.
model = client.models.upload(
    transformer,                       # torch.nn.Module, 1,200 layers / 110M parameters
    name="Encoder",
    feature_names=["token_id"],
    externalize_above=16_384,          # more tensors as binary, smaller JSON spec
    progress=lambda stage, done, total: print(stage, done, "/", total),
)
# tensors 0 / 1188 ... tensors 1188 / 1188, structure 1 / 1

doc = client.models.structure(model.id, detail="summary")
print(doc.scale["tier"], doc.stats["parameters"], len(doc.blocks))

block = client.models.structure(model.id, root="encoder.layers.17", depth=2)
tensors = client.models.tensors(model.id)
print(tensors.total_bytes, tensors.data[0]["key"], tensors.data[0]["shape"])

weights = client.models.structure_parameter_values(model.id, "encoder.layers.17.attn.q_proj.weight", limit=1_000_000)
curl "https://api.kaimei.ai/v1/models/encoder/structure?root=encoder.layers.17&depth=2&detail=summary" \
  -H "Authorization: Bearer $KAIMEI_API_KEY"

curl "https://api.kaimei.ai/v1/models/encoder/structure/parameters/encoder.layers.17.attn.q_proj.weight?format=f32&offset=0&limit=1000000" \
  -H "Authorization: Bearer $KAIMEI_API_KEY" --output q_proj.f32