CareersResearch

AI Researcher

Turn “why did the model do that?” into methods that are exact where they can be and honest where they cannot, then ship them to every model on Kaimei.

How to apply The open problems

The role

Kaimei explains and monitors the machine-learning models teams already run: every prediction gets its reasons, every model its anatomy, every deployment a watch. Each of those rests on a method, from Shapley values and LIME to occlusion, Grad-CAM, counterfactual search and the population stability index. Each method rests on a question the field has not settled: is this explanation faithful to what the model computes, or only plausible to the person reading it?

As an AI Researcher you will own that question at Kaimei. You will read widely, design and test methods on public models, and carry the ones that hold up into the API, the Python client and the dashboard. The work is done when a user can call it, and the response says what was computed and where it stops being exact.

Kaimei is a small team, and everyone who works on it writes code and reads the research. You will too.

What you will do

Take an open problem in interpretability from the literature to a method with a stated guarantee, a known cost and a test that would catch it being wrong.
Build the evaluation first: faithfulness and stability checks on public models, run before a method is described anywhere.
Implement methods in production Python behind the REST API and the Python client, inside the API’s time budget.
Write up what you ship: the method, its limits, and a public notebook that reproduces the result.
Work with the engineers who build the API and dashboard, so a result reaches users as an attribution bar, a heatmap or an alert.

What you would work on

Six open problems, each starting from something that ships.

Kaimei’s methods are documented with their limits. These are the limits we would most like to move. Pick one, or bring your own.

Faithful, not just plausible

Ships today

KernelSHAP is exact up to 12 features. Permutation Shapley reports a standard error per feature and LIME its local fit. Image explanations add an evidence check: the class probability with the focus region hidden, and with only that region kept.

shapshapleylimeevidence

Open question

One faithfulness measure that works across methods, tables and images, and is cheap enough to attach to every response.

Explanation methods

Attribution at width

Ships today

Above 96 features, Shapley sampling screens for the features that move the output, in groups when there are tens of thousands. It samples over the strongest 48 and reports the rest as one residual, so the attributions still sum to f(x) − E[f(z)].

shapleyscreeningresidual

Open question

Screening that respects correlated features, with a bound on what the residual can hide.

Wide inputs

From layers to mechanisms

Ships today

Networks of any size upload as structure, a 110-million-parameter transformer included. For one input, the activations endpoint returns each layer’s feature maps, saliency and strongest channels, with class attention per layer. Grad-CAM reads the last spatial feature map.

structureactivationsgradcam

Open question

Which internal features carry a decision, and whether they mean what they appear to mean, stated so a reviewer can check it.

Layer activations

Counterfactuals people can act on

Ships today

A minimal counterfactual search changes at most eight features, three by default, and never touches the ones you freeze, such as age.

counterfactualmax_changesfrozen_features

Open question

Counterfactuals that stay within the data, respect causal links between features and still hold after the model is retrained.

The counterfactual object

When the reasons drift

Ships today

Drift analysis compares the latest logged inputs with the reference rows feature by feature, using the population stability index, and checks the output distribution too.

driftpsioutput_drift

Open question

Catching the moment a model’s reasons change, before its inputs or its accuracy visibly do, with alerts rare enough to trust.

Monitor production traffic

Same accuracy, different reasons

Ships today

The Python client sets up to three models’ explanations side by side. Robustness analysis measures how often decisions flip under small input noise and names the fragile features.

compare_explanationsrobustness

Open question

When equally accurate models disagree about why, which explanation should a user act on, and what does the disagreement say about the data?

Insights

You might be a good fit if you

have done research in interpretability, attribution, causal inference or a neighbouring field, and can point to the work;
know Shapley values well enough to say where they mislead: correlated features, the choice of baseline, perturbations that leave the data;
write Python that other people can maintain, and are at home in PyTorch and NumPy;
would rather show a result on a public model than describe it;
can explain a hard idea in plain language. It is most of what explainability is.

Strong candidates may also have

A PhD, or equivalent research experience, in machine learning, statistics or a related field.
Interpretability work on transformers or vision models: probing, attribution through layers, sparse features or circuits.
Experience with distribution shift, uncertainty estimation or monitoring models in production.
Contributions to open-source interpretability tools.
Experience carrying research into a product used by people outside the lab.

How to apply

Send us three things through the contact form.

01
Your workA CV, or a link that does the job better: a personal site, Google Scholar or GitHub.
02
One thing you madeA paper, a library, a notebook or a model you took apart, and what you would do differently now.
03
One open problemA paragraph on one of the six problems above: where you would start, and how you would know you were wrong.

Apply for this role Other roles

Read the methods, then write to us.

Every method on this page is documented with its limits, and the Titanic example runs end to end in a public notebook. Come with opinions.

Explanation methods Titanic example