skip to content

Real cluster. Real fault. Real engineer. Find out.

The mechanics in detail: how the environments are built, what the evaluator watches, how an interview is graded, and what we deliberately refuse to build.

50+
Kubernetes labs
<1 min
Environment boot
5
Scored criteria
0
Cameras, ever

01

A real cluster, not a simulation

Every lab is a live single-node Kubernetes cluster with a genuine fault injected into it, booted in under a minute. The candidate gets VS Code and a terminal in a desktop browser with cluster-admin on their own cluster: they can break it further, and some of them do. Every interview lab has a 25 to 45 minute time box. Engineers practice on a separate set of labs: same format and same grader, different labs. Most code sandbox vendors cannot provision this.
Interview labsFilter by difficulty
AllBeginnerIntermediateAdvanced
  • Firmware will not start after a config change
    Intermediate· 25 min
  • Auth is down and the on-call runbook did not help
    Intermediate· 25 min
  • The rollout stopped at 2 of 5 and the deadline has passed
    Advanced· 35 min
  • Five services are down and only one of them is broken
    Advanced· 45 min
  • Restore the service and leave it costing 30% less
    Advanced· 45 min

02

Grader scripts decide. The model explains.

This is the part worth understanding, because it is the opposite of how most AI grading works. When a candidate submits, scripts run outside the sandbox and check the cluster: did the symptom clear, was the actual root cause fixed, and does the fix hold. That is the verdict, Solved or Not solved, and a model cannot overrule it. Only then does an AI layer score Diagnosis, Blast radius, Verification and Root-cause explanation, and every score it returns must quote a real command at a real timestamp. A quote that does not resolve to something that happened is retried once, then withheld rather than published. How grading works →
Rubric · senior screen
  • Fix correctness
    grader scripts
    4 / 4
  • Diagnosis
    AI, capped by reads
    3 / 4
  • Blast radius
    code sets the ceiling
    2 / 4
  • Verification
    AI, capped by reads
    3 / 4
  • Root-cause explanation
    AI vs the answer key
    3 / 4
Solved or not solved is never the model’s call. It is the first row, and code decides it.

03

What the report tells you

Not a percentile. A score out of 100 across the five criteria, a recommendation, the time to the first real look at the broken object, how many changes they made before they had evidence, and what state they left the cluster in. It ends with a short list of questions for the next round, each tied to a moment in the command log.
Interview report · #4-217Solved · Advance
  • Commands in the right direction71%
  • Hypotheses tried first2
  • Dead ends in / out1 / 1
  • Commands that changed state6
Scored from the whole interview. Every score cites a real command.

04

Blast-radius score

Deleted a namespace to make the symptom go away? Disabled a liveness probe and never re-enabled it? Force-deleted where a safe operation existed? Code classifies each of those from the command log, and it sets the ceiling on the score. The model is only ever allowed to go lower, never higher. We have not found another assessment that scores harm done while fixing. Recovering from your own mistakes counts for you; hiding them does not.
Blast radius1 finding
Disabled liveness probe, never re-enabled
at 09:14 · deployment/payments-api · still off at submit
No destructive shortcuts
nothing deleted to hide the symptom
Solved, with a guard left off

05

AI-allowed by design

Most competitors sell AI detection, an arms race they lose. We run none. Candidates bring whatever they would have open during a real incident, and nothing about the interview depends on guessing what wrote a given command, because Solved or Not solved comes from the cluster, and the cluster does not care who typed it. Honesty note: this is architecture, not enforcement. A candidate can use their own laptop, and we say so plainly.
Candidate AI policyallowed
  • AI toolsallowed, any model, no disclosure needed
  • Detectionnone, we do not run any
  • Proctoringnone: no camera, no mic, no screen
  • Tab switchingnot watched, not scored
  • What is gradedthe cluster, and the commands that got there
The same policy for every candidate, stated up front in the brief

06

The full command log, not screen video

Hiring managers do not trust assessment scores. They trust watching the fix happen. Every report carries the full command log of the interview: every command, exit code, and dead end, in order. It is text, not screen video: kilobytes instead of gigabytes, searchable, and nothing from the candidate’s screen, camera, or mic ever leaves their machine.
Command log · interview #4-21741 KB of text
[09:14:02] $ kubectl get deploy payments-api -o yaml | grep -A4 livenessProbe
[09:14:31] $ kubectl edit deploy payments-api # ← the moment: probe removed
[09:15:07] $ kubectl rollout status deploy/payments-api
No video. No camera. No mic. Ever.

Roadmap

What’s next

Not shipped yet, listed anyway, customer feedback decides the order. What is deliberately not here: screen recording, webcams, or proctoring of any kind. The command log is the terminal, full stop.

In-session AI (opt-in, candidate told up front)

An AI assistant inside the lab terminal. The full prompt trail is graded alongside the commands: what context they gave it, which suggestions they caught as wrong, whether they verified before applying. Bring-your-own-AI stays allowed either way.

Randomized fault variants

One lab compiled into many concrete faults, with the invite picking one. Today each lab injects a fixed fault, so a leaked walkthrough is blunted by the grader rather than by the fault itself. Variants are the structural fix and they are being built.

Lab suggestions from a job description

Upload the JD, get a shortlist of labs that test what it actually asks for, with your own constraints (how many labs, how long each) respected. Today you filter the library yourself, and every lab says in plain English what it puts under pressure, so it is a two-minute job rather than an impossible one.

Live shadow mode (opt-in, candidate told up front)

Watch the interview in real time, read-only. For panel interviews that want to talk through the incident afterward, not surveillance.

Team incident drills

Same format, aimed at the team you already have: scheduled game days, blast-radius scores per engineer, and a report you can bring to the postmortem. Practice the 3 AM page at 3 PM.

More stacks

Every lab today is Kubernetes, and the platform can only boot Kubernetes sandboxes. We would rather say that than imply a breadth we do not have. Linux, Docker, Terraform, and CI/CD come next, then databases and cloud-provider labs: Postgres on fire, a pipeline that ships the wrong artifact, a cloud bill exploding.

Candidate skill profile

A verified record of solved labs: what was broken, what they fixed, and what the fix cost. Signal that travels with the engineer, not the employer.

Environments, data, and what we refuse

The questions your security reviewer will ask, answered before they ask them.

Disposable every time

Every interview and practice run gets its own environment, built for that attempt and destroyed when it ends. Candidates are expected to break things, so nothing they touch is shared with anyone else.

Nothing to install

The candidate needs a desktop browser. No VPN, no cloud account, no agent on their machine, no admin rights. Nothing from their laptop is touched.

What is recorded

Terminal commands, their exit codes, and diffs of files changed inside the environment. That is the complete list, and it is what the command log is made of.

What is never recorded

No camera, no microphone, no screen capture, no biometrics, no keystroke dynamics. None of it exists in the product, and adding any would mean changing the privacy policy first.

Retention and deletion

Interview reports are kept 12 months. Practice reports stay until you delete them.

Your code stays yours

Labs run on our own environments, not your repositories. A custom lab is built to resemble your stack, not to contain it, so nothing proprietary leaves your side.

Full detail in the privacy policy →

How Faultybox compares

Categories, not names. You know who they are.

FaultyboxCode-sandbox platformsQuiz platforms
Real infrastructurelive Kubernetes cluster, cluster-adminsandboxed snippetsmultiple choice
Solved or not decided bygrader scripts against the live clustertest cases on outputan answer key
Durability checkedthe fix must hold
Blast radiusscored
AI policyallowed, no detection, no proctoringdetection arms racebanned / proctored
Command logevery interviewsometimes

Check the fit before you start: who this is for →

The box is faulty. Want in?

Free to start, no card. Hiring teams get 3 interviews and 3 trial runs on every interview lab. Engineers get 3 free practice runs on the free labs.