The detail

How SuperTests judges a candidate

The checks behind every report, how to run each role, and what it saves.

Beyond the code

AI writes the code. We test the judgment.

One paste into an AI does not finish a task like this one.

Example task · Python Engineer

  1. Design trade-off

    The task

    Add refunds to the orders service.
    Refunds must be safe to retry.
    Explain how you chose to do it.
    
  2. Checking the AI’s work

    What the AI writes

    def refund(order, amount):
        gateway.refund(order.id, amount)
        order.status = "refunded"
    Looks right. A retry refunds twice.
    
  3. Diagnosis and failure cases

    What a strong candidate does

    + idempotency key per refund
    + reject orders that are not paid
    + test: a retry refunds once
    

The seven checks

Seven checks show how each candidate works with AI and without it.

Four on how they use AI, two on the work, one for your team.

  • Every prompt and AI reply is recorded and replayable, so you see how they worked.
  • Hidden tests they never saw judge the code, so a clever prompt alone cannot pass.
  • Pastes and tab-leaves are logged and traced. You read the evidence, then decide.
check 01 · prompt quality
candidate → AI
✓ gives context✓ sets a constraint✓ asks for proof
✓ every prompt kept in the report

Prompt quality

How clearly they brief the AI: context, constraints, what good is.

check 02 · challenges the ai
Here is a CSV parser that splits each line on commas.
That breaks on quoted commas. Handle quotes and add a test for it.
Updated: quote-aware split, plus the test.
✓ pushed back · 2 rounds
✓ the whole conversation, replayable

Challenges the AI

Whether they push back on a weak answer instead of taking it.

check 03 · tests ai changes
  1. the AI suggested a fix
  2. ran the tests on it
  3. edited 3 lines
  4. accepted it once tests passed
✓ tested before it was kept

Tests AI changes

Whether they run the code after the AI changes it.

check 04 · owns the code
pasted from the AI14 lines
reworked by hand9 lines
typed by hand31 lines
✓ 89% of the final code is theirs
✓ every paste traced to its source

Owns the code

How much of the final code they wrote or reworked themselves.

check 05 · passes hidden tests
hidden tests · run on submit
empty file4 ms
quoted commas6 ms
10,000 rows812 ms
mixed line endingsfailed
✓ 3 of 4 passed
✓ tests they never saw

Passes hidden tests

Whether it works on tests they never saw.

check 06 · plays fair
integrity trail
left the test tab0 times
pasted from outside the test0 times
time in the test58 min
✓ a clean trail
✓ every tab leave and paste logged

Plays fair

Whether they stayed inside the test: tab leaves and outside pastes are flagged.

check 07 · team review
Briefs the AI like a lead.
Checked every answer it gave.
Let’s make the offer.
✓ Hired · your team’s call
✓ people make the call

Team review

Your team reads the evidence together and makes the call.


Three ways to run it

Run each role your way: proctored, AI allowed, or live.

assessment · priya-rathod
38:12REC
CameraMicrophoneScreen share
session rhythm46 min
✓ integrity 961 internal paste, no deduction
01

Proctored, with a full record.

Camera, microphone and screen capture the candidate agrees to first, with every flag on a timeline.

  • consent-first
  • server clock
ai transcript · priya-rathod
Will merge_windows handle an empty list?
No — spans[0] raises. Add a guard: if not spans: return []
AI budget100% left
  • verified AI output before accepting
  • asked about edge cases unprompted
02

Allow AI, and score how well they use it.

An assistant in the workspace on a budget you set. Every prompt is captured and scored.

  • AI toggle
  • transcript scored
live · final round
1def rate_limit(user, now):
2 window = now - 60candidate
3 hits = user.hits(window)
4 return interviewer
private notes · rating★★★★★
03

Pair live, with a record of who wrote what.

A shared editor, tagged by author. Your notes and rating save to the candidate.

  • co-editing
  • author-tagged replay

Who signs off

Built for every person who has to say yes.

The same report answers the engineering lead, the recruiter and HR.

  • Engineering managers and VPs

    See how they built it, not just that it passed.

    • Hidden tests the candidate never saw judge the code itself.
    • Every prompt, paste and tab-leave is recorded and replayable.
    • Python and Java today, with Django and Spring Boot.
    Open a sample report
  • Talent sourcing and partners

    One link per candidate. Nothing to install.

    • No account or install for candidates; work auto-saves.
    • No ATS to connect: send the link from wherever you work.
    • First 10 free, then $5 each. Credits never expire.
    Need an ATS connection? Tell us which
  • HR and people leaders

    Evidence you can explain to a candidate.

    • Nothing is rejected automatically; your team decides.
    • Every deduction names its reason and timestamp.
    • Consent before capture. Never sold, deleted on verified request.
    Read Trust & security

For the whole team

One workspace for every role, reviewer and decision.

The report is the start. Everyone who signs off works from the same record.

  • One view per role

    See each role’s candidates as they move: invited, in the test, submitted, reviewed. Open the ones waiting on you first.

  • Ask a teammate to review

    Request a structured second opinion: a verdict, an overall rating, and ratings per dimension, kept next to the evidence.

  • An AI copilot for the busywork

    Draft the job description and the questions with it. Anything that affects a candidate waits for your approval.


Savings calculator

How much SuperTests saves you.

Enter your hiring volume and engineer cost. We show what you get back each month. Every figure is a sum you can check.

Assessment type
Live screen a report replaces

One assumption: reading a report takes about 10 minutes, so each assessment gives back 60 − 10 = 50 engineer-minutes.

Your estimated saving, every month
$6,164
That is $73,968 a year saved, after what SuperTests costs.
Engineer time saved
83.3h × $80
$6,664
SuperTests cost
100 × $5
− $500
You save
$6,664 − $500
$6,164

One avoided bad senior hire (an estimated $250,000) pays for 41 years of assessment at this volume.