Prompt quality
How clearly they brief the AI: context, constraints, what good is.
The checks behind every report, how to run each role, and what it saves.
One paste into an AI does not finish a task like this one.
Example task · Python Engineer
Add refunds to the orders service. Refunds must be safe to retry. Explain how you chose to do it.
def refund(order, amount): gateway.refund(order.id, amount) order.status = "refunded" Looks right. A retry refunds twice.
+ idempotency key per refund + reject orders that are not paid + test: a retry refunds once
Four on how they use AI, two on the work, one for your team.
How clearly they brief the AI: context, constraints, what good is.
Whether they push back on a weak answer instead of taking it.
Whether they run the code after the AI changes it.
pasted from the AI14 linesreworked by hand9 linestyped by hand31 linesHow much of the final code they wrote or reworked themselves.
empty file4 msquoted commas6 ms10,000 rows812 msmixed line endingsfailedWhether it works on tests they never saw.
left the test tab0 timespasted from outside the test0 timestime in the test58 minWhether they stayed inside the test: tab leaves and outside pastes are flagged.
Your team reads the evidence together and makes the call.
Camera, microphone and screen capture the candidate agrees to first, with every flag on a timeline.
merge_windows handle an empty list?spans[0] raises. Add a guard: if not spans: return []An assistant in the workspace on a budget you set. Every prompt is captured and scored.
A shared editor, tagged by author. Your notes and rating save to the candidate.
The same report answers the engineering lead, the recruiter and HR.
Engineering managers and VPs
Talent sourcing and partners
HR and people leaders
The report is the start. Everyone who signs off works from the same record.
See each role’s candidates as they move: invited, in the test, submitted, reviewed. Open the ones waiting on you first.
Request a structured second opinion: a verdict, an overall rating, and ratings per dimension, kept next to the evidence.
Draft the job description and the questions with it. Anything that affects a candidate waits for your approval.
Enter your hiring volume and engineer cost. We show what you get back each month. Every figure is a sum you can check.
One assumption: reading a report takes about 10 minutes, so each assessment gives back 60 − 10 = 50 engineer-minutes.
One avoided bad senior hire (an estimated $250,000) pays for 41 years of assessment at this volume.