How Aptora Evaluates Candidates

When a candidate finishes an Aptora assessment, you get back a scorecard: an overall score, a hiring recommendation, a short summary, and a breakdown across a handful of categories. This page explains what each piece of that scorecard means so you can review one in a minute or two and trust the result.

What we evaluate

Every assessment is scored across five default categories. Each one targets a different facet of how a candidate works, not just whether their final code runs.

• Problem Completion — How much of the problem the candidate actually finished, and whether they verified their own work. We give credit for completed and validated parts of the problem, not for code that merely looks done.
• Prompt Crafting — How effectively the candidate directed the AI agent. Strong candidates write clear, specific prompts that show they understand the problem (or honestly probe what they don’t yet understand). Using AI heavily is fine — using it carelessly is not.
• Problem Solving — How the candidate worked, not just what they produced. We look for iterative development, frequent testing, and grounded debugging when things break.
• Design Taste — Prioritization and judgment under a time limit. Did they spend their time on what mattered? Were their design choices reasonable for a time-boxed assessment?
• Code Quality — Whether the final code is readable, well-organized, and reasonably efficient.

Default weights

Categories don’t count equally. By default:

• Problem Completion — 30%
• Problem Solving — 30%
• Prompt Crafting — 20%
• Design Taste — 10%
• Code Quality — 10%

These reflect what most teams told us they care about: did the work get done, and was it done in a sound, repeatable way. You can change them per assessment (see Customization).

How scoring works

Each category gets a score from 1 to 10, along with a short reasoning blurb, a list of strengths, and a list of weaknesses. The overall score is a weighted average of the category scores, rounded to a whole number.

The grader is intentionally demanding. A 7 or above means genuinely strong performance — not “the code compiled.” Most reviewers find the calibration matches what they’d give the same session if they sat through it themselves.

Every scorecard also carries one of four hiring recommendations:

• Strong Hire — Clearly above the bar across the dimensions that matter for the role.
• Hire — Meets the bar with some room to grow; worth advancing.
• No Hire — Doesn’t meet the bar on at least one important dimension.
• Strong No Hire — Significant gaps; not worth additional engineering time.

The recommendation reflects the overall picture, not just the arithmetic. If there’s a critical gap in one dimension, the recommendation can be lower than the weighted average would suggest.

Customization

The default rubric is a starting point, not a fixed scoring engine. Per assessment, you can:

• Adjust the weights to emphasize what matters for the role. A platform team might weight Code Quality more heavily; a startup engineering role might lean further into Problem Solving and Design Taste.
• Add or remove categories. Custom categories take a name, a weight, and your own grading instructions describing what “good” looks like.
• Write general or category-specific grading instructions. These take precedence over the defaults — if you tell the grader to value a specific behavior, it will.
• Define seniority levels with their own time limits and grading nuance, so the same assessment can be calibrated differently for a junior vs. a senior candidate.

For a walkthrough of where to configure these, see How to Create and Share Assessments.

Integrity signals

The assessment runs in a controlled environment, and we surface a few signals that help you catch obvious misconduct:

• Tab switching — When a candidate leaves the assessment tab, we record it and note how long they were away. Brief checks are normal; prolonged or repeated absences right before correct solutions appear are not.
• External pastes — When code is pasted in from outside the environment, we flag it, especially when it’s large or arrives at a suspicious moment.

The grader weighs these patterns, not individual events. A single tab switch isn’t a problem. A candidate who switches away while stuck, returns, and immediately pastes a working solution is a problem — and the scorecard will say so. The final call is always yours.

What you see on the scorecard

When you open a candidate’s results, you’ll see:

• The overall score (1–10) and the hiring recommendation
• A two-sentence overview of the session
• Up to three top strengths and three top weaknesses
• A per-category breakdown: the score, a one-line headline, reasoning, strengths, and weaknesses

Most teams use this to make a clear go/no-go decision in a minute or two. For context on where Aptora fits relative to your other interview stages, see Where Aptora Fits in Your Interview Process.

Support

Questions about the rubric, calibration, or how to tune it for your role? Reach out at johnny@tryaptora.com.