How the assessment works and is scored

This page describes the assessment completely, including what it does not do yet. A test is only as credible as its method, so the method is public.

Levels

Associate: Branch office, 3 incidents and 1 change request, 90 minutes, pass mark 70.
Professional: Enterprise edge (FortiGate + Catalyst), 4 incidents and 2 change requests, 150 minutes, pass mark 70.

Every candidate gets a different exam

When you start, the server draws a random seed. The seed decides the site names, the addressing plan, the VLAN numbers, which faults are injected and which change requests you get. The catalogue has 26 faults and 4 change requests across the two network templates, each with a reference fix that our automated tests prove works. Faults are chosen so that each one causes a symptom of its own: every incident ticket describes something a user would actually report, never the cause.

Graded on what the network does

Every ticket has requirements written as intents: this PC gets an address in that subnet, this host reaches the Internet, this server answers HTTPS from the Internet but not SSH, guests cannot reach staff. They are checked by sending traffic through the simulated network at the end. The commands you used do not matter, only the result, so any correct design passes.

Score: outcome, no harm, method

Outcome, 70%. The share of all ticket requirements that are met at the end.
No harm, 15%. Everything that worked at the start and is not part of a ticket must still work at the end. Breaking more than one such service fails the attempt whatever the score.
Method, 15%. Half for looking before changing (up to three diagnostic commands before your first configuration change), half for verifying after changing (a diagnostic command after each group of changes). This rewards a habit, not speed.

To pass you need the pass mark, at least half the tickets fully solved (partial progress on many tickets does not add up to a pass), and at most one working service broken. Distinction: 90 or more, every ticket solved and nothing broken. Time per ticket and the number of changes are reported but not scored.

Graded on the server

Your console log (every line, with a timestamp) is uploaded while you work. When you submit, or when time runs out, the server rebuilds your exam from its seed, re-types your log into it and grades the resulting network. Lines logged after the time limit are ignored. Nothing your browser reports about its own state is trusted.

Rules and integrity

Open book: documentation and the built-in command reference are allowed. Other people are not.
Certification attempts: one per level every 7 days. Practice runs are unlimited and use different seeds.
Certificates are valid for three years and can be verified at their public link.

Limits of the pilot

Identity is not proctored. A certificate shows that the account holder passed, not who sat the exam.
AI assistants cannot be ruled out. Unique networks, symptom-only tickets, a time limit and method scoring reduce their value, but we do not claim the test is AI-proof.
Pass marks are provisional. During the pilot we measure, for every fault and change request, how often it is solved and how well it separates strong candidates from weak ones, and we will review the pass marks with practising engineers before the pilot ends.
The network is simulated. Commands outside what the simulator supports are rejected; outcome-based grading means you are never marked down for choosing a different valid command that the simulator does support.

Adaptive mode (pilot)

In adaptive mode you get one incident at a time, each on its own freshly generated network, across both templates. After every item the server grades it (solved means fixed with nothing else broken), re-estimates your ability with a Rasch model (Bayesian estimate with a standard normal prior), and serves the unused fault whose difficulty is closest to that estimate. It stops after 4 to 8 items, once the estimate is precise. The result is a practical skill index from 200 to 800 with a 95% range. Item difficulties start as expert estimates and are re-estimated from pilot responses; until then adaptive runs do not issue certificates.

Live interviews

Employers can run the same network live during a video interview: the interviewer injects faults from the catalogue while watching the candidate's consoles and a live board of every requirement. The candidate's browser records everything, injected faults included, so the session is graded and replayed exactly like an assessment: per fault, whether and when it was fixed.

NetOps Bench

The same environment, graded the same way, is offered to AI agents on public seeds (NetOps Bench). Humans can take those exact networks as a practice run, so people and agents are compared on identical problems.

Your data

We store your command log and result with your account. Your result is private unless you share your certificate link. If an employer invited you, they see your result, and the session replay only if you agreed to it when you started.