Every ISO/IEC 17024 system rises or falls on the strength of its exams. You can have great assessors, clear procedures, and a perfect manual — but if your exam can’t prove competence fairly and consistently, accreditation becomes shaky.
In my years helping certification bodies prepare for ISO/IEC 17024 accreditation, most findings come down to one missing link: no clear exam blueprint or controlled item bank.
These two documents are the backbone of assessment integrity. The blueprint shows that your exam covers the right competencies. The item bank proves you control every question from creation to retirement.
In this guide, you’ll learn exactly how to build, maintain, and document both so your assessments stand up to any audit.
Understanding the Exam Blueprint — Mapping Competence to Assessment (Clause 7.3)
Think of the exam blueprint as your roadmap. It connects the competence criteria in your certification scheme to the specific questions or tasks used to test them.
It answers questions like:
What knowledge areas will be tested?
How many questions per area?
What’s the weight or importance of each domain?
Which assessment methods apply — written, oral, or practical?
A well-built blueprint ensures balance. It prevents over-testing one area and missing another. It also demonstrates fairness, which is a core requirement in ISO/IEC 17024 Clause 7.3.
Pro Tip: Use a simple table showing domains, weights, and question counts. Assessors love seeing that every topic has clear logic behind it.
Common Pitfall: Writing the exam first and building the blueprint afterward. That’s backwards — and assessors will notice.
Developing the Exam Blueprint Step-by-Step
Here’s a practical way to design your blueprint:
Start with the job or task analysis. Identify the key duties and expected outcomes for the role.
Group tasks into domains or competencies. These become the main exam sections.
Assign weightings. Use importance and frequency to decide how much each domain counts toward the total score.
Select assessment methods. Choose how each domain will be tested — multiple choice, simulation, observation, or oral exam.
Set the passing score methodology. Define how the cut-score will be calculated (Angoff, modified Ebel, or expert judgment).
Example Blueprint Snippet:
Domain
Weight (%)
No. of Items
Assessment Type
Reference
Technical Knowledge
40
40
MCQ
Clause 7.3.1
Practical Application
35
20
Performance Assessment
Clause 7.3.1
Ethics & Safety
25
15
Scenario Questions
Clause 7.3.1
Pro Tip: Have your blueprint validated by subject-matter experts. Their endorsement provides defensible evidence of relevance.
Building and Maintaining the Item Bank (Clause 7.3.1 & 7.4)
If the blueprint is the roadmap, the item bank is the warehouse — a controlled database of every validated question you use or plan to use.
Each item in the bank should have metadata such as:
Domain or competency area.
Difficulty level.
Type (MCQ, practical, oral).
Date created, reviewed, and validated.
Author and reviewer names.
Current status (active, under review, retired).
Pro Tip: Assign a unique ID to each item (e.g., KB-T1-Q034). It simplifies tracking and avoids duplication.
Common Pitfall: Keeping item banks in unsecured spreadsheets with no validation history. Accreditation assessors expect version control and documented reviews.
Example: One certification body I supported stored its questions in multiple Word files with no traceability. During assessment, the accreditation team flagged it immediately — they had no way to prove question validity or security.
Item Validation & Review Process — Ensuring Reliability
Every question must earn its place in the exam. ISO/IEC 17024 expects that each item is validated before use and monitored after.
Your validation process should include:
Content validation: Does the question test the intended competence?
Psychometric review: Is it the right difficulty level and discrimination index?
Bias review: Is it free from cultural or gender bias?
Post-exam analysis: Review item performance after delivery.
Pro Tip: Maintain a “Validation Record Log.” Include reviewer comments, pilot-test results, and revision dates.
This record shows assessors that your questions aren’t just written — they’re proven.
Recordkeeping & Security Requirements (Clause 9)
Your item bank and exam data are sensitive intellectual property. ISO/IEC 17024 requires you to show how you protect, back up, and control access to them.
Your documentation should include:
Who has access (by role, not by name).
How access is granted and revoked.
Backup frequency and storage method.
Destruction process for retired or compromised items.
Pro Tip: Treat your item bank like a financial system. Every edit, review, or deletion should leave a trace — with timestamps and responsible person logged.
Common Pitfall: No documented backup plan. I’ve seen one case where a single hard-drive crash wiped out an entire question bank. Recovery took months and delayed recertification.
Continuous Improvement & Periodic Review
Exams must evolve with the profession. Clause 7.3.1 expects periodic review to ensure continued validity.
Include in your system:
Annual item-performance analysis.
Scheduled review of the exam blueprint (every 3–5 years).
Feedback integration from assessors and candidates.
Revision log showing updates made to items and weightings.
Pro Tip: Hold an annual “Exam Review Committee” meeting. Document minutes, decisions, and rationale — it’s powerful evidence during accreditation.
FAQs — Common Questions on Exam Blueprints & Item Banks
Q1: How often should we update our item bank? At least once a year, or whenever major scheme or regulation changes occur. Continuous micro-updates are even better.
Q2: Do small certification bodies need psychometric data for every item? Not always. If resources are limited, documented expert review and pilot testing are acceptable alternatives.
Q3: How long should we keep old exam versions? Keep all versions and results for at least one complete certification cycle after they’re retired. Auditors often ask for historical evidence.
Strong Exams Build Trust and Accreditation Confidence
An exam that’s fair, validated, and traceable tells accreditation bodies that your system works — not just in theory but in practice.
Your exam blueprint shows structure and logic. Your item bank shows control and evidence. Together, they form the proof that your certification body certifies competence objectively and consistently.
In my experience, once certification bodies start maintaining a properly versioned blueprint and secure item bank, audit findings drop sharply and credibility rises fast.
If you’re ready to strengthen your system, QSE Academy’s ISO/IEC 17024 Exam Blueprint & Item-Bank Templates make it simple — pre-formatted, clause-referenced, and fully editable for your scheme.
Melissa Lavaro is a seasoned ISO consultant and an enthusiastic advocate for quality management standards. With a rich experience in conducting audits and providing consultancy services, Melissa specializes in helping organizations implement and adapt to ISO standards. Her passion for quality management is evident in her hands-on approach and deep understanding of the regulatory frameworks. Melissa’s expertise and energetic commitment make her a sought-after consultant, dedicated to elevating organizational compliance and performance through practical, insightful guidance.
