The National Commission for the Certification of Crane Operators (NCCCO) exams are not meant to be easy. They are meant to challenge a crane operator’s knowledge, skills and abilities to determine if they know the information necessary to keep their lifts safe.
But how do those questions get on the test and who ensures that the questions are correct? The NCCCO Exam Management Committees that is who. The NCCCO Exam Management Committees are made up of crane users and subject matter experts from manufacturers, union representatives, training companies, construction and industry.
At the end of January, Crane Tech had the privilege to host the Written Exam Management Committee (for the Mobile Crane Operator certification exams) at our headquarters. This was one of the more than 15 times we have hosted such meetings in Florida during the middle of winter. But it was not all fun in the sun, the Committee got right to work on topics such as: reviewing how the exams are performing using psychometric and statistical data, evaluation of examinee comments received, considering the load charts being used on the exams, and, when necessary, the development of new questions, etc.
So, if you were thinking that once the exam was developed it was never looked at again, this is not the case. Prior to the meeting, NCCCO staff analyze the performance of the written and practical examinations to determine if there are any concerns that the Committee needs to address on an individual question level. When the Committee reviews EACH of these outliers along with any comments, they discuss the concern and collectively determine if a change needs to be made to the wording of the question, the response options (a, b, c, d), or to inactivate the question all together. Any updates can then be made real-time to the database that houses the exam questions, so that future exams reflect the most accurate version.
While this process may seem easy, we can assure you that it is not. The review of questions includes using reliability statistics such as the percent of candidates answering a given question correctly (P value) and a discrimination factor (point biserial, or r-bis). These data, along with the number of times the question has been asked and the distribution of the answers among the response options, are just a few of the analyses discussed at these meetings.
The bottom line?
The written exams are living documents. They undergo regular maintenance reviews to make sure the exams perform to standards. Each exam must test the knowledge of those seeking certification in a fair, valid and reliable way. So, if you take the exam and find a few questions challenging, it’s supposed to be that way. However, the WEMC encourages individuals taking the exam to write in comments related to questions they struggle with; you never know it could actually lead to a change on a future exam.
Remember, Safety through Education is more than just our motto, it is our guiding principle. In fact this passion drives us so much that in 2008 Crane Tech donated our proprietary software, which was developed for our own testing needs, to NCCCO so they could use it to house the CCO exam questions and statistical data and maintain the quality of the exams. This is the software that NCCCO still uses today, and that is a fact we are very proud of.
P.S. If you want more information on how NCCCO monitors exam performance check out this post on the NCCCO’s website.