Learner competency is calculated based on the last 3 experiences of a bundle. The level is determined based on accuracy and speed of delivering instruments.
The levels are calculated based on the following criteria:
The Scoring system is used for immediate debrief and learner guidance but is not used as a criteria for assessing competency levels.
Accuracy is measured by comparing the presented instrument against the requested instrument. Multiple incorrect presentations against the same request will be counted against accuracy. Unexpected Presentation where an instrument is delivered to the tray when it was expected to the hand is reported in scoring and feedback but is not counted against Accuracy.
Each instrument presentation has a variable time allowance for instrument identification which starts with the request plus a fixed time allowance for delivery. The time taken from the request until the correct instrument is delivered is the recorded time. Two independent time variables are calculated:
Sum of Time Lost where instruments are delivered after the allowed delivery time i.e. late; and
Sum of Time Saved where instruments are delivered during the identification period indicating the learner correctly anticipated or quickly recognized the instrument.
Both variables are summed independently and a learner can have both Time Lost and Time Saved within the same experience indicating variable performance across the complete instrument set.
Performance is observed over a set of experiences to ensure that competence is not only demonstrable but also repeatable.
Although the complete experience history is maintained the current competency level is assessed only over the 3 most recent experiences. Assessment is automatic and continuous therefore the current level may go up or down based on the next experience.
The Competency Level assessment is hierarchical in that each level is based on and gated by the previous level e.g. Speed of delivery resulting in good Timing is not a determining factor until the Learner is sufficiently and repeatably Accurate.
Has completed less than 3 experiences or has caused a Never Happen event to occur in VR (Violated sterile field, Dropped an instrument)
D - Inaccurate
Accuracy: has passed more than 1 incorrect instrument in a single experience or has an average of 1 incorrect instrument pass over the last 3 experiences (i.e. no completely correct experiences).
C - Slow
Accuracy: At least one experience has zero incorrect instruments and no experience has more than 1 incorrect instrument pass so an average of less than 1 incorrect instrument over the last 3 experiences(Accurate).
Timing: Delivering instruments outside of the allowed time such that the aggregate Time Lost divided by the Total Allowed Time per experience is greater than or equal to 5%
B - Competent
Accurate over last 3 experiences; and
Timing: Generally delivering instruments within the allowed time such that the aggregate Time Lost divided by the Total Allowed Time per experience is less than 5% (On Time)
A - Proficient
Accuracy: No incorrect instruments passed over last 3 experiences;
Timing: On Time delivery along with an positive anticipation time or reaction time measured as aggregate Time Saved divided by the Total Allowed time per experience is 5% or greater.