Community
By Daryl Lee Spiewak, CEM, TEM, MEP, Lead Trainer for the CEM Commission, and Chair, IAEM-Global Communications Work Group
Last month we began a new discussion on Program Maintenance and Improvement with a general overview of the requirements. This month we continue our discussion on Program Maintenance and Improvement with an emphasis on the corrective action and continuous improvement processes.
As we noted in last month’s article, we periodically review and evaluate our emergency management policies, program, procedures, and capabilities using performance objectives, and then document those findings – often in the form of an After Action Report – in accordance with the organization’s records management policies and procedures. Those results form the basis for our Corrective Action Process.
NFPA 1600 version 2016 requires us to establish a corrective action process and to take corrective action on deficiencies identified. According to FEMA, corrective actions are “concrete, actionable steps intended to resolve capability shortfalls identified in exercises or real-world events.” These corrective actions may include items such as “updating or changing plans and procedures, organizational structures, and/or management processes.” They also might include the requirement for “additional training, equipment, or resources” (IS120c - Introduction to Exercises). FEMA (IS 120c and IS 130a) and the U.S. Department of Homeland Security (HSEEP) established a formal procedure for evaluating emergency management exercises through a standardized evaluation process followed by the development of an After Action Report and Improvement Plan. Use a similar procedure for reviewing and evaluating actual emergency responses as well as emergency management policies, program, procedures, and capabilities to determine areas requiring corrective action.
We begin the process by identifying our performance objectives to be evaluated. For emergency management exercises, the performance objectives identified for evaluation are found in the Exercise Evaluation Guides (EEGs). For actual response and regular reviews, program objectives may be found in the organization’s comprehensive emergency management plan; laws, regulations, and ordinances; policies; procedures; and training and equipment records.
Once the performance objectives are identified, we need to develop our evaluation plan, which specifies how we will collect the necessary data for analysis. We identify and train, if necessary, the evaluators and send them out to observe performance and collect the data. After all the data are collected, the next step is to compare actual performance against expected performance standards and determine why there are differences.
There are a number of methods used to analyze differences between what happened and what was supposed to happen. Some of them are called statistical analysis, cause and affect, What if, Kaizen, and Lean Six-Sigma. FEMA recommends we use a method called Root-Cause Analysis (also referred to as the Five Why’s method). Using this method, we ask why each thing did or did not happen and drill down until we arrive at the final root cause. The results of the root cause analysis are used to develop recommended corrective actions.
According to IS-130a, “Each corrective action should identify what will be done to address the recommendation; who (person or agency) should be responsible; and a time frame for implementation. A corrective action should contain enough detail to make it useful.” That means they should be “con- crete (specific and measureable), actionable steps that are intended to resolve capability shortcomings identified in exercises or real-world events.”
Organizational leaders review the draft AAR/IP, make any neces- sary adjustments to the plan, and approve the final results. The emergency program manager follows up with each action officer to ensure that improvements are implemented and revise the CEMP (comprehensive emergency management plan) as required to capture the changes. The changes are tested and evaluated in follow-up exercises and actual response as well as periodic reviews to ensure continuous improvement of the program elements.
The final component of Program Maintenance and Improvement is implementing continuous improvement. NFPA 1600 version 2016 states, “the entity shall effect continuous improvement of the program through the use of pro- gram reviews and the corrective action process.”
FEMA’s IS 120c tells us, “Identifying strengths, areas for improvement, and corrective actions resulting from exercises help organizations build capabilities as part of a larger continuous improvement process.” The same idea applies to periodic program reviews.
Some principles of continuous improvement identified in IS- 120c and IS-130a include:
As for the previous standards, NFPA 1600 does not describe how to conduct a program review or a program improvement process. For the exam, we need to refer back to various FEMA Independent Study Courses and the HSEEP Manual that told us how to conduct evaluations of our exercises and tests and how to develop an improvement plan.
The process and procedures are similar. Therefore, review independent study courses related to exercises, such as IS-120c – An Introduction to Exercises. FEMA’s Independent Study Course 130a – How To Be An Exercise Evaluator was published in February 2018 and should be reviewed too. The earlier version, IS 130 - Exercise Evaluation and Improvement Planning, is no longer available on the FEMA website.
The application process for both the CEM and the AEM does not require the candidate to address Program Maintenance and Improvement specifically as one of the required Knowledge, Skills, and Abilities (KSA) components for the emergency management essay. However, it does require discussion of Prevention, Preparedness, Response, Recovery, and Mitigation activities, all of which should be a part of the program reviews. In addition, candidates may choose Program Maintenance and Improvement as a key component of their Problem Statement and write about that (being sure to cover all the KSAs). Many successful certification candidates do.
Here are two core-type questions for our analysis in this article.
1. How do emergency management program evaluators identify issues for corrective action recommendations?
a. Comparing exercise or performance objectives to actual performance. b. Conducting a Controller/ Evaluator debriefing following an exercise. c. Conducting routine inspections of equipment, supplies, and training. d. Reviewing all the After Action Reports.
This question is asking you to know the procedures and methods evaluators use to make improvement recommendations. We know the first step is to identify performance objectives and standards. Then data are gathered on actual performance and compared against those objectives and standards. Analyses are conducted to determine why discrepancies exist between objectives and actual performance, and issues requiring corrective actions are identified. Responses b., c., and d. all relate to data gathering, while response a. says to compare actual performance against performance objectives. It is the discrepancies between the two that are used to recommend corrective actions to improve and sustain. Therefore, the only correct response is a.
2. Which of the following is a concrete and actionable corrective action?
a. Conduct additional training for all primary and secondary EOC staff. b. Develop procedures to implement the EOC secondary communications system. c. Obtain more references for use in the EOC during actual incident response. d. Update the local policy on Active Shooter response and recovery.
This question is asking you to determine which response is specific and measureable. Reading each response, we find that response a. recommends additional training, but it doesn’t specify what training is needed although it does specify who is to receive the training. Response b. appears good, because it specifies the action needed (new procedure) for a specific item (the EOC secondary communications system). Response c. isn’t specific enough because it doesn’t recommend which references to add to the EOC, and response d. isn’t specific enough because it doesn’t tell us what in the policy needs to be updated. Therefore, the correct response is b.
When reading the questions and responses, be sure you understand exactly what the question is asking of you and read each response before selecting the correct one. It is too easy to get distracted and select a response that appears to be correct but is not the correct response for the question being asked.
This completes our discussion on the program elements making up the CEM/AEM examination. The CEM Commission is in the process of updating the examination questions database and converting the examination from a paper hardcopy version into an online version. Once that process is completed, I will review it here and begin discussing any pertinent changes to the examination references and types of questions posed.
Next month I will begin a discussion on the essay, as that still seems to cause the majority of resubmission letters to candidates. As usual, please send any questions you have about the examination or the certification process to me at info@iaem.com, and I will address them in future articles.
IAEM Bulletin, March 2018
AEM® and CEM® are registered trademarks of the International Association of Emergency Managers.