Community
By Daryl Lee Spiewak, CEM, TEM, MEP, Lead Trainer for the CEM Commission, and Chair, IAEM-Global Communications Work Group
Our last discussion on the CEM examination completed the topic of Exercises and Tests with a focus on exercise design and development with an emphasis on generating corrective actions listed in the improvement plan. This month we begin a new discussion on Program Maintenance and Improvement with a general overview of the requirements.
Emergency managers perform two broad categories of activities. The first category is the core functions, and these are those functions performed during emergency response and recovery. The second category includes the program functions performed on a daily basis. See FEMA Independent Study Course 230d, Fundamentals of Emergency Management, for more details about them.
Maintenance and improvement of your emergency management program contains many of the same elements that we previously discussed with exercises and tests, as we shall see. But first, how do we determine if our program meets performance standards or if it needs to be improved? What tools or techniques do we use to make this determination?
Performance is determined in two different ways – by measurement and by evaluation. Measurement is an assessment or observation where you determine exactly (using concrete numbers) what your organization has accomplished. For example, the number of trained (in a specific task or skill) personnel, numbers and types of operational equipment on hand, numbers and types of public awareness events conducted, and the number of plans exercised and updated. This is an important and useful tool for both comparison (before and after) and for management (establishing priorities).
Evaluation, on the other hand, is where you actually observe performance and compare it to established standards and competencies. Based on those results, you then determine if the actual performance meets expectations. If so, then seek to maintain proficiency. If not, improvements are in order. NFPA 1600, version 2016, specifies we use evaluation: “The entity shall maintain and improve the program by evaluating its policies, program, procedures, and capabilities using performance objectives.”
Notice that last part – “Using performance objectives.” This is another key component often overlooked when evaluating program performance, because it is very easy to rely solely on measurement criteria because numbers are easy to count and are used to impress. But numbers by themselves do not tell the whole story. Is having 100 personnel trained in the organization’s incident management system enough? Too much? Not enough? To know for sure, we need to compare these numbers against performance objectives just as we do for exercises and tests.
Using performance objectives is only one criterion. NFPA 1600 further requires, “The entity shall improve effectiveness of the program through evaluation of the implementation of changes resulting from preventive and corrective action.” So not only do we have to evaluate our program’s performance, we also must evaluate our change process for both preventive and corrective actions. We must come full circle and evaluate whether the changes we implemented actually provided the maintained or improved performance we expected.
NFPA 1600, version 2016, requires we conduct program reviews “on a regularly scheduled basis and when the situation changes to challenge the effectiveness of the existing program.” The “regularly scheduled basis” criterion is an easy one. It could be yearly, biannually, every five years, etc. Establish one if your senior executives have not done so for you already. The second part is a little trickier – changing situations that challenge your program’s effectiveness. It requires you to pay attention to a multitude of events occurring all around you and your program.
Here is a list of things that change over time that could have a large impact on your program’s effectiveness. When changes in any of these things occur, you should re-evaluate your existing program and determine if changes and improvements are necessary. The list is provided by NFPA 1600, version 2016, and it is not all-inclusive. There may be additional things that will impact your program which you need to be aware of and monitor for changes.
(1) Regulations (2) Hazards and potential impacts (3) Resource availability or capability (4) Entity’s organization (5) Funding changes (6) Infrastructure, including technology environment (7) Economic and geographic stability (8) Entity operations (9) Critical suppliers
And as we do for our exercises and tests, activities, you need to include post-incident analyses, reviews of lessons learned, and reviews of your program’s performance. All of these will provide you with critical information and criteria for evaluation of your emergency management program.
Just like everything else we do, the job is not complete until the paperwork is completed. You need to document your program reviews and evaluations in accordance with your organization’s records management policies and procedures. Then make those documents and records available to your upper managers for their review and follow-up actions.
As for the previous standards, NFPA 1600 does not describe how to conduct a program review or a program improvement process, so for the exam we need to refer back to various FEMA Independent Study Courses and the HSEEP Manual that told us how to conduct evaluations of our exercises and tests and how to develop an improvement plan. The process and procedures are similar. Therefore, review independent study courses related to exercises such as IS-120a - An Introduction to Exercises and IS-130 - Exercise Evaluation and Improvement Planning for the core questions. USA candidates should also review the Homeland Security Exercise Evaluation Program (HSEEP) documents.
The application process for both the CEM and AEM does not require the candidate to address Program Maintenance and Improvement specifically as one of the required Knowledge, Skills, and Abilities (KSA) components for the emergency management essay. However, it does require discussion of Prevention, Preparedness, Response, Recovery, and Mitigation activities, all of which should be a part of the program reviews. In addition, candidates may choose Program Maintenance and Improvement as a key component of their Problem Statement and write about that (being sure to cover all the KSAs). Many successful certification candidates do.
Here are two core-type questions for our analysis in this article.
1. How often should an emergency manager perform a review of his or her program?
a. After every policy change. b. Annually. c. In conjunction with each after-action review. d. On a regularly scheduled basis.
This question is asking you to know the triggers for emergency management program reviews. Does every policy change impact your emergency management program? Most decidedly not, so response a. is incorrect. Recall NFPA 1600, version 2018 requires a review on a regularly scheduled basis; it does not specify what that regular schedule should be, so response b. is incorrect and response d. is probably the correct one. Remember annually may be a correct response for your organization, but it is not every organization so it is listed as incorrect here. Response c. seems reasonable, but in reality the lack of time and other resources will prevent you from reviewing your program after every incident involving an AAR. You have too many other associated activities to perform first. In addition, there are a multitude of events that will trigger a program review but not an incident response and AAR. Therefore, the only correct response is d.
2. Does an emergency manager need to conduct further reviews of the emergency management program if thorough after action reviews are conducted following each major incident response?
a. No, the after action review covered everything in the program. b. No, one program review is all that is required. c. Yes, the after action review covered the core emergency management functions. d. Yes, the after action review did not cover the emergency management program functions.
This question is similar to the previous one. It is asking you to understand what makes up your emergency management program, the differences between core and program functions, and when you should conduct performance reviews. From previous study we know after action reviews are conducted following incident response and recovery. They focus on evaluating core emergency management functions. We still have to evaluate our program functions. So the two “no” responses are incorrect. That also makes the first “yes” response in c. incorrect. Therefore, the correct response is d. See IS 230d for the differences between core and program functions.
When reading the questions and responses, be sure you understand exactly what the question is asking of you, and read each response before selecting the correct one. It is too easy to get distracted and select a response that appears to be correct but is not the correct re- sponse for the question being asked.
Next month we continue our discussion on Program Maintenance and Improvement with an emphasis on the corrective action process. We also will analyze some practice exam questions. As usual, please send any questions you have about the examination or the certification process to me at info@iaem.com, and I will address them in future articles.
IAEM Bulletin, February 2018
AEM® and CEM® are registered trademarks of the International Association of Emergency Managers.