by jmorrison
Share
by jmorrison
Share
Incident Command X (ICx) Evaluation Results | NIMS Concepts & Principles
Taken from the STEP Evaluation. For full report with tables, download the PDF report linked on our home page.
Executive Summary
This report presents the results from an evaluation of Incident Command X version number 2.0.2 (hereafter referred to as ICx), a product1 marketed by Battalion 3 Technologies. Evaluation activities are conducted as part of the Supporting Technology Evaluation Project (STEP). STEP is managed by the Preparedness-Technology, Analysis, and Coordination (P-TAC) Center, which is operated by Science Applications International Corporation (SAIC) under contract with the Federal Emergency Management Agency’s National Preparedness Directorate (FEMA NPD). Located in Somerset, Kentucky, the Center includes an emergency operations center (EOC) test environment, complete with supporting technologies, and the Incident Management Test and Evaluation Laboratory (IMTEL), an American Association for Laboratory Accreditation (A2LA) accredited laboratory.
The ICx evaluation was conducted from 06 through 08 August 2013. This was a User Acceptance Test; therefore, it specifically addressed adherence to National Incident Management System (NIMS) concepts and principles. The test does not address technical standards. This test had one objective:
Evaluate the system’s incorporation of NIMS concepts and principles.
ICx is a Microsoft Windows-based software program that serves as a command board for fire service incident commanders. It enables users to track the units (apparatus and associated personnel) assigned to an incident, as well as their assignments. It provides additional assistance in the form of incident-specific checklists for critical actions and personnel accountability reports (PARs) and preset alert and notification timers. An automatic logging function records all significant events for later review. The product can be configured to reflect an individual agency’s policies, standard operating procedures (SOPs), and organizational structure.
The vendor provided end user documentation, technical assistance for initial setup of the product, and one hour of online training for end users. Once setup and training were complete, the STEP team conducted test activities on-site at IMTEL and remotely, using desktop and notebook computers to run ICx and screen-sharing software to coordinate activities. Assessors with subject matter expertise in emergency response and emergency management conducted a test of the system and provided qualitative analysis and feedback on the product based on the concepts and principles from the NIMS document [Ref 3].
NIMS Concepts and Principles
Table 1: NIMS Criteria Rating Summary provides a summary of findings for NIMS criteria. Key elements identified within each NIMS criterion are cited as Minimum Product Requirements. These requirements were derived from the NIMS document and impact the overall rating of the product’s adherence to NIMS concepts and principles. The numbers provided below summarize ratings (i.e., Agree, Disagree, and Not Applicable) for Minimum Product Requirements within each NIMS criterion.
ICx is consistent with all NIMS criteria: Emergency Support; Hazards; Preparedness; Communications and Information Management; Resource Management; Command and Management. Overall, ICx applies to 12 of 24 Minimum Product Requirements, of which 12 are consistent with NIMS concepts and principles. Explanations of all findings are provided in Section 2.0: Results.
1.0 Introduction
This report presents the results from an evaluation of Incident Command X version number 2.0.2, a product marketed by Battalion 3 Technologies. Evaluation activities are conducted as part of the Supporting Technology Evaluation Project (STEP). STEP is managed by the Preparedness-Technology, Analysis, and Coordination (P-TAC) Center, which is operated by Science Applications International Corporation (SAIC) under contract with the Federal Emergency Management Agency’s National Preparedness Directorate (FEMA NPD).Located in Somerset, Kentucky, the Center includes an emergency operations center (EOC) test environment, complete with supporting technologies, and the Incident Management Test and Evaluation Laboratory (IMTEL), an American Association for Laboratory Accreditation (A2LA) accredited laboratory
STEP provides evaluations of supporting technologies relating to incident management and response. Test activities are designed to verify system compliance with National Incident Management System (NIMS) concepts and principles and applicable supporting technology standards. STEP evaluation reports provide the emergency management and response community with data to support purchasing decisions. For more information on the project and types of tests performed, visit the P-TAC Center website at https://www.ptaccenter.org.
A User Acceptance Test was conducted for ICx. The intent of this test was to determine the system’s incorporation of NIMS concepts and principles.
Vendor participation in STEP is voluntary. The use of trade names and test results in this document do not constitute a Department of Homeland Security (DHS) or FEMA endorsement or certification of the use of such commercial hardware or software.
1.1 STEP Project Summary
Systems operating in an incident management environment must be able to interact smoothly across disciplines and jurisdictions. Interoperability and compatibility are achieved through the use of mutually- supporting technologies that are compliant with the concepts and principles of NIMS and FEMA- accepted technical standards. The STEP project is designed to conduct controlled evaluations of products marketed as supporting technologies for emergency management and response. STEP evaluation reports provide feedback to the user community on product performance and compliance with NIMS principles and FEMA-accepted technical standards.
STEP testing takes place in a controlled, EOC-based environment. However, some systems may require an additional or alternate environment, such as a limited field setting. In these cases, the field setting is considered an extension of the laboratory environment. Tests vary in duration based on complexity but typically do not exceed four days. The team is scaled based on the complexity and type of test to include an analyst and assessors or test engineers. Participants adhere to a non-disclosure agreement, which ensures the protection of the vendor’s sensitive information.
The test took place at IMTEL, which is accredited through the A2LA. To achieve and maintain accreditation status, the laboratory meets general requirements for the competencies of testing and calibration laboratories, as provided in International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) 17025:2005. The current scope of
accreditation and associated certification required by ISO/IEC 17025:2005 is available on the A2LA website [Ref 1]. Results presented in Section 2.1.1: Objective 1: Evaluate Incorporation of NIMS Concepts and Principles are within IMTEL’s ISO/IEC 17025:2005 scope of accreditation. In the event that any individual findings fall outside the scope of accreditation, they will be clearly annotated as such. Other individual findings, observations, and results that fall outside the scope of accreditation are marked with an asterisk (*).
1.2 System Description
The vendor provided the majority of the information within this section. Assessors and test engineers did not verify all of the system’s capabilities during the evaluation, only those associated with the standards and criteria tested.
ICx is a Microsoft Windows-based software program that serves as a command board for fire service incident commanders. It enables users to track the units (apparatus and associated personnel) assigned to an incident, as well as their assignments. It provides additional assistance in the form of incident-specific checklists for critical actions and personnel accountability reports (PARs) and preset alert and notification timers. An automatic logging function records all significant events for later review. The product can be configured to reflect an individual agency’s policies, standard operating procedures (SOPs), and organizational structure.
1.2.1 Core Functionality
ICx’s focus is personnel assignment and accountability tracking on the fireground. The functions supporting these tasks are located in a user interface (UI) that mimics a physical (e.g., dry-erase or magnetic) command board. Figure 1: Command Board Overview provides an example of a command board for a commercial structure fire.
Figure 1: Command Board Overview
In the screen capture above, the top of the screen contains the primary menu bar and toolbar and an overall incident timer. The left side of the screen shows the units assigned to incident command roles (Incident Commander [IC], Public Information Officer [PIO], Safety Officer, and Liaison Officer), as well as units that are tasked to the incident but not assigned within the incident.
The central area of the screen shows assignments within the incident, as well as which units are in each assignment. The user can add, remove, or rename assignments to reflect the organizational structure in use for the current incident. Unit assignments are managed via a drag-and-drop UI, mimicking the process of moving tags on a physical command board.
The bottom section of the screen displays the details of the selected unit’s crew. It also contains controls for modifying that crew’s on-air status and radio channel assignment and for splitting that crew into two elements. The user can give a split crew’s elements separate assignments (e.g., Truck 3A to ventilation and Truck 3B to rescue) and can merge them when they resume operating as a single unit.
Figure 2: Unit Indicator Detail, shown above, is a close-up of a sample unit indicator. This includes the unit’s assigned radio channel, its department, its call sign or designation, and the length of time it has been in its current assignment. Once a unit has been in an incident duty assignment for a predetermined length of time (10 minutes by default), the unit indicator’s background color changes from green to yellow. The user can use the unit-level controls to flag a unit as “on air,” indicating that personnel are using self-contained breathing apparatus (SCBA); this starts a separate timer for tracking air use.
The bottom right corner of the screen contains a sketch board. The user can launch this in a separate window to create tactical diagrams. Figure 3: Sketch Board, below, depicts the sketch board in drawing mode.
1.2.1.1 Benchmark and Scene Detail Functionality
In addition to displaying incident command assignments and unassigned units under the Main tab, two additional tabs are available to display incident information. The Benchmarks tab contains a checklist of incident benchmarks. Clicking a benchmark button marks it as achieved, along with the time at which it was completed. This also creates an incident log entry for that status change. The Scene Details tab contains basic dispatch and record-keeping information about the incident, which also is recorded in the ongoing incident log as the user enters it. Figure 4: Main, Benchmark, and Scene Details Tabs illustrates all three of these tabs in use.
1.2.1.2 Incident Log Functionality
The product’s incident log feature automatically records each change in incident, unit, and benchmark status, as shown below in Figure 5: Incident Log. The user also can manually add log entries. The user can print the log and export it to PDF but cannot edit it.
In addition to this text-based log, the user can use a “snapshot” function to save screen images. Each image shows the state of the UI at the time it was taken. This includes the state of the sketch board described in Section 1.2.1: Core Functionality.
1.2.1.3 PAR and Mayday Functionality
As a personnel accountability tool, the product enables the user to perform and track a PAR. The PAR can be logged by unit or by individual crew member, as shown in Figure 6: PAR by Unit or Crew Member (respectively, left and right sides of the figure).
If a unit or crew member fails to report or declares a mayday, ICx also enables the user to track related actions. Declaring a mayday in the product launches a separate unit assignment area on the right side of the screen that is specific to the mayday, as shown below in Figure 7: Mayday. The unit in distress and the units serving in the Rapid Intervention Team (RIT) assignment can be moved here, with the unit in distress highlighted in red (a color which, in the product’s UI, is reserved solely for mayday functions). Declaring a mayday also saves a screen image.
The product also contains a customizable mayday checklist, as shown below in Figure 8: Mayday Checklist. This checklist functions similarly to the benchmark checklist, displaying the time at which each objective was marked as achieved.
1.2.2 Resource and Roster Management
While it is possible to assign units and personnel to an incident in an ad hoc fashion, ICx enables a fire department to configure units and crews prior to an incident. Resources can be organized by department, station, and battalion; can be classified by apparatus or company type; and can be assigned to an incident that is currently active in the product. Figure 9: Resource Management shows the resource management UI.
A similar interface enables the user to list department personnel in ICx and track their crew or shift assignments. Figure 10: Roster Management illustrates this UI. The vendor states that ICx is designed to integrate with their free FireRoster online roster management service to provide more detailed personnel management functions, but assessment of FireRoster’s capabilities was outside the scope of this evaluation.
1.2.3 Reference and Contact Management
ICx provides additional information management functionality through its reference and contact management features. Its reference functions enable users to store PDF documents and to bookmark Web pages within the application. Resources stored in this fashion can be viewed inside ICx. Viewing bookmarked Web pages does require Internet connectivity.
The product’s reference area is pre-loaded with two hazardous materials (HAZMAT) reference sources: the 2012 edition of the Emergency Response Guidebook (ERG) and a bookmark for WebWiser. Figure 11: ERG illustrates the ERG interface.
ICx also includes a contact management UI for storing information on individuals and organizations that may be critical during an incident.
1.2.4 Configuration Options
The user has several options for customizing ICx’s behavior. Most of this customization centers on the incident type templates which provide default values for specific types of responses. The user can adjust many elements of these templates to match department-specific organizational structures and SOPs. These include:
- When the product launches alert dialogs for specific events (e.g., a unit has been on air for __ minutes);
- Which units are dispatched to the first and second alarms for a given incident type (providing one- button ability to assign multiple units to the incident on the tactical screen);
- Default command staff positions and scene assignments;
- The incident’s benchmark and mayday checklists; and
Elements to be recorded in the scene details list.
1.3 Objectives
The following objective established parameters for this evaluation:
Address the incorporation of NIMS concepts and principles. This includes a determination of how the system applies to the criteria for Emergency Support, Hazards, Preparedness, Communications and Information Management, Resource Management, and Command and Management. General questions on the system, including implementation considerations of the product, were also addressed.
1.4 Test Setup
The test was conducted on-site at IMTEL and remotely. The vendor provided a software license for ICx with authorization to install the product on two computers. Using this license, the test engineer established two product configurations for this evaluation:
- a desktop PC running Windows 7, located in the IMTEL laboratory, and
- a notebook PC running Windows 7, used remotely by the lead assessor. 3
The test engineer also established a screen sharing link via WebEx to enable on-site and remote personnelto coordinate their activities.
1.5 Test Schedule
The vendor provided remote training (presentation, demonstration, and hands on) to participants via WebEx on 06 August 2013. The STEP team conducted the ICx test from 06 through 08 August 2013.
1.6 Scope and Limitations
2.0 Results
Results presented in Section 2.1.1: Objective 1: Evaluate Incorporation of NIMS Concepts and Principles and Appendix A: Detailed Results for NIMS Concepts and Principles are within IMTEL’s ISO/IEC 17025:2005 scope of accreditation. In the event that any individual findings fall outside the scope of accreditation, they will be clearly annotated as such. Other individual findings, observations, and results that fall outside the scope of accreditation are marked with an asterisk (*).
2.1 NIMS Concepts and Principles
2.1.1 Objective 1: Evaluate Incorporation of NIMS Concepts and Principles
Assessors evaluated ICx to determine if the system incorporates NIMS concepts and principles and documented results as identified in the following sections for Objective 1. Refer to Appendix A: Detailed Results for NIMS Concepts and Principles for additional details. The incorporation of NIMS concepts and principles is within IMTEL’s ISO/IEC 17025:2005 scope of accreditation. The ratings for the NIMS concepts and principles are based on the professional opinions of subject matter experts with expertise in this specific subject matter.
ICx is consistent with all NIMS criteria: Emergency Support, Hazards, Preparedness, Communications and Information Management, Resource Management, and Command and Management. The sections below summarize test results for NIMS concepts and principles.
2.1.1.1 Emergency Support
ICx applies to 4 of 15 Emergency Support Functions (ESFs): Firefighting; Emergency Management; Search and Rescue; and Oil and Hazardous Materials Response.
ICx applies to 4 of 9 Incident Command functions: Incident Command; Operations; Planning; and Safety.
2.1.1.2 Hazards
ICx can support responses to natural hazards, human-caused events, and technological-caused events.
2.1.1.3 Preparedness
ICx can be used to effectively support the preparedness activities for planning; procedures and protocols; training and exercises; and evaluation and revision.
2.1.1.4 Communications and Information Management
Common Operating Picture:
ICx provides access to critical information. The system offers an incident overview by collating and gathering information that enables users to make effective decisions and it has the capability to be
updated continually in order to maintain situational awareness. Providing the same information about an incident for on-scene and off-scene personnel is not a function of the version of ICx under evaluation.
Interoperability:
The product runs as a single instance (i.e., one license allows one computer to run one copy of the product, and different instances of the product cannot communicate with each other). Thus, because it does not communicate with other information systems, it is not conducive to the SAFECOM Interoperability Continuum data sharing requirements [Ref 2].
Scalability:
ICx can be used during small- and large-scale events. It is designed to be used on scene. It could be used by an Incident Commander at a staging area, base, or camp if that site were co-located with the incident command post (ICP), such as might occur in a search and rescue or wildland firefighting operation. The product is usable by multiple levels of government and by both the public and private sectors.
ICx is intended for on-scene use by incident command staff. Remote or off-scene use, such as would occur in an EOC or a Federal Joint Field Office (JFO), would be possible with robust two-way communication to ensure the user’s situational awareness of the incident. Because the product is optimized for the fire service’s procedures and likely response scenarios, it is not usable across the full spectrum of multi-agency or multi-discipline incidents and events.
Plain Language:
The system and its end user documentation adhere to the principle of plain language (clear text).
Information Security:
As tested, ICx is not designed with redundancy. It does not include a means to authenticate and certify users for security purposes, nor does it provide controls to restrict access to sensitive information. It does not introduce any unique security or vulnerability concerns.
The product is not designed to store personally-identifiable information (PII). However, through use of free-form text fields, it is possible that some users might record PII in the product. This introduces legal and/or security risks which the organization operating the product must address with access control policies and user account permissions
2.1.1.5 Resource Management
ICx addresses the need to manage resources. The product addresses the use of mutual aid agreements and resources through its ability to track units deployed from multiple departments. It provides a means for performing personnel accountability through its assistance with personnel accountability reports (PARs) and mayday declarations.
2.1.1.6 Command and Management
ICx is consistent with 12 of 14 management characteristics of the Incident Command System (ICS): Common Terminology, Modular Organization, Management by Objectives, Incident Action Planning, Manageable Span of Control, Comprehensive Resource Management, Establishment and Transfer of Command, Chain of Command and Unity of Command, Unified Command, Accountability, Dispatch/Deployment, and Information and Intelligence Management.
As observed in the evaluation, the product does not use standard ICS forms. However, it does support the use of a standard ICS organizational scheme for a response, including the ability to expand the organization chart as an incident increases in size or complexity.
2.1.1.7 Implementation and Product Overview
It should take less than two weeks for a department/agency to implement this system (from acquiring and installing to achieving user proficiency). The system is designed for use in a fireground environment by personnel whose concerns do not include detailed computer system knowledge, and this design objective is reflected in the product’s ease of use. Vendor-provided training is comprehensive and applicable to real-world use of the product.
ICx’s user documentation provides an overview of the product’s functionality by means of a PDF that is launched within Adobe Reader. The documentation is organized in parallel with the product’s user interface architecture.
Customer support was available via email and phone during the business hours in which the STEP assessors conducted the evaluation. The vendor states that email contacts will be answered within 24 hours, though most often “within minutes” of receipt.
The product was reliable during the evaluation.
2.2 Additional Observations*
The results in this section are observations made by SMEs during the evaluation. Such observations were not used in determination of any test results and/or ratings in this report and are provided for informational purposes only.
System Capabilities
The product is designed with a limited scope: fire and emergency service response operations. While it could be adapted for use with other ESFs, much of its functionality is specific to the industry standards and best practices of the fire service, and wider use is outside its design intent.
Preparedness
The product’s primary use is during an incident’s response phase. It is not a purpose-built tool for authoring or documenting procedures and policies. However, it is flexible enough that an individual agency can adapt much of its function to that agency’s SOPs for different incident types.
Communications and Information Management
- As tested, the product operates in a single instance (i.e., one license allows one computer to run one copy of the product, and different instances of the product cannot communicate with each other). The vendor states that future versions of the product will be able to share data across multiple instances, providing real-time situational awareness for all users.
- As tested, the product does not include geospatial functions or geospatial data. The vendor states that future versions of the product will be able to interface with computer-aided dispatch (CAD) systems.
- Securing access to the product is the responsibility of the user and must be done through external measures (e.g., physical control of access to the computer running the product, operating system user ID/password schemes, etc.).
- The product is designed for on-scene use by an IC or another command staff member. Remote use would depend on accurate and reliable communications with units in the field and could introduce span of control issues.Resource Management
The product can be configured to reflect multiple agencies’ resources, though actual tasking of these resources to incidents is a function of mutual aid agreements, not the product itself.
- American Association for Laboratory Accreditation. http://www.a2la.org/
- Fiscal Year 2012 SAFECOM Guidance on Emergency Communications Grants. U.S. Department of Homeland Security, Office of Emergency Communications.http://www.safecomprogram.gov/SiteCollectionDocuments/2012_SAFECOM%20Guidance_FIN AL.pdf
- National Incident Management System. U.S. Department of Homeland Security, December 2008.http://www.fema.gov/pdf/emergency/nims/NIMS_core.pdf
- National Response Framework. U.S. Department of Homeland Security, January 2008.http://www.fema.gov/pdf/emergency/nrf/nrf-core.pdf
- NIMS Recommended Standard List (RSL). Federal Emergency Management Agency, January 2009. http://www.fema.gov/pdf/emergency/nims/FY09_Recommend_Standards_List_121708.pdf
- Supporting Technology Evaluation Project (STEP) Guide. Preparedness-Technology, Analysis, and Coordination Center, May 2013. https://www.ptaccenter.org/static/files/STEP-Guide.pdf
STAY IN THE LOOP
