Organizational Assessment in Long-Term Care: The Assessment of Leadership and Risk Management (ALARM) Framework

Organizational assessment (OA) is a diagnostic process undertaken to evaluate work structure, environment, and processes of an organization or a subcomponent.¹ OA can be employed in a variety of different settings and has been use in nonprofit, health care, law enforcement, education, corporate business, and manufacturing sectors.^2-9 The purpose of performing an OA includes revealing the organization’s current state of operations, identifying the organization’s future state based on industry trends, determining the gap between the current and future state, exploring internal issues, and recognizing priorities for goal development and business enhancement.⁴ Information obtained from an OA must be meaningful as the results are used to develop action plans for organizational improvement.⁹ An OA can be employed within various organizational levels, from small departmental teams to international sites of business operations.¹⁰ Approaches to completing an OA include self-assessment, use of a consultant or expert third-party, and peer review.⁹ The OA process can also take the form of a checklist completed by leaders, participants, and stakeholders of an organization or be conducted through an established, comprehensive tool.⁷ Examples of organizational components that often comprise an OA include leadership, strategy, customer, workforce culture, work processes and operations, and performance metrics.⁹

Health care organizations have a long-standing relationship with structured OA. Accrediting and government agencies audit health care organizations, including hospitals, out-patient facilities, and long-term care (LTC) facilities for compliance with federal and state regulations. Additionally, health care organizations may undertake an internal OA to review programs of care, departmental operations, and workplace culture to determine organizational position anticipating future changes in the health care industry.^6,11 Health care organizations have an established trusting relationship with their constituents, invoking a social responsibility to meet the needs of their customers while providing the highest quality care. OA has been used to determine the status of customer satisfaction and identify care processes for improvement in health services and other industries.^5,6,8,12

As the American population continues to age, developing a structured OA for LTC that addresses qualified leadership and controllable risk factors impacting service delivery is fundamental in meeting the health care needs of the US.¹³ Literature has been published on the use of OA within hospitals in both the US and Europe; however, an extensive literature search of keywords related to OA, quality, performance, and health revealed a void for OA specific to LTC. Health care operations are required to comply with state, local, and federal regulations. LTC facilities that receive reimbursement from Medicare or Medicaid, or both, are required to have annual inspections by their respective state health departments. For annual recertification to be granted and payment to continue from governmental agencies, LTC facilities must meet state licensing standards and federal regulations. LTC facilities are specifically required to conduct and document an annual assessment to determine whether the necessary resources are available to care for residents at all times, and to modify the assessment as needed based on facility changes that may occur.¹⁴

Facilities often complete an annual OA to meet regulatory requirements, limiting the purpose of the assessment to adhering to compliance standards. Currently, many LTC facilities use the US Centers for Medicare & Medicaid (CMS) sample template¹⁵ or a nonstandardized template for facility assessment that includes a list of the regulatory standards and short summary of how each regulatory standard is met. Such templates may list services provided by the facility while also quantifying available resources, such as staffing, emergency medical supplies, food, and fuel.

While it is necessary to update such informational compendiums, this annual task neither assists management in determining how well the facility is performing, nor identifies areas in need of quality and service improvement. Thus, we developed an assessment framework that evaluates two key components of facility operations—leaders and operational risk factors—to complement the CMS prescribed assessment.¹⁵

The impetus for developing this resource stems from the lack of knowledge and tools available for LTC facilities to conduct a comprehensive OA leading to sustainable performance improvement through focused action planning. An extensive literature search revealed a gap with regard to OA in LTC settings. Keywords, including organizational assessment, performance assessment, and leadership and risk management tools in long-term care, revealed very few results, and none with the desired frameworks to assist operational leaders. OA has been widely studied in industrial sectors and acute care hospitals, however, research is lacking to aid LTC administration and leadership.

In any sector, leaders at all levels provide the framework for organizational culture. Consequently, it is the responsibility of an organization’s administration to assess and provide leadership that adopts high reliability principles and contributes to quality outcomes. In classic LTC administrative hierarchy, the director of nursing (DON) and administrator are responsible for leadership assessment and must evaluate each of their subordinates at regular intervals. The leadership assessment process is critical for novice administrators to master, as well as experienced administrators who are assuming a position within a new facility or organization.

With the ongoing mass-exodus of experienced administrators and leaders from LTC, exacerbated by the current health care crisis, it has become crucial for novice management to quickly learn the OA process so areas of concern impacting quality and safety can be immediately addressed.^16,17 The LTC labor supply has changed significantly as the COVID-19 pandemic has accelerated the departures of experienced DONs and administrators. While multifactorial, this departure is due in part to the stresses and liabilities associated with these positions amid increasing regulatory demands. In a recent survey of LTC administrators, “48% of respondents indicated that one of the biggest reasons why administrator retention has decreased is due to burnout related to the pandemic and the current operating challenges.”¹⁸ Shortages of clinical staff, namely nurses and certified nursing assistants, has further compounded operational challenges. The reimbursement gap between LTC and other health care settings continues to accelerate the transition of staff from LTC to higher paying venues.¹⁷

Experienced LTC administrative professionals intuitively perform assessments of their organization’s processes, systems, and subordinates, often without documenting their findings and results. For organizational success, data from assessment must be captured and used in the form of an action plan. Additionally, current literature is silent on an OA tool that is specific for LTC; thus, a standardized framework is necessary to guide LTC administrative leaders on such endeavors.

Regulations outlined in §483.70 of Title 42 of the Code of Federal Regulations (42CFR) require an annual facility assessment be performed.¹⁴ The sample assessment for compliance with 42CFR has not been used as a true assessment but simply as an inventory of facility capabilities and supplies. According to Wiener,¹⁹ “despite [the Omnibus Budget Reconciliation Act of 1987], federal and state regulations still emphasize inputs, manuals, paperwork, and structural capacity rather than resident outcomes.” CMS’s current facility assessment sample tool does not include a complete review of key areas critical to facility performance and resident outcomes. The authors propose a more structured assessment for LTC OA through use of the Assessment of Leadership and Risk Management (ALARM) framework.

ALARM consists of two sections: leadership assessment and organizationally recognized high-risk areas. The leadership assessment uses elements of the Organizing for Quality Framework (OQ),²⁰ developed through a study of high-performing American and European hospitals. In the framework, Bate, Mendel, and Robert identify specific factors that made these organizations successful in delivering quality care. Through a multifaceted approach employing case studies, observation, and interviews, the research team identified six, inter-related, core challenge areas as key for quality and service improvement in hospitals.²⁰ The lack of process in each of the six challenge areas led to organizational problems in quality and service delivery. The OQ framework was chosen as the basis for our framework due to the similarities in operational challenges faced in both acute and LTC settings.

The challenge areas outlined in the OQ framework include structural, political, cultural, educational, emotional, and physical/technological. Rather than assessing the organization as a whole, ALARM uses indicators from each OQ challenge area to assess LTC leaders. The tool focuses not only on the leader’s specific tasks, but also on his or her so-called soft skills; interpersonal skills such as listening and providing feedback, as well as time management and the ability to work as part of a team are some of the key soft skills that effective leaders possess The tool reviews each leader’s ability to manage, motivate, serve as an agent for change, interact with colleagues and subordinates, and promote workplace culture, efficiency, and innovation.

The second arm of the ALARM framework, located in Appendix B, is a risk management tool. This companion assessment, to be used in conjunction with the leadership assessment, reviews compliance in key risk areas, both clinical and nonclinical. Safety, security, and preparedness are central topics to managing risk. Leaders must be mindful of potential risks to prevent or mitigate their impact. The framework provides cueing in common risk areas for consideration during facility rounds, policy review, or staff meetings. To illustrate, the Daily Operations/Regulatory Compliance section of the framework includes items that leaders must continuously monitor, such as infection control practices, reportable events, and federally mandated meetings (ie, resident council). Upon completion, the results of this evaluation are used to help existing and new leaders prioritize areas for improvement to increase quality of care.

As identified by Wiig and colleagues,²¹ “there is a lack of effective leadership tools for quality and safety improvement work,” in LTC. Using the OQ framework areas in developing an LTC assessment tool is a logical extension. As LTC residents often reside in these facilities for lengthy periods, establishing sustainable service improvement is essential for satisfaction, safety, and ultimately, high quality care. Sustainable service improvement must start with processes driven by people in management positions who are empowered to act as agents of change. Thus, the framework was designed to embed leadership criteria from each of the six core challenge areas to easily evaluate leader performance. The ALARM framework offers a clinical and nonclinical risk factor assessment to be used as a review mechanism of potentially deficient areas. These areas may be traced back to the leadership of specific department areas or be used across several domains.

Similar to the survey process that administrators and DONs are familiar with, the ALARM framework provides an organized approach to assessing performance and identifying and prioritizing deficient areas requiring action planning to correct identified problem areas. As with any assessment framework, ALARM does not encompass every leadership component or risk factor facing an organization. It does, however, cast a wide enough net to provide a basis for sound and supported decision making. Upon completion of both assessments, decisions regarding key leaders can be made in an organized and comparative manner using a systematic framework. New leaders joining the organization will have an advantage upon hire to focus on areas already identified for improvement efforts. The following exemplars based on our experiences illustrate specific uses in which ALARM has proven beneficial.

An experienced administrator recently took a new position in a postacute care facility and was first tasked to evaluate the DON. While qualified on paper, even a brief preliminary review of the DON’s performance raised serious concern. It quickly became apparent the DON was not performing key duties and operations. Furthermore, clinical nursing for patients was not at the level of quality required. Amid the COVID-19 pandemic, basic infection control policies, procedures, and visible practices were not being enforced. Inconsistencies in safety and incident reporting, staff education, recordkeeping, departmental communication meetings, handoff procedures between shifts, and follow-up with resident and family concerns were just a few of the glaring deficits identified. Requests for policies and operational follow-ups from the new administrator were ignored, and the DON avoided any personal interaction. Additionally, other directors expressed concerns regarding the DON, citing her unwillingness to work on interdisciplinary projects and lack of follow-up with operational issues.

Soon after taking the new role, a staff nurse made a significant medication error but reported it to the DON. This same staff nurse approached the administrator a week later to inquire about the outcome of the safety report she had filed. During this time, the DON had not informed the administrator or other team members of the incident. She also failed to complete an investigation, follow-up, or required in-service education with those involved as organizational policy outlined. This incident, coupled with other documented situations, attested the need for a change in nursing leadership. During a performance conversation in which termination was planned, the DON resigned, attributing her departure to “the new atmosphere” being created. Undoubtedly, she was correct; as a new administrator, it was essential to establish an interdisciplinary culture of accountability, safety, and collaboration. Although newly appointed, the experienced LTC administrators intuitively assessed the DON’s performance without an associated framework to support and document the process, findings, and decisions. Yet, in the future, this decision could be enhanced with a framework. Thus, leadership assessment, part of the ALARM framework, shown in Appendix A was developed and used to objectively examine the remaining leaders and resulted in replacing 10 of 12 positions. As a result of using an OA framework, such as the leadership assessment, this particular facility now has a leadership team that is high-performing and earns awards for quality, including recognition among the highest-performing facilities locally and nationally.

Assuming a nursing leadership role, in a new environment, is never easy. An experienced leader may have an advantage over a novice, but regardless of length of experience, setting foot in a new environment is analogous to docking a ship in a new port. Successful docking takes careful planning, and thorough environmental assessment, including who and what lies beyond the port.

Upon starting the DON position at a new facility, particularly during times of immense strain such as the height of the COVID-19 pandemic, experienced leaders instinctively know what to do first. Experience dictates that to prioritize safety, especially in light of a pandemic, one must perform an assessment of the facility’s infection control components. Such assessment would include the current infection rate and status, a cohort plan, nursing leadership knowledge, existing policies and procedures, and overall culture of the nursing department.

An extensive review and update of policies and procedures, with special focus on the pandemic plan, was completed. Review revealed that clinical staff lacked understanding of COVID-19 virus transmission modes. This critical knowledge deficit negatively impacted residential cohort plans implemented by the previous DON; faulty decisions resulted in non-favorable clinical outcomes, namely spread of COVID-19 and resident morbidity and mortality.

Recognizing the importance of knowledge and skills among clinicians, an education and training program was launched for core clinical leaders, including the Infection Preventionist, Staff Development Coordinator, and Nursing Supervisors. Line staff were trained in proper infection control protocols including handwashing, isolation precautions, and donning/doffing of personal protective equipment.

Collaborative actions were subsequently taken with other department leaders in Maintenance, Housekeeping, and Dietary. Additionally, steps were taken to ensure that the physical structure of the nursing units supported appropriate resident cohorting, provided proper clinical and nonclinical space (eg, break rooms), family visitation space with appropriate ventilation, and created other clinical rooms (eg, testing and screening areas) that maximized safety. A review of housekeeping chemicals, application and contact times, path of linen/tray transport, and waste disposal lead to modifications in those processes.

Upon completion, it was realized that an organizational and clinical leadership assessment had been performed intuitively, with no clearly established guideline or assistive tool. All actions undertaken were purely based on instinct, clinical acumen and experience, but with nothing to show a systematic evaluation process.

Actions taken to address the identified clinical shortcomings and practices that were noncompliant with CMS and Department of Health mandated standards had a positive impact, including preventing illness and saving lives. The facility previously experienced high morbidity and mortality rates. Upon implementation of the revised pandemic plan, no facility-acquired infections resulted in death and only a few infections were facility-acquired. This experience prompted administration and nursing to develop the ALARM tools to evaluate additional managers and high-risk, problem-prone areas. Our goal was to create a standardized and highly reliable method on which to base decisions.

As cited in the exemplars presented, LTC administrators and DONs would benefit from an accepted framework rooted in quality and service improvement. The experienced administrator evaluates key areas albeit without a standardized approach to assess key performance domains. The ALARM framework serves to guide operational decisions and enhance the CMS assessment template. It not only fulfills the federal mandate, but also serves as a basis for objective action planning. Using a standardized assessment for leadership personnel decisions also serves as a risk management tool. ALARM may be customized to individual facilities through addition of criteria within the respective categories of the leadership and risk management frameworks. Through repeated use of ALARM, individual organizations may opt to add criteria within the six core areas of the leadership framework and facility-specific, problem-prone areas in the risk management framework. Successful OA yields specific actionable recommendations for improvement and organizational investment that management is willing and empowered to implement.^2,7

Dawn A. Giakas, DBA, MPH, BS, LNHA, CALA, FABC¹ • Courtney Mulder, BSN, RN, TCRN, CEN¹ • Agustin Guido III, BSN, RN, CDONA, CDP, CFPS¹ • Jacob Bailey, BSN, RN¹ • Todd E. Tussing, DNP, RN, CENP, NEA-BC¹ 

Affiliations:
¹Department of Nursing, The Ohio State University, Columbus, OH 43210 

Disclosures:
The authors report no relevant financial relationships.

Address correspondence to:
Dawn A. Giakis DBA, MPH, BS, LNHA, CALA, FABC
College of Nursing, The Ohio State University 1585 Neil Avenue Columbus, Ohio, USA, 43210
Phone: 732-289-1970
Email: Giakas.2@osu.edu