Design

This was a population-based intervention study, using de-identified record-linkage health data in Western Australia between 2002 and 2013.

Participants and data source

The study utilised the WA Emergency Department Data Collection (EDDC) and the WA Hospital Morbidity Data Collection (HMDC), of which WA Department of Health is the custodian. The EDDC contained all 3,219,905 ED presentations, corresponding to 1,206,466 patients, to the five-participating hospital EDs between 1 January 2002 and 31 December 2013, except for Hospital E whose data were included from 1 July 2004 (when this hospital moved to the same ED information system as the other four hospitals). The HMDC was used for identifying hospital admissions following ED presentations (see Data definitions below).

Outcome measures

Ten outcome measures (five principal and five secondary) are assessed in this study. The five principal measures are: Access Block (defined as ED length of stay (EDLOS) longer than 8 hours) for an admitted patient, ED occupancy rate, ED attendances and re-attendance (within 7 days of index ED discharge) and ED length of stay (EDLOS). These are designated ‘principal’ as they are considered as the direct markers of the intended outcomes or changes expected of the FHR. Secondary measures include: time to be seen by ED clinician, ED efficiency (indicated by the percentage of ED attendances with EDLOS ≤ 4 hours), Did-Not-Wait (DNW–left ED before being seen by an ED clinician), admission, and short-stay admission (i.e., with discharge within 24 hours).

Data definitions

Identification of a hospital admission subsequent to an ED attendance was via match-merging records from the EDDC and HMDC. Records from the HMDC would need to have their status flagged as ‘emergency’ admissions to be considered in the match-merging process with EDDC records. The timeframe that allowed for an ED-Morbidity match was ED discharge time being within [-2 hours, +1 hour] of the subsequent hospital admission time. The absolute value of 1-hour buffer time was included to take into account the potential inaccuracy/imprecision of the time recorded in the HMDC; as the data custodian recommended these times be rounded up to the hour unit. One extra hour was allowed after ED discharge time and before hospital admission time, taking into account logistic reasons, such as time taken for finalizing hospital paperwork and/or transfer to another hospital for admission.

ED occupancy rate served as an index of ED overcrowding, whereby ED occupancy rate greater than 100% indicates an overcrowded ED environment. It was calculated for every one-hour block throughout the study period for each hospital ED, and defined as the total ED patient count divided by the concerned ED’s functional capacity. Functional capacity are numerical values agreed by the Western Australian Department of Health, the ED concerned and the respective hospital, as the volume of patients that the ED can functionally receive and treat at a given point in time.

Data analysis

Data were first summarised in quarters and presented graphically as time series for the 10 outcomes separately. The impact of the FHR on these ED outcome measures was assessed statistically using Interrupted Time Series (ITS) analysis, which is the recommended technique for analysing data from a population-based interventional study such as this one [22–24]. Several assumptions of the data need to hold for the ITS analysis to be applied correctly [25]. Among these, the following four key assumptions were checked and satisfied for our data. First, the outcome data (i.e., dependent variable) are of continuous or discrete (e.g., count) type. Second, the relationship between the outcome/dependent variable and the explanatory/independent variables (usually related to the intervention and its timeframe) is linear, or can be transformed to be so. Third, ideally there needs to be at least 12 data (time) points before intervention and 12 points after intervention, to ensure a reasonable degree of accuracy and reliability in trend detection and estimation. Fourth, within each data point, a minimum of 100 observations is required, to ensure “an acceptable level of variability of the estimate” at each point [25].

In the ITS analysis, the pre-FHR period was comprised of 29 Quarters (14 January 2002 to 13 April 2009 inclusive) for the four Stage 1 hospitals, and 21 Quarters (26 July 2004 to 25 October 2009 inclusive) for the Stage 2 hospital. Similarly, the post-FHR period spanned over 18 Quarters (14 April 2009 to 13 October 2013 inclusive) for the four Stage 1 hospitals, and 16 Quarters (26 October 2009–25 October 2013) for the Stage 2 hospital. Results from these analyses are reported in relation to: (i) trend pre-intervention, (ii) immediate change in levels from pre-intervention to post-intervention, and (iii) change in trend post-intervention.

Data on Access Block were log-transformed before being subjected to the ITS analysis, and then back-transformed (i.e., exponentiated) once modelled, to ensure the predicted values derived from the statistical models were non-negative. This consideration was not required for the ITS analysis of the other nine outcomes, where data in (original) normal base were used. Accordingly, estimates derived from the ITS models are to be interpreted as “differences” for these nine outcomes, and as “ratios” for Access Block.

Inferential statistics presented in this paper are accompanied by the respective 95% Confidence Intervals (CI). Statistical significance levels are not adjusted for multiple outcome analyses in this study, because such adjustments are argued by some as unnecessary and potentially misleading [26, 27]. However, where effects are found to be of strong statistical significance (p<0.001), these are also denoted accordingly in the presentation of the results.

Data were managed and analysed using SAS software version 9.3.

Ethics clearance

The study received ethics approval from the respective Human Research Ethics Committees of Western Australian Department of Health, University of New South Wales, and The University of Western Australia, as well as governance approval from all participating hospitals.

Patient (and system) characteristics

Checks for data completeness and validity led to the removal of 5,103 ineligible records (0.16% of all available records). Of these, 4,568 (89.5%) were coded as ‘Dead on Arrival’, while the remaining 535 records had missing, duplicated, or invalid time measures (e.g., discharge time recorded as before presentation time). Accordingly, a total of 3,214,802 ED presentation records, corresponding to 1,203,513 persons, were deemed eligible and retained for reporting in this study.

Table 1 describes the socio-demographic, clinical and environmental characteristics of all eligible ED attendances during the study period 2002–2013. Gender distribution was similar across all five hospitals, with a slightly higher percentage of males. Over 90% of the ED attendances were by residents of the Perth metropolitan area, where the participating hospitals were located. Distribution of socio-economic disadvantage varied somewhat with the hospital locations. The clinical profile was mostly similar for all hospitals, with up to one-third of the presentations being due to injuries, and approximately one-third of the presentations using ambulance transport. Triage categories were distributed similarly for most hospitals, with Hospitals C and E appearing to have lower proportions of high-acuity cases. Proportions of attendances during ‘Day shift’ in general were higher than ‘Evening’ or ‘Night shifts’. Distributions of attendances across the other three time-related factors (influenza season, weekend, and holiday) were similar for the five hospitals.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Socio-demographic, clinical and environmental characteristics of ED patients and presentations by hospital.

https://doi.org/10.1371/journal.pone.0193902.t001

Table 2 provides a summary of results from the ITS analyses. These are further described below, in conjunction with Figs 1–5. Supporting information and a brief description of the ITS technique, modelling strategy and interpretation of results are presented below [26NR, 27] (S1 File).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1.

ITS analysis for rate of ED attendance across WA hospitals (A to E). Estimated trend lines for Without FHR (broken line) and With FHR (solid line). Series with markers represent actual data.

https://doi.org/10.1371/journal.pone.0193902.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Results of ITS analysis for rate of ED re-attendance across hospitals: Estimated trend lines for Without FHR (broken line) and with FHR (solid line).

Series with markers represent actual data.

https://doi.org/10.1371/journal.pone.0193902.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. ITS analysis for median length of stay in ED (measured in hours): Estimated trend lines for Without FHR (broken line) and with FHR (solid line).

Series with markers represent actual data.

https://doi.org/10.1371/journal.pone.0193902.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Results of ITS analysis for mean ED occupancy rate: Estimated trend lines for Without FHR (broken line) and with FHR (solid line).

Series with markers represent actual data.

https://doi.org/10.1371/journal.pone.0193902.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Results of ITS analysis for rate of access block: Estimated trend lines for Without FHR (broken line) and with FHR (solid line).

Series with markers represent actual data.

https://doi.org/10.1371/journal.pone.0193902.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Summary results per hospital of attendance rates, time in ED, flow and disposition.

https://doi.org/10.1371/journal.pone.0193902.t002

Principal outcomes

Secondary measures

Principal findings

Collectively speaking, the FHR produced improvements in several areas of ED functioning. These include: shortening ED length of stay and enhancing ED efficiency for most hospitals; alleviating access block and ED overcrowding dramatically for all hospitals; reducing time to be seen by ED clinicians; and reducing rates of ED re-attendance and DNW in some hospitals.

The most consistent and substantial benefits of the FHR program were observed in the ‘flow’ measures, especially access block. This was expected, given that the primary objective of the program was to improve ED and hospital efficiency and reduce congestion and patients waiting. Also notable was the stable, or even downward, trend in ED re-attendance within 7 days. This provides some indication that improvements in flow were not made at the expense of patient care, a concern noted from the UK FHR experience [28]. Nevertheless, we also acknowledge that other, more pertinent measures of unintended adverse patient outcomes (e.g., prolonged total hospital stay, mortality) were not explored in this paper.

Conversely, it is also interesting to note that, while the FHR could be seen as a time-based target, its effect on the two time-based outcome measures, namely ED length of stay and time to be seen by ED clinician, were less uniform than that on the ‘flow’ measures. Broken down by acuity (ATS) categories, the data showed that: ED length of stay for lower acuity (ATS 4 and 5) attendances were already low, and hence remained unchanged with the FHR. The majority of improvement in ED length of stay was seen in the higher acuity levels (ATS 1 through 3) attendances (S1 Fig). By contrast, time to be seen by ED clinician was already short for the highest acuity (ATS 1 and 2) attendances, and hence exhibited no apparent improvement in association with the FHR (S2 Fig). This is to be expected as clinical resources are directed to higher priority patients. The same preliminary analysis (S2 Fig) also suggested that time to initial ED assessment for lower acuity (ATS 4 and 5) attendances may have improved with the FHR at the expense of moderate acuity (ATS 3) attendances for some hospitals. These observations warrant a specific focus of inquiry.

Admission rates increased for most hospitals in post-FHR and in one hospital this trend was accompanied by an increase in short-stay admissions. There had been some clinical concern, documented in the international literature (and commented locally) that efforts to meet the FHR target could lead to a somewhat ‘strategic’ increase in admissions, particularly admissions for low-severity ED attendances, in order to ‘stop the clock’[29]. This would result in a ‘dilution’ effect in admission practice. However, preliminary investigation of our admission data (S3 Fig) based on the ATS categories, provides no basis for this concern. In fact, there seemed to be an increased admission of higher acuity (ATS categories 1–3) cases.

Rate of ED attendance on the whole was already increasing over time, independently of the FHR, except for a significant upward trend observed post-FHR for Hospital A, whilst it’s pre-FHR trend was stable, unlike most other participating hospitals. It is interesting and worth noting that the attendance rate did not increase uniformly as a result of the FHR and its publicity. Similarly, there was no evidence for the ‘dilution’ effect [29] in the acuity of ED attendances due to the FHR (S4 Fig).

Strengths and weaknesses of the study

Despite over a decade since the UK FHR was introduced, and several similar interventions being introduced worldwide to alleviate the overcrowding and related issues faced by EDs, little systematic research has been published in the peer-reviewed literature [30]. Also previous studies were not focused on a comprehensive range of ED performance indicators like ours, and were limited to fewer selected outcomes, such as ED length of stay [31, 32], DNW rate [33], overcrowding and access block [34] and mortality [21, 31, 32, 34, 35]. Further buttressing this strength, our study presents the longest longitudinal series of data in the field (12 years), with data from five sites with different profiles. Our study applied the ITS statistical technique to more accurately evaluate changes in the outcome measures that are attributable to the FHR.

Another strength of our study is the large data coverage, which has allowed us to reliably measure ED occupancy over time for each participating hospital, hence providing an accurate estimate of the extent of ED overcrowding throughout the long study period. ED overcrowding represents a key aspect of the ED and hospital environment, which should be of significant value in subsequent related studies in assessing specific patient outcomes of interest, such as: “Does presenting to an overcrowded ED and hospital environment put the patient at significantly higher risk of mortality?” Some previous studies applied a perhaps less informative proxy measure for ED overcrowding (e.g., via access block [34, 36]); while others attempted to measure the construct using a more original method.[4, 5] Nonetheless, to our knowledge, none of these estimates has the combined strength of accuracy and durability as demonstrated in our study.

These strengths are juxtaposed by two shortcomings. First, while reporting on an extensive range of ten outcome measures related to ED and hospital functioning, we needed to restrict our scope of analysis to time trends and intervention effects only (and where particularly relevant, triage categories), in order to fit into the scope of one paper. As such, examination of factors considered important such as patient age, comorbidity, mode of transport, socio-economic disadvantage, as well as system and environmental factors such as ED shift, ED environment (e.g., extent of overcrowding) and seasonal factors, and their interplay with the intervention effect, have not been included here. Second, with WA being the first Australian state to implement the FHR–hence acquiring a lengthy series of longitudinal data, it is not currently possible to include a similarly long series of data from hospitals in other Australian jurisdictions. Consequently, with the five sites under investigation, the level of homogeneity may not have been ideal, which is why we presented the results for the five sites separately. Hence, the generalisability of our findings may be limited [21].

Another limitation in this study is its primary focus on the intervention’s impact on ED performance indicators, and this focus is not extended to ‘downstream’ effects on specific inpatients’ outcomes, except for the study’s examination of admission and access block measures, which concern inpatients. On this point, we also note some recent literature examining such issues. For example, Perera and colleagues [37, 38] reported positive and negative outcomes such as reduction in ED waiting times and increased use of short stay units as well as prolonged inpatient stays and an increased rate in inter-unit transfers for certain subgroups of inpatients (e.g., those utilizing acute surgical services) after the FHR intervention. Our findings are consistent with these studies; however, we have not reported on specific conditions as it was beyond the scope of the study.

Explanations and implications of the study’s findings

Despite between-hospital variations in certain outcomes as noted earlier, the FHR experience in WA has overall been positive for the observed measures. This reflects the potential value of investing resource and effort into the hospital system resulting in improved functionality, which improves the patient experience (at least in the ED environment). Although mortality outcome has not been assessed in this report, we have addressed some important clinical questions, including whether the FHR (i) brought about unintended adverse, or suboptimal consequences such as increased ED re-attendance; (ii) was associated with a dilution effect in acuity of ED presentations; (iii) ‘bed-shuffling’ of patients just to ‘stop the clock’ and meet the time-based target by increasing admissions, and particularly short-stay admissions. As summarised in Results and discussed in the Principal Findings section earlier, we found no clear evidence from our study to serve as a basis for any of these concerns, which is positive.

Our results also suggest that the FHR in WA was not regarded simply as a time-based target by the participating WA hospitals. Rather, it was a major driver or catalyst for system and practice changes on many levels. For example, for some hospitals, despite demonstrating modest changes in their median EDLOS, their other ED experiences such as access block and ED overcrowding, still benefited from adopting the intervention. Changes and improvements, to some extent, were different for different participating hospitals, depending on their inherent structure and setup (e.g. public or private funding, tertiary or non-tertiary, treating adults or children or mixed) and context (e.g. city centre or suburban, patient demography and socio-economic disadvantage), to name a few. Accordingly, these factors need to be taken into account when attempting to translate this study’s findings into meaningful changes to policies and practices for other hospitals.

Future research

We are exploring the exact mechanisms of the introduction and effects of the FHR on the ED and hospital functioning such as mortality, comorbidities, and other variables described above, as well as the national impact. This is important mainly for two reasons: (a) to assess further the question of any FHR unintended adverse consequences, and (b) to compare with preliminary national and international findings [1, 21, 30, 39]. To meaningfully translate our research findings to clinical and administrative changes, we consider that simulation modelling will be a useful tool.