The Business Case for Pediatric Asthma Quality Improvement
The Business Case for Pediatric Asthma Quality Improvement
The evaluation was designed as an intent-to-treat, randomized, controlled trial with randomization at the practice level. The Monroe Plan identified 25 eligible primary care practices in Rochester, New York and the surrounding area, defined as those that served at least 20 Monroe Plan-enrolled children with asthma. Practices were first stratified by three characteristics: size, measured by the number of eligible Monroe Plan members; Federally Qualified Health Center status; and single versus multiple physicians for small practices. Practice size (small, medium and large) was determined by the number of Monroe Plan-enrolled children with asthma (fewer than 100, between 100 and 200 and > 200). The number of eligible members was used to measure practice size, because the incentive payment was tied to eligible members and not to the overall size of the practice based on total visits. Practices were randomized to either treatment (13 practices) or control arms (12 practices), and treatment group practices were invited to participate in the PACE intervention. Participation was voluntary and two treatment group practices declined to participate in the intervention; however, Monroe Plan expenditures on services provided by the practices were included in the evaluation (intent-to-treat approach) nonetheless.
With the dual goals of reducing unnecessary, high-cost utilization for children with asthma and improving outcomes, in November 2008, the Monroe Plan implemented the PACE project. The Monroe Plan offered the PACE intervention to all practices randomized to the treatment group. The intervention period lasted slightly >3 years, although the evaluation period was limited to two and a half years due to data availability. The centerpiece of the intervention was a $5 per month incentive fee paid to each treatment practice for each eligible Monroe Plan member assigned to the practice. All children with asthma that met inclusion criteria were considered eligible; not only those for which a chart audit was performed. In exchange for the incentive, practices conducted chart audits every 6 months on a sample of their eligible Monroe Plan-enrolled children with asthma. The sample sizes were determined by the Monroe Plan, but the individual patient charts were selected by the practices at random. Treatment practices with fewer than 60 eligible Monroe Plan members were required to complete chart audits for 20 eligible children; those with 60–400 eligible members were required to conduct chart audits for 28% of eligible children; and those with >400 eligible members were required to complete chart audits for 100 eligible children. These requirements were established by the Monroe Plan based on findings from a previous chart audit intervention with physician practices.
Practices were required to electronically report chart audit results to the Monroe Plan using an on-line tool. Practices in the treatment arm received training on the use of the online chart audit reporting tool and direct feedback on chart audit results compared with peer practices. Feedback was provided after each round of chart audits (seven rounds in total) through in-person meetings between the practice physician leading the chart audit activity and the medical director of the Monroe Plan. In addition, treatment practices participated in twice yearly asthma learning collaborative meetings. These meetings were structured as hour-long lunchtime meetings during which invited speakers presented on asthma care topics such as the definition and appropriate use of asthma action plans and monitoring of asthma symptoms. Practices were also given time to interact with each other so that they could discuss the intervention, best practices and challenges. The goal of the meetings was to improve providers' understanding of asthma care guidelines and to give providers strategies for improving the frequency with which they provided guideline-concordant asthma care. Control practices did not receive the incentive fee, were not asked to conduct chart audits and were not offered performance feedback or participation in the learning collaborative meetings.
The study population consisted of all children in treatment and control practices that met eligibility criteria. To be included, children needed to meet the following requirements: (i) be a Monroe Plan enrollee, though not necessarily continuously enrolled, during the study period (1 January 2008 to 30 June 2011 which included one baseline year and 2.5 intervention years), (ii) have an asthma diagnosis (defined by an ICD-9 code of 493.xx in any position on a medical claim in the year before or during the intervention period), (iii) be older than 2 but younger than 19 at study entry and (iv) be affiliated with a treatment or control practice as determined by the Monroe Plan primary care provider designation. Children aged 2 and younger were excluded because of the challenges associated with firmly diagnosing asthma in very young children. A total of 7731 children met eligibility criteria.
Data for the study were obtained from two sources. Costs of investing in and operating PACE were collected directly from the Monroe Plan and included costs for personnel; contracted services; office, travel and training; equipment; incentive payments to practices; and indirect (overhead) costs. Costs were recorded in the period in which they were incurred. Evaluation-related costs were also collected but were separately identified since these costs would not have been incurred in the absence of the PACE evaluation (evaluation-related costs not shown).
Fee-for-service payments made by the Monroe Plan to primary care providers were collected from Monroe Plan claims data for a baseline period (1 January 2008–31 December 2008) and the two and half year intervention period (1 January 2009–30 June 2011). Fee-for-service payments were aggregated to the practice level and converted to annual per member per month expenditures on the following sources of care: inpatient; outpatient; office; ED; pharmacy; ambulance, home health, capitation and other; and expenditures on all sources of care.
We conducted weighted difference-in-differences regression analyses to assess whether there were statistically significant differences in per member per month payments in treatment practices relative to control practices following the intervention. Analytic weights were defined as the number of member months contributed by each practice. The unit of analysis was the practice. We analyzed total expenditures to assess whether the intervention was successful, in aggregate, at reducing spending by the Monroe plan. We also ran separate analyses for expenditures on each of the six sources of care (inpatient; outpatient; office; ED; pharmacy; ambulance, home health, capitation and other) to provide greater insight and potential explanations of results for aggregate expenditures. Analyses were estimated using weighted ordinary least squares (OLS), and robust standard errors were calculated to account for clustering at the practice level. Analyses using practice-level fixed effects revealed similar results; thus, only OLS results are presented. All analyses were conducted in Stata 13.1.
We calculated cost savings (or losses) to the Monroe Plan arising from changes in health care utilization using actual PMPM payments for each category of service in the baseline year and each intervention year for treatment and control practices. Cost savings or increases for treatment practices were compared with those found in control practices to identify incremental changes in the treatment practices. A negative net difference reflects savings to the Monroe Plan. A positive net difference reflects increases in costs to the Monroe Plan in intervention years.
To analyze return on investment, we combined PMPM payment data with intervention investment and operating costs reported by the Monroe Plan to conduct a discounted cash flow analysis. PMPM payments were annualized by multiplying by the number of months and by the number of eligible Monroe Plan-enrolled children with asthma in each intervention period. Total PMPM savings or payment increases as well as costs of operating the PACE intervention in each of the three intervention years were discounted back to the baseline period using a discount rate of 3% to reflect inflation and time preferences based on discussions with the Monroe Plan. We added discounted operating costs to the initial, baseline period investment costs to arrive at total intervention costs. Next, we subtracted total costs of the intervention from discounted PMPM savings (or cost increases) to arrive at the project's net present value. A positive net present value indicates the dollar contribution of the intervention to the Monroe Plan. A negative net present value means that the intervention had negative value; that is, benefits from the intervention were not enough to cover actual and opportunity costs.
Methods
Study Design
The evaluation was designed as an intent-to-treat, randomized, controlled trial with randomization at the practice level. The Monroe Plan identified 25 eligible primary care practices in Rochester, New York and the surrounding area, defined as those that served at least 20 Monroe Plan-enrolled children with asthma. Practices were first stratified by three characteristics: size, measured by the number of eligible Monroe Plan members; Federally Qualified Health Center status; and single versus multiple physicians for small practices. Practice size (small, medium and large) was determined by the number of Monroe Plan-enrolled children with asthma (fewer than 100, between 100 and 200 and > 200). The number of eligible members was used to measure practice size, because the incentive payment was tied to eligible members and not to the overall size of the practice based on total visits. Practices were randomized to either treatment (13 practices) or control arms (12 practices), and treatment group practices were invited to participate in the PACE intervention. Participation was voluntary and two treatment group practices declined to participate in the intervention; however, Monroe Plan expenditures on services provided by the practices were included in the evaluation (intent-to-treat approach) nonetheless.
The Intervention
With the dual goals of reducing unnecessary, high-cost utilization for children with asthma and improving outcomes, in November 2008, the Monroe Plan implemented the PACE project. The Monroe Plan offered the PACE intervention to all practices randomized to the treatment group. The intervention period lasted slightly >3 years, although the evaluation period was limited to two and a half years due to data availability. The centerpiece of the intervention was a $5 per month incentive fee paid to each treatment practice for each eligible Monroe Plan member assigned to the practice. All children with asthma that met inclusion criteria were considered eligible; not only those for which a chart audit was performed. In exchange for the incentive, practices conducted chart audits every 6 months on a sample of their eligible Monroe Plan-enrolled children with asthma. The sample sizes were determined by the Monroe Plan, but the individual patient charts were selected by the practices at random. Treatment practices with fewer than 60 eligible Monroe Plan members were required to complete chart audits for 20 eligible children; those with 60–400 eligible members were required to conduct chart audits for 28% of eligible children; and those with >400 eligible members were required to complete chart audits for 100 eligible children. These requirements were established by the Monroe Plan based on findings from a previous chart audit intervention with physician practices.
Practices were required to electronically report chart audit results to the Monroe Plan using an on-line tool. Practices in the treatment arm received training on the use of the online chart audit reporting tool and direct feedback on chart audit results compared with peer practices. Feedback was provided after each round of chart audits (seven rounds in total) through in-person meetings between the practice physician leading the chart audit activity and the medical director of the Monroe Plan. In addition, treatment practices participated in twice yearly asthma learning collaborative meetings. These meetings were structured as hour-long lunchtime meetings during which invited speakers presented on asthma care topics such as the definition and appropriate use of asthma action plans and monitoring of asthma symptoms. Practices were also given time to interact with each other so that they could discuss the intervention, best practices and challenges. The goal of the meetings was to improve providers' understanding of asthma care guidelines and to give providers strategies for improving the frequency with which they provided guideline-concordant asthma care. Control practices did not receive the incentive fee, were not asked to conduct chart audits and were not offered performance feedback or participation in the learning collaborative meetings.
Study Sample
The study population consisted of all children in treatment and control practices that met eligibility criteria. To be included, children needed to meet the following requirements: (i) be a Monroe Plan enrollee, though not necessarily continuously enrolled, during the study period (1 January 2008 to 30 June 2011 which included one baseline year and 2.5 intervention years), (ii) have an asthma diagnosis (defined by an ICD-9 code of 493.xx in any position on a medical claim in the year before or during the intervention period), (iii) be older than 2 but younger than 19 at study entry and (iv) be affiliated with a treatment or control practice as determined by the Monroe Plan primary care provider designation. Children aged 2 and younger were excluded because of the challenges associated with firmly diagnosing asthma in very young children. A total of 7731 children met eligibility criteria.
Data
Data for the study were obtained from two sources. Costs of investing in and operating PACE were collected directly from the Monroe Plan and included costs for personnel; contracted services; office, travel and training; equipment; incentive payments to practices; and indirect (overhead) costs. Costs were recorded in the period in which they were incurred. Evaluation-related costs were also collected but were separately identified since these costs would not have been incurred in the absence of the PACE evaluation (evaluation-related costs not shown).
Fee-for-service payments made by the Monroe Plan to primary care providers were collected from Monroe Plan claims data for a baseline period (1 January 2008–31 December 2008) and the two and half year intervention period (1 January 2009–30 June 2011). Fee-for-service payments were aggregated to the practice level and converted to annual per member per month expenditures on the following sources of care: inpatient; outpatient; office; ED; pharmacy; ambulance, home health, capitation and other; and expenditures on all sources of care.
Analysis
We conducted weighted difference-in-differences regression analyses to assess whether there were statistically significant differences in per member per month payments in treatment practices relative to control practices following the intervention. Analytic weights were defined as the number of member months contributed by each practice. The unit of analysis was the practice. We analyzed total expenditures to assess whether the intervention was successful, in aggregate, at reducing spending by the Monroe plan. We also ran separate analyses for expenditures on each of the six sources of care (inpatient; outpatient; office; ED; pharmacy; ambulance, home health, capitation and other) to provide greater insight and potential explanations of results for aggregate expenditures. Analyses were estimated using weighted ordinary least squares (OLS), and robust standard errors were calculated to account for clustering at the practice level. Analyses using practice-level fixed effects revealed similar results; thus, only OLS results are presented. All analyses were conducted in Stata 13.1.
We calculated cost savings (or losses) to the Monroe Plan arising from changes in health care utilization using actual PMPM payments for each category of service in the baseline year and each intervention year for treatment and control practices. Cost savings or increases for treatment practices were compared with those found in control practices to identify incremental changes in the treatment practices. A negative net difference reflects savings to the Monroe Plan. A positive net difference reflects increases in costs to the Monroe Plan in intervention years.
To analyze return on investment, we combined PMPM payment data with intervention investment and operating costs reported by the Monroe Plan to conduct a discounted cash flow analysis. PMPM payments were annualized by multiplying by the number of months and by the number of eligible Monroe Plan-enrolled children with asthma in each intervention period. Total PMPM savings or payment increases as well as costs of operating the PACE intervention in each of the three intervention years were discounted back to the baseline period using a discount rate of 3% to reflect inflation and time preferences based on discussions with the Monroe Plan. We added discounted operating costs to the initial, baseline period investment costs to arrive at total intervention costs. Next, we subtracted total costs of the intervention from discounted PMPM savings (or cost increases) to arrive at the project's net present value. A positive net present value indicates the dollar contribution of the intervention to the Monroe Plan. A negative net present value means that the intervention had negative value; that is, benefits from the intervention were not enough to cover actual and opportunity costs.
