Health Systems Research Service - QUERI Economic Analysis Guidelines

Last updated: March 10, 2021

The VA Quality Enhancement Research Initiative (QUERI) develops and tests interventions to improve the quality of VA care and methods to enhance implementation and dissemination of these interventions. Economic analysis can provide important information needed to evaluate these quality improvement efforts. However, economic analysis can be more informative for some QUERI projects than others. Evaluations of interventions that are costly, are expected to have large effects on health care utilization, or are ready to be implemented widely (e.g. the clinical effectiveness has been shown)[i] are most appropriate for economic analyses.

A. Types of analysis

There are several common forms of economic analysis that are appropriate for QUERI studies, including cost-identification, cost-effectiveness analysis (CEA), and budget impact analysis (BIA), also called business case analysis.

1. Cost-identification

The goal is to identify the unit cost of the intervention. A variety of methods can be employed to determine intervention costs. For studies that change the organization or delivery of care, micro-cost methods¹ are usually employed to estimate the cost of providing the intervention. Micro-costing involves detailed collection of activities and costs associated with those activities (personnel costs, supplies, space, training, and contracts). In a few cases, reasonable estimates can be made from published studies or existing datasets.

Pros:

• Often the necessary first step in a BIA or CEA
• Results can be informative, especially if there is information on the underlying production function, which might offer insights into efficiencies or quality

Cons:

• Can be highly context specific
• May require careful tracking of activities, which can be time consuming

2. Cost-effectiveness analysis (CEA)

CEA analyzes the costs and outcomes of an intervention relative to usual care. CEAs that use utility as the outcome can also be referred to as cost utility analysis. The result of a CEA is expressed as a ratio called the incremental cost-effectiveness ratio (ICER). The numerator of the ICER is the difference in costs generated by the people in the intervention arm and the costs of those in the comparator arm. The denominator is the difference in outcomes, which is recommended by CEA guidelines to be Quality Adjusted Life Years (QALYs).

The standard CEA assumes the audience or the perspective for the study is society as a whole. As a result, it considers the cost to all relevant parties impacted by an intervention, and might include patients, family members, providers and the health care system. Standard CEA considers average costs, not incremental costs, which includes fixed costs, facility overhead, and depreciation. A CEA done from a payer perspective would limit the costs included to only those incurred by the payer or patient, respectively.

Standard CEA considers costs and benefits through a lifetime horizon often via modeling because long-term costs and benefits are rarely measured prospectively for several decades in a clinical trial. These are discounted to reflect the decline in economic value that results from delay. CEA should include sensitivity analyses, which evaluate the uncertainty on study findings. For an example of a CEA using QALYs, see Pyne et al., 2010.² CEAs can also use clinical endpoint instead of QALYs, but there are limitations with these approaches and they are not further discussed here.

Pros:

• The gold standard for understanding the value of a new health care technology
• Provides very useful information that guides the optimal allocation of care

Cons:

• Requires a substantial investment in money and time (often years) to build the decision models
• Not easy to use to evaluate broad strategies (e.g., increasing access to care)

3. Budget impact analysis (BIA)

A BIA considers costs from the perspective of the payer, comparing the costs of two or more interventions, over the short-term (e.g. 1-5 years).³ The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) has issued guidance on BIA that should be considered. BIAs often consider the cost of the intervention, cost of implementation and down steam health care costs that changed as a result of the intervention. Discounting of future costs and benefits is usually not done and it may be reasonable to exclude fixed costs. Every BIA should include a sensitivity analysis that varies its parameters through a range of reasonable values.

Pros:

• Often useful for decision makers to understand how a new intervention will affect their budget

Cons:

• Ignores non-financial benefits (e.g., quality of life)
• Estimating the cost of implementation requires careful tracking of activities, which can be time consuming

4. Implementation

In all of the above analyses, inclusion or exclusion of the costs of implementation should be described in the analysis plan. These costs must be specifically differentiated from other costs to understand the economic implications for implementation strategies. To date, very few comparative economic evaluation analyses of implementation interventions have been reported.^4,5 However, there is increasing demand for QUERI economic analyses to involve the estimation of the costs of the implementation intervention strategy used to enhance the uptake of an effective program or practice (e.g., Hybrid Type III implementation intervention studies - see Curran et al., 2012⁶). An excellent example of the estimation of implementation costs is found in Liu et al., 2009.⁷

Table 1 Comparison of the basic elements of economic analyses

B. Presentation issues

The report of an economic evaluation must document data sources, methods, and assumptions. For a summary of components that should be included when presenting an economic evaluation, see Drummond and Jefferson, 1996.⁸

C. Training and Resources

Training and resources for VA employees. Non-VA researchers will not have access to some of these items.

1. Cyber-seminars

VA researchers present many cyber-seminars on topics related to economic analysis within QUERI. Future and past cyber-seminars, including archived recordings, may be found on the HSR web site at http://www.hsrd.research.va.gov/cyberseminars/default.cfm.

2. Resource centers

Health Systems Research (HSR) funds resource centers.

• The Health Economics Resource Center (HERC) (herc.research.va.gov) documents economics data and provides publications and training on economic methods. (Visit the corresponding intranet web site to download most documents.)
• VA Information Resource Center (VIReC) (virec.research.va.gov) documents most other VA datasets and presents cyber-seminars on informatics topics.
• Center for Information Dissemination and Education Resources (CIDER) (cider.research.va.gov) coordinates the HSR cyber-seminar series and information dissemination
• Center for Evaluation and Implementation Resources (CEIR) (queri.research.va.gov/ceir/default.cfm) provides consultation and training in evaluation and implementation science theory, methods, and application
• Partnered Evidence-Based Policy Resource Center (PEPReC) (https://www.peprec.research.va.gov/) refines VA measurements and investigates relationships among access to care, productivity, demand, and capacity to improve policy and planning. It also coordinates and supports the design and implementation of randomized program evaluations with strong potential to improve the quality and efficiency of VA healthcare.

3. Journal articles and book chapters

a. Costing and the stages of implementation

Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. Feb 2004;8(6):iii-iv, 1-72.

Liu CF, Rubenstein LV, Kirchner JE, et al. Organizational cost of quality improvement for depression care. Health Serv Res. Feb 2009;44(1):225-244.

McInnes DK, Solomon JL, Shimada SL, et al. Development and evaluation of an internet and personal health record training program for low-income patients with HIV or hepatitis C. Med Care. Mar 2013;51(3 Suppl 1):S62-66.

McIntosh E. Economic evaluation of guidelines implementation strategies. In: Changing professional practice: theory and practice of clinical guidelines implementation. Thorson T, Mäkelä M, eds. Copenhagen: Danish Institute for Health Services Research and Development, 1999. DSI Report no. 99.05. URL: http://www.dsi.dk/projects/cpp/monograph/DSI9905.pdf.

Severens JL. Value for money of changing healthcare services? Economic evaluation of quality improvement. Qual Saf Health Care 2003;12(5):366-371.

Smith MW, Barnett PG. The role of economics in the QUERI program: QUERI Series. Implement Sci. 2008;3:20.

Vale L, Thomas R, MacLennan G, Grimshaw J. Systematic review of economic evaluations and cost analyses of guideline implementation strategies. Eur J Health Econ 2007;8(2):111-121.

Wagner TH. Rethinking how we measure costs in implementation research. J Gen Intern Med. Nov 2020;35(Suppl 2):870-874.

Wagner TH, Yoon J, Jacobs JC, So A, Kilbourne AM, Yu W, Goodrich DE. Estimating Costs of an Implementation Intervention. Medical Decision Making. Nov 2020;40(8):959-967.

Yoon, J. Including Economic Evaluations in Implementation Science. J Gen Intern Med. 2020;35:985–987.

b. Cost-effectiveness analysis methods (without reference to implementation)

Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the British Medical Journal. BMJ 1996;313:275-283.

Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG. Cost-effectiveness in Health and Medicine: Second Edition. New York: Oxford University Press, 2016.

The following paper summarize contents of the Neumann et al. book cited above.

Sanders GD, Neumann PJ, Basu A, Brock DW, Feeny D, Krahn M, Kuntz KM, Meltzer DO, Owens DK, Prosser LA, Salomon JA, Sculpher MJ, Trikalinos TA, Russell LB, Siegel JE, Ganiats TG. Recommendations for Conduct, Methodological Practices, and Reporting of Cost-effectiveness AnalysesSecond Panel on Cost-Effectiveness in Health and Medicine. JAMA. 2016;316(10):1093–1103. doi:10.1001/jama.2016.12195

c. Budget impact analysis

Luck J, Parkerton P, Hagigi F. What is the business case for improving care for patients with complex conditions? J Gen Int Med 2007;22(Suppl 3):396-402.

Mauskopf JA, Sullivan SD, Annemans L, et al. Principle of good practice for budget impact analysis: report of the ISPOR Task Force on Good Research Practices – Budget Impact Analysis. Value in Health 2007;10(5):336-347.

Nicholson S, Pauly MV, Polsky D, et al. How to present the business case for healthcare quality to employers. Appl Health Econ Health Policy 2005;4(4):209-218.

Sullivan SD, Mauskopf JA, Augustovski F, et al. Budget impact analysis-principles of goodpractice: report of the ISPOR 2012 Budget Impact Analysis Good Practice II Task Force. Value Health. Jan-Feb 2014;17(1):5-14.

Wagner TH, Dopp AR, Gold HT. Estimating downstream budget impacts in implementation research. Medical Decision Making. Nov 2020;40(8):968-977.

d. Sensitivity analysis

Briggs A, Goeree R, Blackhouse G, O’ Brien B. Probabilistic analysis of cost-effectiveness models: choosing between treatment strategies for gastroesophageal reflux disease. Medical Decision Making 2002;4:290–308.

Briggs A, Schulpher M, Claxton K. Decision modelling for health economic evaluation. Oxford: Oxford University Press, 2006.

Doubilet P, Begg CB, Weinstein MC, Braun P, McNeil BJ. Probabilistic sensitivity analysis using Monte Carlo simulation. A practical approach. Medical Decision Making 1985;5(2):157-177.

References

• 1. Pyne JM, Fortney JC, Tripathi SP, Maciejewski ML, Edlund MJ, Williams DK. Cost-effectiveness analysis of a rural telemedicine collaborative care intervention for depression. Archives of general psychiatry. Aug 2010;67:812-821.
• 2. Sullivan SD, Mauskopf JA, Augustovski F, et al. Budget impact analysis-principles of good practice: report of the ISPOR 2012 Budget Impact Analysis Good Practice II Task Force. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research. Jan-Feb 2014;17:5-14.
• 3. Vale L, Thomas R, MacLennan G, Grimshaw J. Systematic review of economic evaluations and cost analyses of guideline implementation strategies. The European journal of health economics : HEPAC : health economics in prevention and care. Jun 2007;8:111-121.
• 4. Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health technology assessment. Feb 2004;8:iii-iv, 1-72.
• 5. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical care. Mar 2012;50:217-226.
• 6. Liu CF, Rubenstein LV, Kirchner JE, et al. Organizational cost of quality improvement for depression care. Health Serv Res. Feb 2009;44:225-244.
• 7. Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ (Clinical research ed.). Aug 3 1996;313:275-283.

[i] Note: when the efficacy of a clinical intervention has not yet been shown, it is rarely appropriate to undertake a cost-effectiveness or budget impact analysis.

Last updated: March 10, 2021