What provides information on what and how the study will contribute and who will benefit from it?

Correspondence to Professor G Ross Baker, Department of Health Policy, Management and Evaluation, University of Toronto, 155 College Street, Toronto, Ontario, Canada M5T 3M6; [email protected]

Accepted 2010 Oct 17.

Copyright © 2011, Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

Abstract

Background

Efforts to improve the implementation of effective practice and to speed up improvements in quality and patient safety continue to pose challenges for researchers and policy makers. Organisational research, and, in particular, case studies of quality improvement, offer methods to improve understanding of the role of organisational and microsystem contexts for improving care and the development of theories which might guide improvement strategies.

Methods

This paper reviews examples of such research and details the methodological issues in constructing and analysing case studies. Case study research typically collects a wide array of data from interviews, documents and other sources.

Conclusion

Advances in methods for coding and analysing these data are improving the quality of reports from these studies.

Keywords: Quality improvement, case study, qualitative research, healthcare quality improvement, research

The gap between the knowledge of what works and the widespread adoption of those practices has become a major preoccupation of researchers and a challenge for funders and policy makers.1–3 Recognition of this ‘quality chasm’ (the term that the US Institute of Medicine used to describe the distance ‘between the healthcare we have and the care we could have’4) has led to an increased focus on quality improvement and implementation science to advance understanding of how to promote evidence-based practice. In turn, the focus on implementation has led to the development of multiple theories and frameworks to guide implementation,5–7 but no framework has demonstrated widespread results in practice.

There seems to be no immutable formula for successful implementation of innovations. While rational decision-makers would like the effectiveness of new technologies (including new work routines, devices and medications) to be the primary determinant of their adoption, research suggests otherwise. Healthcare systems are complex and variable. While some teams or organisations provide a ‘receptive context’ for innovation,8 others resist, having limited interest or abilities to implement new ideas. Decades of research in organisational and social sciences suggest that the nature of the innovation and the organisational, professional and health system contexts into which they are introduced influence their adoption.7 9–11 Thus, creating more effective, evidence-based care relies not just on developing and disseminating the evidence, but also on building knowledge of the ways in which innovations can be embedded into ongoing practice. Understanding the structures and processes of change is as critical as the knowledge of what works. In this paper, we outline how case study research can contribute a more detailed understanding of how to improve care. Case study methods are underutilised in quality improvement research, and given the growing calls to understand how innovation works in different contexts12–14 these methods could be a valuable addition to current approaches. We begin by illustrating the insights from case study research, and then examine the contribution of case study research to theory. Next we discuss strategies for analysing case study data and the scientific soundness of such information, ending with a discussion of the need for case studies to enhance the scientific understanding of quality improvement.

Insights from case study research

Three examples of how qualitative organisational research informs our understanding of the adoption of healthcare innovations illustrate the value of this research. Denis and colleagues15 studied the adoption of four innovations in several Quebec hospitals. They found that the strength of evidence of the innovation was not the only factor influencing adoption. Organisational arrangements, clinical skills and other more ambiguous elements that were open to interpretation and negotiation were also critical. In another study examining innovations in acute care and primary care settings in the UK, Ferlie16 identified the critical role of boundaries between professional groups. Unlike some prior studies where high levels of professionalisation facilitated adoption of innovations, Ferlie's research found that the varying roles, social boundaries and distinctive cognitive styles of different professional groups can limit the adoption of new technologies. For example, the introduction of an anticoagulation service was slowed by disagreements between cardiologists, primary care physicians, nurses and IT system designers about the appropriate indications for treatment.

The adoption of minimally invasive cardiac surgery for coronary artery bypass graft or valve replacement surgery in 16 US hospitals provides a third example. Edmondson and colleagues17 found that successful implementation depended on team learning processes rather than resources, academic status or innovation history. Innovative procedures like minimally invasive cardiac surgery disrupt established work routines. Establishing the necessary new routines for minimally invasive cardiac surgery depended on staff perceptions of psychological safety (the sense that ‘well-intentioned interpersonal risks will not be punished’), team stability and a collective learning process supported by leaders.

Each of these research projects used case study methods to identify the novel aspects of the process of implementing innovation. The research teams collected and analysed data from interviews, clinical data and documents. These research projects examined individuals or teams in context; they were embedded multiple case designs.18 Although the researchers had detailed knowledge of potentially relevant factors, these were primarily exploratory studies, examining which aspects of the innovation, the individuals and teams and the larger organisations influenced the adoption of the innovation.

The case study methods used in these three studies offer valuable tools in exploring the effectiveness of quality improvement more broadly. While case study research is a well-established method in organisational research, it appears to be less common in organisational health services research. Case study research designs involve the collection of qualitative (and often quantitative) data from various sources to explore one or more organisations or parts of organisations and the characteristics of these contexts.19 Some criticise case study research because they believe that the small sample size and lack of controls undermine the ability to generalise,20 while others worry that the analysis of case study data is often unsystematic.21 Yet case studies, because they detail specific experiences in particular contexts, offer the opportunity to learn more about the relationship of organisational processes and context to the success or failure of quality improvement efforts.

Contributions of case studies to theory

Case studies can inform the development of more robust theory that identifies the links between problem, intervention and outcome. Robert Yin, in his classic book,22 notes that case study research is particularly helpful when researchers want to answer questions of how or why things work in real life contexts. Theory generated from cases may help to make sense of the complex relationships that underline healthcare practice and elucidate why efforts to improve care succeed in some circumstances, but not in others.

Christensen and Carlile23 note that theory building (the creation of a ‘body of knowledge’ or understanding) occurs in two ways or stages; first there is a descriptive or inductive stage where researchers observe phenomena and describe and measure what they see (see figure 1). Based on these observations, researchers develop constructs that abstract the essence of what has been observed, classify or categorise these observations, and identify relationships between them. Through these activities, researchers develop theories or models which organise the aspects of the world they study. Second, in a deductive process, researchers test and improve these theories by exploring whether the same correlations exist in different data sets. This hypothesis testing allows the theory to be confirmed or rejected, and it also permits further specification of the theory to define the phenomena more precisely or specify the circumstances under which correlations hold. Where the goal of research is discovery or new explanations, case studies may offer a more powerful research design than experimental methods.24 25

Open in a separate window

Figure 1

Process of building theory.

Edmondson and McManus26 add to Christensen and Carlile's outline of the process of theory building and testing by identifying the importance of ‘methodological fit’ between theory building and different research methods. They suggest the appropriateness of different types of data varies depending on the research questions posed, the current state of the literature and the contribution envisaged from the research. Qualitative data, including interviews, observation and document analysis, are most appropriate for research where theory is nascent, and the research questions are exploratory. On the other hand, where theory is mature, survey methods and statistical testing focused on confirmation of hypotheses are more appropriate.

Organisational case studies have been an effective way to build theory in organisational research.18 Eisenhardt and Graebner27 note that ‘[a] major reason for the popularity and relevance of theory building from case studies is that it is one of the best (if not the best) of the bridges from rich qualitative evidence to mainstream deductive research. Its emphasis on developing constructs, measures and testable theoretical propositions makes inductive case research consistent with the emphasis on testable theory within mainstream deductive research.’ Some authors28 argue that single case studies provide more detail and offer ‘better stories’ which are helpful in describing phenomena. But others assert that multiple case studies provide a stronger base for theory building.22 27 Multiple case studies are powerful, since they permit replication and extension among individual cases. Replication enables a researcher to perceive the patterns in the cases more easily and to separate out patterns from change occurrences. Different cases can emphasise varying aspects of a phenomenon and enable researchers to develop a fuller theory. Fitzgerald and Dopson19 identify four common types of multiple case study designs, each based on a different logic. These include (1) matching or replication designs intended to explore or verify ideas; (2) comparison of differences, including cases selected for their different characteristics; (3) outliers, comparison of extremes to delineate key factors and the shape of a field; and (4) embedded case study designs where multiple units are examined to identify similarities and differences.

Despite growing numbers of studies on quality improvement in healthcare, there is limited growth in a more general theory about improvement. For example, there is a growing view that improvement interventions should be tailored to potential barriers. Yet, as Bosch notes,29 in many cases it is difficult to assess whether such tailoring was done based on a priori barrier identification, and explicit use of theory to match the intervention to the identified barriers. Bosch adds that ‘the translation of identified barriers into tailor-made [quality improvement] interventions [and their] implementation is still a black box for both educational and organisational interventions’ (p. 161). Case studies might contribute useful information to develop relevant theory. More broadly, case study research provides methods to examine organisational processes over time, examining the interplay of interventions with team dynamics or leadership strategy. For example, studies by Baker30 and Bate31 of high-performing healthcare organisations illustrate the challenges of creating, spreading and sustaining effective practice in organisations. Some case study research has followed organisations over extended time periods repeating interviews with key informants (eg, Denis' work on strategic change40 41). Unlike survey research and RCTs, case study research can analyse the process of implementation and unpack the dynamics of change.

Data collection and analysis

Organisational case studies can include a wide array of data, including interviews, documents, ethnography, survey data and observations. Although the case study is generally viewed as a qualitative method, it may include quantitative data. For example, Greenhalgh's study of the impact of ‘modernisation initiatives’ of the delivery of care in London42 used a wide range of methods and data, including interviews, document analysis and ethnography. Other organisational case study research17 32 40 43 has adopted a similar mix of data sources.

Case study research typically generates large quantities of data, which makes analysis critical, but complex. Moreover, the methods for aggregating data across projects are not well developed. Coffey and Atkinson note that the use of coding and sorting, and the identification of themes are ‘an important, even an indispensable, part of the qualitative research process.’44 Yet, there are challenges to such methods, since coding individual experiences can lead to ‘decontextualisation,’ fragmenting such meanings and making them difficult to identify.45 These problems are accentuated in multiple cases where results may reflect differences between the methods used, or the interests and orientation of various researchers. Even within the same research project, different investigators may take the lead in different cases. Dopson adds several other considerations about chronology: ‘Were the studies synchronous? Were they prospective or retrospective? Were they longitudinal or cross-sectional? How variable were the political and organisational contexts?’ (p. 6).32 Multiple case studies are difficult to report, given the space constraints for journal publication,27 and the use of extensive tables risks mimicking the presentation of quantitative data, stripping the illustrative detail from the case presentations.19

Synthesis across studies can help to build a more generalisable understanding of organisational strategies to support improvement. Yet views vary on whether we can synthesise research from multiple case studies undertaken independently. In their review of studies examining efforts to integrate evidence into clinical decision-making in UK healthcare, Dopson and colleagues32 compared and synthesised their findings reanalysing the original studies to identify themes, recoding their reports and then assessing the outputs generated by the five researchers involved (see table 1). Such tables offer a bird's-eye view of the extent to which common themes inform different case studies, but such summaries are divorced from understanding how these issues are inter-related within each case.

Table 1

Identifying research themes across studies of innovation diffusion32

ThemeDopson and Gabbay33 (1995)Wood et al34 (1998)Dawson et al35 (1998)Clinical Standards Advisory Group36 (1998) (Gabbay et al)Fitzgerald et al37 (1999)Dopson et al38 (1999)Locock et al39 (1999)1. Evidence is not sufficient32333332. Evidence is socially constructed23323333. Evidence is differentially available23233124. Hierarchies of evidence exist33332335. Other sources of evidence22333236. The importance of professional networks23333337. The role of professional boundaries23333228. Context as an influence32333339. The role of opinion leaders223333310. The enactment of evidence3333333

Open in a separate window

1, Theme is present; 2, strong evidence of theme; 3, very strong evidence of presence.

Methodological rigour

Efforts to create such syntheses raise issues about methodological rigour. For those researchers who adopt a positivist framework, the test of good case studies builds on four criteria used to assess the rigour of field research: internal validity, construct validity, external validity and reliability.22 These criteria might be applied to case studies in the following ways (see table 2).

Table 2

Framework for an investigation of the methodological rigour of case studies40

Internal validityConstruct validityExternal validityReliabilityResearch framework explicitly derived from literature (diagram or explicit description of causal relationships)Data triangulation (archival data; interview data; participatory observation; direct observation)Cross-case analysis (either multiple case studies or nested cases within one organisation)Case study protocol (how was case study conducted)Pattern matchingReview of transcripts and drafts by peersRationale for case study selection (and why appropriate in view of research questions)Case study database (with all available documents, interviews transcripts, etc)Theory triangulation (different theoretical lenses or bodies of research used as research framework or for interpretation)Review of transcripts and drafts by key informantsDetails on case study contextsOrganisation's actual name provided explicitly in the reportClear chain of evidenceClear explanation of data-collection circumstancesClear explanation of data analysis procedures

Open in a separate window

Gibbert and colleagues46 reviewed case studies published in the organisation/management literature between 1995 and 2000. They found research procedures enhancing external validity in 82 of 159 papers, and procedures supporting reliability in 27 of these papers. Few papers provided evidence of internal or construct validity. Yin proposes pattern matching; explanation building; addressing rival explanations and using logic models as strategies to address internal validity.22 Eisenhardt offers a series of questions that reflect on the match between method and results: ‘Have the investigators followed a careful analytical procedure? Does the evidence support the theory? Have the investigators ruled out rival explanations?’18 (p. 548). Non-positivist researchers employ other methods to ensure the soundness of their findings; for example, see Lincoln and Guba.47

An alternative measure of the rigour of case study research focuses on how good the theory is that emerges from this research. Pfeffer48 suggests that good theory is parsimonious, testable and logically coherent. Good theory should also address critical issues of interest to organisations and interested parties. Insights from other disciplines and attempts to seek out anomalies in other authors' work that might inform research in different areas are other strategies that may enrich the quality of case study research, improving the theory that results.48

Despite the need for more robust theory, why are there so few organisational case studies of quality improvement? Some candidate explanations might include: (1) the limited number of organisational scholars working in this area; (2) the dominance of alternative research paradigms that dismiss case study research; (3) difficulties in securing funding; (4) the lack of publication outlets; and (5) the absence of a clear understanding of the relationship of case study research to the development of theory, and the testing of theory using randomised control trials and other methods. Still, the emergence of several strong research groups in the UK, Canada and the USA, and growing numbers of high-quality publications offer hope. What is missing in quality improvement research is a clear understanding of how case study research could contribute to the broader research enterprise, enriching the qualitative understanding of the complex processes of improving healthcare delivery.

Conclusions

Comparative case study research provides useful methods for identifying the factors facilitating and impeding improvement. Although valuable in their own right, such methods also offer the opportunity to enrich more traditional approaches to assessing interventions, helping to explain why some interventions are unsuccessful, or why they seem to work effectively in some contexts but not in others. Efforts to improve patient safety and quality of care need to take into account the complexities of the systems in which these improvements are being introduced. Case study methods provide a robust means to guide implementation of effective practices.

Footnotes

Competing interests: None.

Provenance and peer review: Not commissioned; externally peer reviewed.

References

1. Dougherty D, Conway PH. The ‘3T's’ road map to transform US health care—the ‘how’ of high-quality care. JAMA 2008;299:2319–21 [PubMed] [Google Scholar]

2. Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8:iii–iv, 1–72. [PubMed] [Google Scholar]

3. Lavis JN, Robertson D, Woodside JM, et al. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q 2003;81:221–48, 171–2. [PMC free article] [PubMed] [Google Scholar]

4. Institute of Medicine Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy of Sciences, 2001 [Google Scholar]

5. Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50. [PMC free article] [PubMed] [Google Scholar]

6. Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care 1998;7:149–58 [PMC free article] [PubMed] [Google Scholar]

7. Greenhalgh T, Robert G, Macfarlane F, et al. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q 2004;82:581–629 [PMC free article] [PubMed] [Google Scholar]

8. Pettigrew A, Ferlie E, McKee L. Shaping Strategic Change: Making Change in Large Organizations, The Case of the National Health Services. London: Sage, 1992 [Google Scholar]

9. Damanpour F. Organizational innovation—a metaanalysis of effects of determinants and moderators. Acad Manage J 1991;34:555–90 [Google Scholar]

10. Wolfe RA. Organizational innovation—review, critique and suggested research directions. J Manage Stud 1994;31:405–31 [Google Scholar]

11. Rogers EM. Diffusion of Innovations. New York: Free Press, 1995 [Google Scholar]

12. Pawson R, Tilley N. Realistic Evaluation. London: Sage Publications, 1997 [Google Scholar]

13. Berwick DM. The science of improvement. JAMA 2008;299:1182–4 [PubMed] [Google Scholar]

14. Davidoff F. Heterogeneity is not always noise: lessons from improvement. JAMA 2009;302:2580–6 [PubMed] [Google Scholar]

15. Denis JL, Hebert Y, Langley A, et al. Explaining diffusion patterns for complex health care innovations. Health Care Manage Rev 2002;27:60–73 [PubMed] [Google Scholar]

16. Ferlie E, Fitzgerald L, Wood M, et al. The nonspread of innovations: the mediating role of professionals. Acad Manage J 2005;48:117–34 [Google Scholar]

17. Edmondson AC, Bohmer RM, Pisano GP. Disrupted routines: Team learning and new technology implementation in hospitals. Administr Sci Q 2001;46:685–716 [Google Scholar]

18. Eisenhardt KM. Building theories from case study research. Acad Manage Rev 1989;14:532–50 [Google Scholar]

19. Fitzgerald L, Dopson S. Comparative case study designs: their utility and methods. In: Buchanan DA, Bryman A, eds. The Sage Handbook of Organizational Research Methods. Thousand Oaks, CA: Sage, 2009:465–83 [Google Scholar]

20. Campbell DT. Degrees of freedom and the case study. Comp Polit Stud 1975;8:178–93 [Google Scholar]

21. Miles MB. Qualitative data as an attractive nuisance—problem of analysis. Administr Sci Q 1979;24:590–601 [Google Scholar]

22. Yin RK. Case Study Research: Design and Methods. 2nd edn Thousand Oaks, CA: Sage, 1994 [Google Scholar]

23. Christensen CM, Carlile PR. Course research: using the case method to build and teach management theory. Acad Manage Learn Educ 2009;8:240–51 [Google Scholar]

24. Vandenbroucke JP. Observational research, randomised trials, and two views of medical science. PLoS Med 2008;5:e67. [PMC free article] [PubMed] [Google Scholar]

25. Greenhalgh T, Russell J, Swinglehurst D. Narrative methods in quality improvement research. Qual Saf Health Care 2005;14:443–9 [PMC free article] [PubMed] [Google Scholar]

26. Edmondson AC, McManus SE. Methodological fit in management field research. Acad Manage Rev 2007;32:1155–79 [Google Scholar]

27. Eisenhardt KM, Graebner ME. Theory building from cases: opportunities and challenges. Acad Manage J 2007;50:25–32 [Google Scholar]

28. Dyer WG, Wilkins AL. Better stories, not better constructs, to generate better theory—a rejoinder to Eisenhardt. Acad Manage Rev 1991;16:613–19 [Google Scholar]

29. Bosch M, van der Weijden T, Wensing M, et al. Tailoring quality improvement interventions to identified barriers: a multiple case analysis. [Review] [39 refs]. J Evaluat Clin Practice 2007;13:161–8 [PubMed] [Google Scholar]

30. Baker GR, MacIntosh-Murray A, Porcellato C, et al., eds. High Performing Healthcare Systems: Delivering Quality by Design. Toronto, Ontario: Longwoods Publishing, 2008 [Google Scholar]

31. Bate P, Mendel P, Robert G. Organizing For Quality: The improvement journeys of leading hospitals in Europe and the United States. Oxford: Radcliffe Publishing, 2008 [Google Scholar]

32. Dopson S, FitzGerald L, Ferlie E, et al. No magic targets! Changing clinical practice to become more evidence based. Health Care Manage Rev 2002;27:35–47 [PubMed] [Google Scholar]

33. Dopson S, Gabbay J. Getting research into practice and purchasing (GRiPP). Oxford: NHS Executive, 1995 [Google Scholar]

34. Wood M, Ferlie E, FitzGerald L. Achieving change in clinical practice: Scientific, organisational and behavioral processes. Warwick: University of Warwick, CCSC, 1998 [Google Scholar]

35. Dawson S, Sutherland K, Dopson S, et al. The relationship between R&D and clinical practice in primary and secondary care: Cases of adult asthma and glue ear in children, final report: Judge Institute of Management Studies, University of Cambridge and Said Business School, University of Oxford, 1998 [Google Scholar]

36. Clinical Standards Advisory Group Clinical effectiveness. London, 1998 [Google Scholar]

37. FitzGerald L, Ferlie E, Wood M, et al. Evidence into Practice? An exploratory analysis of the interpretation of evidence. In: Mark A, Dopson S, eds. Organizational behaviour in health care. London: Macmillan, 1999 [Google Scholar]

38. Dopson S, Miller R, Dawson S. Differences in clinical practice: The case of glue ear. Quality in Health Care 1999;8:108–18 [PMC free article] [PubMed] [Google Scholar]

39. Locock L, Chambers D, Surender R, et al. Evaluation of the Welsh Clinical Effectiveness Initiative National Demonstration Projects: Final Report. Southhampton: Templeton College, University of London and Wessex Institute for Health Research and Development, University of Southhampton, 1999 [Google Scholar]

40. Denis JL, Lamothe L, Langley A. The dynamics of collective leadership and strategic change in pluralistic organizations. Acad Manage J 2001;44:809–37 [Google Scholar]

41. Denis JL, Langley A, Cazale L. Leadership and strategic change under ambiguity. Organ Stud 1996;17:673–99 [Google Scholar]

42. Greenhalgh T, Humphrey C, Hughes J, et al. How do you modernize a health service? A realist evaluation of whole-scale transformation in London. Milbank Q 2009;87:391–416 [PMC free article] [PubMed] [Google Scholar]

43. Stetler CB, Ritchie JA, Rycroft-Malone J, et al. Institutionalizing evidence-based practice: an organizational case study using a model of strategic change

What provide information to the reader on how the study will contribute?

Who are the beneficiaries of your study and what will they benefit from your study?

Who can benefit from research?

How will your study contribute to the existing body of knowledge?