Overview of the design process for mixed method evaluation laure sharp and joy frechtling developing evaluation questions selecting methods for gathering the data. Overview of the design process for mixed method evaluations. Depending on choices made across four dimensions, mixed methods can provide an investigator with many design choices which involve a range of sequential and concurrent strategies. Mixed methods research requires a purposeful mixing of methods in data collection, data analysis and interpretation of the evidence.
This technical note provides guidance on using mixedmethods for evaluations. Integrating quantitative and qualitative data in mixed. The selection of a research approach is also based on the nature of the research problem or issue being addressed. This brief focuses on using mixed methods to evaluate patientcentered medical home pcmh models. Lessons from a mixedmethods approach to evaluating active. Excerpt there is rarely a single evaluation methodology that can fully capture all of the complexities of how programs operate in the real world.
Best practices for mixed methods research in the health. Mixedmethods evaluation 151 acknowledgment the authors wish to extend. Our references are a combination of methods texts from social sciences and exam. Challenges in using mix methods in evaluation stefan cojocaru conditions for those who consult the evaluation report 7 triangulation of the evaluators for ensuring the dependability of data, for the purposes of extracting and building the most appropriate codes 8 testing the hypothesis and analysing the possible alternative explanations able to provide relevant information about the. The unique feature of mixed methods approaches is that they seek to. Many definitions of mixed methods are available in the literature e. Discuss and agree on the minimum criteria for mm, in order to be able to justify its. Does the research question justify the use of mixed methods. Identify an approach to design in addition to using fixed and emergent mixed methods designs, researchers also use different approaches for designing their mixed methods studies.
Mixed methods research draws on potential strengths of both qualitative and quantitative methods,3. Examples of evaluation designs at each point on the quant qual continuum. Obeying these rules is not a guarantee of a strong design, but it does contribute to it. Multimethodology or multimethod research includes the use of more than one method of data collection or research in a research study or set of related studies. The strengths of the multiple, mixedmethods in this evaluation included. For the purpose of this article, we use the following definition of mixed methods research johnson et al. Mixed methods is a research approach, popular in the social, behavioral, and health sciences, in which researchers collect, analyze, and integrate both quantitative and qualitative data in a single study or in a sustained long. This article provides researchers with knowledge of how to design a high quality mixed methods research study. It concludes by suggesting topics for future researchincluding the use of mixed methods in constructing counterfactuals, and in conducting reasonable evaluations within severe time and budget constraints. Mixed methods research is viewed as the third methodological movement and as an approach it has much to offer health and social science research. Mixed method evaluation, by definition, requires expertise in designing and implementing different methods, as well as in analyzing, interpreting and integrating the findings that they generate.
Mixed method research defined cynthia joffrion prlog. The technical note provides guidance to usaid staff and partners on how mixedmethod evaluations are conducted and important considerations when managing a mixed. Definition method evaluation systematically integrates two or more evaluation methods, potentially at every stage of the evaluation process, usually drawing on both quantitative and qualitative data. The use of mixed methods in randomized control trials white. Given the youthful status of discussions about the meaning of mixed. Currently, the fundamental study techniques which are widely used are quantitative, qualitative and mixed methods. Best practices for mixed methods research in the health sciences embedding data. A mixedmethod evaluation systematically integrates two or more evaluation methods, usually drawing on both quantitative and qualitative data. Mixed methods also mirror the way individuals naturally collect informationby integrating quantitative and qualitative data. Defining features of these designs are reported along with quality control methods, and ethical concerns.
Applying mm approaches at each stage of the evaluation part iii. In this form of integration, a dataset of secondary priority is embedded within a larger. Can be in a single study or in multiple phases of a study. Mixed methods research design varies depending on the purpose of the study and philosophical background of the researcher. For example, sports stories frequently integrate quantitative data scores or number of errors with qualitative data descriptions and images of highlights to provide a more complete story than either method would alone. A mixed methods design is characterized by the combination of at least one qualitative and one quantitative research component. It is the descriptive data from observation or interviews which are not structured taylor, 2009. Thus, the evaluation team must include a combination of skills. This is a pdf file of an unedited manuscript that has been. Best practices for mixed methods research in the health sciences. This guidance note explains the essential elements of the mixed methods approach and how it can be used in an impact evaluation, while highlighting potential applications and benefits for ngos.
For each evaluation question, the task is to identify the best available method for answering that question. Consequently, evaluators must find creative ways to combine different evaluation frameworks, tools and techniques1hence the growing interest in mm approaches. Defining mixed methods evaluation oxford handbooks online. The evaluation of community engagement may need both qualitative and quantitative methods because of the diversity of issues addressed e. Introduction to mixed methods in impact evaluation. Understanding mixed methods approaches in evaluation involves understanding the philosophical stances, theoretical perspectives, and practical strategies that have emerged in the evaluation world. To design a mixed study, researchers must understand and carefully consider each of the dimensions of mixed methods design, and always keep an eye on the issue of validity. This definition of mixed methods in evaluation acknowledges the grounding of. From this starting point, a list of evaluation questions is identified. Mixed methods, program evaluation, impact assessment, context. Impact evaluations should make maximum use of existing data and then fill gaps with new. For example, the investigator can gather qualitative data to assess the personal experiences of. Mixed methods research characteristics of mixed methods research collect and analyze both quantitative and qualitative data.
Mixed methods research begins with the assumption that investigators, in understanding the social and health worlds, gather evidence based on the nature of the question and theoretical orientation. Increased interest in and demand for mixed methods evaluations. Agree on use of mixed methods as the generic term for the area, even while not necessarily agreeing on a fixed definition of that term. Though the advantages of employing mixed methods are convincing.
The complexity of the albd projects called for a mixedmethods evaluation to. Understanding mixed methods approaches in evaluation involves understanding the philosophical stances, theoretical perspectives, and practical strategies. This approach to research is used when this integration provides a better understanding of the research problem than either of. A framework for viewing mixed methods research a definition of mixed methods research, and the importance of this approach, historically and today purpose and organization because mixed methods is a new design, researchers need an introduction to the approach, guidance as to how to conduct the design, and information. It is up to the researcher to determine how to use mixed methods designs. A theorydriven evaluation perspective on mixed methods research huey t.
Consequently, when it comes to designing an evaluation, experience has proven that the evaluator must keep in mind that the specific questions being addressed and the audience for the answers must influence the selection of an evaluation design and tools for data collection. As a methodology, it involves philosophical assumptions that guide the direction of the collection and analysis of data and the mixture of qualitative and quantitative data in a single study or series of studies. Two important components in each definition are that the approach to. This approach to research is used when this integration provides a better understanding of the research problem than either of each alone. Introduction to mixed methods in impact evaluation better evaluation. Welcome to todays offee reak presented by the evaluation and. To utilize this design effectively, you must understand both quantitative and qualitative research. The scope includes delineating where mixed methods research may be used most effectively, illuminating design and procedure issues, and determining the logistics of conducting mixed methods research. This tips includes the rationale for using a mixedmethod evaluation design, guidance for selecting among methods with an example from an evaluation of a. Sampling strategies for quant and qual oriented mm evaluations 19 3. Pdf challenges in using mix methods in evaluation researchgate. The starting point should be priority programs to be evaluated or policies to be tested.
Evaluation design for the hypothetical project laure sharp step 1. Different types of triangulation used in mixed method evaluations. Mixed methods researchers use and often make explicit diverse. Determine appropriate data sources and data collection approaches to obtain answers to the final set of.
Mixed methods is a research approach whereby researchers collect and analyse both quantitative and qualitative data within the same study. Data collection and analysis methods should be chosen to match the particular evaluation in terms of its key evaluation questions keqs and the resources available. A journal of mixed methods research is planned by sage. Mixed methods research is a methodology for conducting research that involves collecting, analysing and integrating quantitative e. Building on such realization, this chapter provides a critical overview of why and how mixed methods have been used in evaluation for decades. Quantitative, qualitative, and mixed research methods in. The unique feature of mixed methods approaches is that they seek to integrate social. The choice of methods should fit the need for the evaluation, its timeline, and available resources holland et al. Leech n, onwuegbuzie a, 2008 a typology of mixed methods research designs. Highlight the reasons why researchers employ a mixed methods. A theorydriven evaluation perspective on mixed methods. Define mixed methods research by incorporating the definition in chapter 1 that focuses on combining both quantitative and qualitative research and methods in a research study see a more expanded view of defining mixed methods research in ohnson, onwuegbuzie.
For the moment, a generic definition of mixed methods research presented by creswell. Mixed methodology is a design for colleting, analyzing, and mixing both quantitative and qualitative data in a single study or series of studies to understand an evaluation problem. Mixed methods research, defined a mixed methods research design is a procedure for collecting, analyzing, and mixing both quantitative and qualitative research and methods in a single study to understand a research problem. Definition mixed methods in general, mixed methods research represents research that involves collecting, analyzing, and interpreting quantitative and qualitative data in a single study or in a series of studies that investigate the same underlying phenomenon. When reading mixed methods research or writing a proposal using mixed methods to answer a research question, the six questions below are a useful guide12. Is the method sequence clearly described, logical in flow and well aligned with study aims.
Unquestionably, the three approaches are not as discrete as they first appear. Bergman 2008 so first, the worlds shortest definition of mixed methods. Multiple and mixed methods in formative evaluation. There is rarely a single evaluation methodology that can fully capture all of the complexities of how programs operate in the real world. Qualitative study involves evaluation of data, like different interviews, videos, pictures and objects such as artifacts. Using mixed methods to evaluate complex interventions 21 3. Mixed methods research is more specific in that it includes the mixing of qualitative and quantitative data, methods, methodologies, andor paradigms in a research study or set of related studies. The key word is mixed, as an essential step in the mixed methods approach is data linkage, or integration at an appropriate stage in the research process. Welcome to todays offee reak presented by the evaluation. They argue that mixed methods research is one of the three major research paradigms quantitative research, qualitative research, and mixed methods research. Hence it is likely that any one study will contain a mix of methods. Introduction to mixed methods in impact evaluation interaction.
For quantitative, qualitative, and mixed methods, we offer a basic definition, aims, appropriate research questions, or hypotheses. Based on examples of evaluations, caracelli and greene believe that mixing. The fields of applied social science and evaluation are among those which have shown the greatest popularity and uptake of mixed methods research designs. Jul 23, 2010 mixed methods research design varies depending on the purpose of the study and philosophical background of the researcher. Useful resources and exemplary study references are shared. There are several approaches to design that have been discussed in the liter. Mixed methods research design implicates merging or incorporating qualitative and quantitative research and data in a single study. A range of quantitative, qualitative and theorybased approaches for defining the counterfactual. A separate section compares and contrasts evaluation criteria for the three approaches. Definition mixed methods mixed methods research is a research design with philosophical assumptions as well as methods of inquiry. Because mixed methods is a new design, researchers need an introduction to the approach, guidance as to how to conduct the design, and information about the specific procedures involved. Chapter 5 overview of the design process for mixed method evaluations.
The use of mixed methods in randomized control trials. Using mixed methods in monitoring and evaluation the university. For the purpose of this presentation, well use a simple definition for mixed methods. Chen university of alabama at birmingham the purpose of this article is to discuss the conceptual framework and strategies used in theorydriven evaluations in relation to mixed methods research and to explore the opportunities and challenges emerging from theorydriven.
334 186 1426 1263 1137 251 1518 1411 1538 1336 781 652 900 1282 277 807 1044 872 850 1200 1045 49 1227 853 102 390 769 179 1390 1191 1055 956 1250 573 147 219 1006