SAGE Research Methods

Understanding and Evaluating Research: A Critical

Guide

Author: Sue L. T. McGregor

Pub. Date: 2019

Product: SAGE Research Methods

DOI: https://dx.doi.org/10.4135/9781071802656

Methods: Theory, Research questions, Mixed methods

Disciplines: Anthropology, Education, Geography, Health, Political Science and International Relations,

Psychology, Social Policy and Public Policy, Social Work, Sociology

Access Date: January 11, 2023

Publishing Company: SAGE Publications, Inc

City: Thousand Oaks

Online ISBN: 9781071802656

© 2019 SAGE Publications, Inc All Rights Reserved.

Overview of Research Design and Methods

Learning Objectives

• Distinguish between methodology, method, and research design

• Compare and contrast research design as logical and as logistical

• Appreciate the difference between reconstructed logic (quantitative) and logic-in-use (qualitative), es-

pecially an emergent research design

• Explain the five logics of mixed methods research designs

• Appreciate the link between research inquiry and research design

• Describe the purpose and importance of the Methods section of a research report

• Compare and contrast qualitative and quantitative intellectual inquiries

• Identify the major reporting components (subheadings) of qualitative and quantitative research re-

ports

• Compare and contrast the most agreed-to approaches and terms about research integrity, rigor, and

quality that are used in each of the three methodologies, and learn the attendant strategies to meet

the standard for the specific research methodology

Introduction

This chapter focuses on the constructs of research design and the Methods section in a research paper. Re-

search design is a larger construct than methods, to be explained shortly. But within a research paper, once

the authors have stated the research question, developed an introduction to the study, and presented a re-

view of the literature (and maybe a theoretical framework), their next step is to provide a description of the

strategies used to collect and analyze data pursuant to the research question—that is, their methods. This

chapter provides a generic discussion of methods, followed with much more detail in Chapter 9 (qualitative

methods) and in Chapter 10 (quantitative and mixed methods).

As a caveat, a detailed discussion of how to use specific methods is beyond the scope of this overview chap-

ter, or even this book. There is no attempt to explain how to do a survey, conduct a scientific experiment, pre-

pare a case study, or engage in ethnographic research where researchers immerse themselves in the lives of

the participants. That being said, the general discussions in Chapters 9 (qualitative) and 10 (quantitative and

SAGE

© 2018 by SAGE Publications, Inc.

SAGE Research Methods

Page 2 of 27 Understanding and Evaluating Research: A Critical Guide

mixed methods) will address the basic conventions pursuant to preparing, conducting, and reporting these

types of research, which entails identifying common methods.

This generic chapter will begin with a discussion of the larger construct of research design, including the link

between research design and research inquiry, research design as logic and logistical, and the most common

research designs organized by the three methodologies: qualitative, quantitative, and mixed methods. The

conversation then shifts to a general overview of methods (distinguished from methodology). The purposes of

the Methods section are identified followed with general introductions to the major differences between qual-

itative and quantitative inquiries, the major reporting components (subheadings) of each of these research

reports, and the topic of rigor and quality in each of the three methodologies.

Etymology and Definition of Methods and Research Design

Method is Greek methodus, “for mode of inquiry or investigation.” It stems from meta, “after,” and hodos, “a

travelling, a pursuit, a way of teaching or going” (Harper, 2016). In effect, method refers to investigating or

inquiring into something by going after or pursuing it, especially in accordance with a plan. It involves tech-

niques, procedures, and tasks used in a systematic, logical, and orderly way (Anderson, 2014). Within the

context of conducting and reporting research, it is the stage wherein researchers design instruments, appa-

ratus, or procedures or gain site access (if relevant), obtain a sample, and then collect and analyze data from

that sample (or entire population) (Johnson & Christensen, 2012). As was discussed in Chapter 2, this book

distinguishes between method and methodology, with the latter connoting the philosophical underpinnings of

the study.

The other term used in this chapter is research design. Research is French recercher, “to search.” In the con-

text of this book, it refers to the accumulation of data that are interpreted, leading to new knowledge. Design

is Latin designare, “to mark out, devise, choose, designate.” A design can be defined as a plan used to show

the workings of something before it is made or created. It can also mean the underlying purpose of some-

thing, in this case, the search for knowledge (Anderson, 2014; Harper, 2016). From a technical stance, the

research design refers to the overall strategy that researchers choose to integrate the different components of

their study in a coherent and logical way, thereby ensuring they can effectively address the research question

using the new knowledge created from the study (Labaree, 2016). Research design also entails logic (Yin,

1984), to be discussed shortly.

SAGE

© 2018 by SAGE Publications, Inc.

SAGE Research Methods

Page 3 of 27 Understanding and Evaluating Research: A Critical Guide

Research Design

Many disciplines mistake research design for methods (de Vaus, 2001). This section explains how this book

distinguishes between these terms, respecting the lack of a consensus in the scholarly arena for their usage.

Research design is a larger construct than method. Per above, methods refer to technical procedures, tech-

niques, or steps taken to obtain information and analyze data for a study. A design is a plan made before

something is done (Anderson, 2014). Designing research is a process that entails both logic (thinking and

reasoned judgments) and logistics (doing), with logic coming first, inherently shaping logistics (methods) (Yin,

1984).

Research Inquiry and Research Design

The logic and thinking that researchers use to design their research is affected both by the (a) methodology

(which shapes the research questions and all assumptions underlying the effort), and (b) type of research

inquiry they are conducting. In short, (a) exploratory research strives to reach a greater understanding of a

problem, usually laying the groundwork for future studies; (b) descriptive research seeks more information

so as to accurately describe something in more detail, creating a fuller picture by mapping the terrain; and

(c) explanatory research seeks to connect ideas to understand causal inferences (explain relationships) (de

Vaus, 2001; Suter, 2012; Yin, 1984). These approaches apply to both quantitative and qualitative research

methodologies (except explanatory), with qualitative also seeking to (d) illuminate meaning and subjective

experiences and (e) understand processes and structures (Blaxter, 2013; Shank & Brown, 2007).

Articulating Research Purpose in Research Design

Each of these five types of research inquiry represents the deeper purpose of the study (the problem), or the

reasons for doing it, which is why Yin (1984) said research design is logical (i.e., it entails reasoned judg-

ments). Each type of inquiry offers a different reason for why the study is needed (e.g., to describe, explore,

find meaning, or theorize). Authors must not confuse research purpose (reason for inquiry) with methodolo-

gy, research design, research question, or methods (see example 8.1). When identifying the nature of their

research inquiry, they can use headings in their paper such as Justification for the Study, Importance of the

Study, or Objectives of the Study (Newman, Ridenour, Newman, & DeMarco, 2003). A clearly stated research

SAGE

© 2018 by SAGE Publications, Inc.

SAGE Research Methods

Page 4 of 27 Understanding and Evaluating Research: A Critical Guide

purpose will help readers formulate a realistic set of expectations about a study and better ensure they eval-

uate the quality of the study’s design within the context of the author’s purpose (Knafl & Howard, 1984) (see

Chapter 6).

Example 8.1 Research purpose versus question The problem is the deeper, more complex rea-

son why the researcher is conducting the study (e.g., to explore, describe, explain, or find meaning).

Newman et al. (2003) recounted a quantitative study in which the research question was incorrectly

presented as the research problem: “What is the effect of making a substantial reduction in class

size on student achievement?” The researchers erroneously characterized class size as the problem

when in fact students’ lack of achievement was the problem and the reason why this explanatory

study was needed (i.e., to explain). In this study, reducing class size was but one solution to in-

creasing student achievement. By losing focus on what the real problem was (lack of achievement),

the researchers designed an inappropriate study if they wanted to explain it. An unfortunate conse-

quence of authors neglecting to clearly state their purpose and problem is that some readers may

uncritically accept their results and change their practice when they should not.

Research Design as Logical and Logistical

Research designs guide the methods decisions that researchers must make during their studies, and they

set the logic by which interpretations are made at the end of studies (Creswell, 2008). To further appreciate

the link between research design logic and method, authors can consider this metaphor. Before builders or

architects can develop a work plan or order building materials, they must first establish the type of building

required, its uses, and the needs of the occupants; that is, they must think about their entire build and justify

any design decisions they make. Their work plans (methods) to construct the building then flow from this logic

(i.e., their reasoned judgments about the build). The same idea holds for a study’s research design (de Vaus,

2001).

Research design as logic concerns researchers thinking about what sorts of data are needed to answer the

research question, including what methods might be most appropriate to generate those data. The type of

research inquiry (i.e., the purpose behind the research) shapes the overall structure of the study, especial-

ly the methods (Kotler, 2000; Newman et al., 2003). Research design as logical equates to a blueprint with

SAGE

© 2018 by SAGE Publications, Inc.

SAGE Research Methods

Page 5 of 27 Understanding and Evaluating Research: A Critical Guide

specific, sequenced (sometimes iterative) steps that are be completed to bring the plan to closure (i.e., the

methods, which are the focus of this chapter). Research design as logistical refers to the work plan developed

by the researcher to collect evidence and analyze data to answer the research question and respect the type

of research inquiry (the logic). Logistical means planning and organizing to make sure things are where they

need to be so an activity or process can happen effectively (Anderson, 2014). The logic affects the logistics

(methods), and the logistics reflect the logic (Yin, 1984) (see Figure 8.1).

Quantitative Research Design Logic

Quantitative research uses a predetermined, fixed research plan based mostly on reconstructed logic. This

logic of research is based on organizing, standardizing, and codifying research into explicit rules, formal pro-

cedures, and techniques so others can follow the same linear plan and reconstruct the study. This is the logic

of “how to do research” and is highly organized and systematic (Jarrahi & Sawyer, 2009; Neuman, 2000).

The type of research inquiry determines the research design created using this logic (see Table 8.1). Should

they create a cross-sectional design (collect data once from one sample), a repeated cross-sectional design

(collect data once from different samples), a longitudinal design (collect data from one sample over time), a

one-subject design, an experimental design, a case study, or some other design (Kotler, 2000)?

SAGE

© 2018 by SAGE Publications, Inc.

SAGE Research Methods

Page 6 of 27 Understanding and Evaluating Research: A Critical Guide

Figure 8.1 Research Design as Logic and Logistical

Table 8.1 Three Types of Research Inquiries, With Examples of Quantitative Research Designs

Exploratory Research Inquiry Descriptive Research Inquiry Explanatory Research Inquiry

Cross-sectional design

Case study design

Cross-sectional design

Longitudinal design

Case study design

Cross-sectional design

Experimental design

Case study design

SAGE

© 2018 by SAGE Publications, Inc.

SAGE Research Methods

Page 7 of 27 Understanding and Evaluating Research: A Critical Guide

Qualitative Research Design Logic

The research designs included in Table 8.1 (based on only reconstructed logic) do not adequately represent

“the logic and processes of qualitative research [which] lacks such an elaborate typology into which studies

can be pigeonholed” (Maxwell, 2008, p. 214). Maxwell (2008) said “this does not mean that qualitative re-

search lacks design” (p. 215). Instead, qualitative research requires a broader and less restrictive concept of

research design, in which researchers use “‘logic-in-use’ [as well as] ‘reconstructed logic’ [to accommodate

the] ‘design in use’ [principle]” (p. 216). This is called an emergent research design wherein the original plan

changes as the research unfolds, meaning it is nonlinear (Creswell, 2009) (discussed in Chapter 9). Regard-

less, the end result is data that are then analyzed, interpreted, and discussed, leading to conclusions, impli-

cations, and recommendations (de Vaus, 2001; Suter, 2012; Yin, 1984).

As a final caveat, de Vaus (2001) explained that researchers should not equate a particular logistical method

with a particular research design logic. It is also erroneous to equate a particular research design with either

quantitative, qualitative, or mixed methods approaches. Instead, authors need to bear in mind the link be-

tween (a) the purpose of the research (logical inquiry) and (b) their research design (both logic and logistics)

(Yin, 1984) and then introduce their Methods section accordingly (see examples 8.2 and 8.3).

Example 8.2 Quantitative research design and method This exploratory, quantitative research

inquiry employed a cross-sectional research design. Data were collected from a purposive sample

using the survey method, specifically a piloted questionnaire designed for this study. Descriptive sta-

tistics were used to analyze the data using Minitab software, and the results were reported using

frequencies, percentages, and means (averages).

Example 8.3 Qualitative research design and method This qualitative research inquiry employed

an emergent research design, using the phenomenological method. Data were collected from a

snowball sample of individual participants by way of interviews. The data were thematically ana-

lyzed, and findings were reported using quotes and a supportive narrative.

SAGE

© 2018 by SAGE Publications, Inc.

SAGE Research Methods

Page 8 of 27 Understanding and Evaluating Research: A Critical Guide

Mixed Methods Research Design Logic

Authors of mixed methods studies should avoid rhetorical logic, meaning they should not assume that one

strand of data is only there to embellish their analysis of the other strand and is not really considered to be

a necessary part of their analytical interpretation or argument. Mixed methods explanations and interpreta-

tions require more challenging logics. Mason (2006) identified five logics, one being rhetorical. Parallel logic

assumes each strand has its own logic (see above), and authors would run these in parallel and report two

different sections, one for each strand. A third approach is corroborative logic, which concerns itself with data

triangulation. Researchers would strive to use data from each strand to corroborate each other (confirm or

give support). If researchers use an integrative logic, they likely choose this at the beginning of the research

design process so they can intentionally link insights from both data streams to get a better picture of the

whole phenomenon (see Chapter 10).

Mason (2006) identified multidimensional logic as the most challenging type of mixed methods logic. “The

argument is that different methods and approaches have distinctive strengths and potential which, if allowed

to flourish, can help [researchers] understand multi-dimensionality and social complexity. . . . The logic imag-

ines ‘multi-nodal’ and ‘dialogic’ explanations which are based on the dynamic relation of more than one way

of seeing and researching. This logic requires that researchers factor into their accounts the different ways of

asking questions and of answering them” (pp. 9–10). It differs from the other logics, which assume data inte-

gration rather than a data intersection. The latter “involves a creative tension between the different methods

and approaches, which depends on a dialogue between them” (p. 10). This dialogue cannot occur without

everyone involved embracing a logic that respects multiple dimensions and points of view (researchers them-

selves and research methodologies, with attendant assumptions, as discussed in Chapter 2).

Review and Engagement

When critically reading a research report, you would

□ Ascertain whether the authors used the term research design when introducing their methods, with-

out confusing the two concepts

□ Make sure they shared their thinking and reasoning about how best to answer their research ques-

SAGE

© 2018 by SAGE Publications, Inc.

SAGE Research Methods

Page 9 of 27 Understanding and Evaluating Research: A Critical Guide

tions (i.e., explained the logic used when creating their research design, especially what type of data

were needed to answer their research questions)

□ Per the above, determine if they included a section titled Justification for or Importance of the Study

□ Determine if they properly referred to reconstructed (deductive) logic (quantitative) or logic-in-use

(qualitative emergent research design) or if they referenced mixed methods logics

□ Determine if they clarified their research design (see Table 8.2)

□ Determine if they explicitly stated the type of research inquiry they employed (exploratory, descrip-

tive, explanatory, meaning seeking, or understanding processes and structures)

Most Common Research Designs

Table 8.2 summarizes the most common research designs for each of qualitative, quantitative, and mixed

methods studies, discussed in much more detail in Chapters 9 and 10. These approaches to designing re-

search differ because of methodological distinctions, discussed in more detail in the second part of this

overview chapter.

Table 8.2 Main Types of Qualitative, Quantitative, and Mixed Methods Research Designs

Qualitative Research Designs (in-

volve changing tactics over the

course of the study)

Quantitative Research Designs (involve adhering to a formal

plan with no deviation)

Mixed Methods Research Designs (in-

volve some prioritized combination of

strategy and tactics)

• Interpretive—insights from inter-

preting data change the research

design

• Investigative—traces out a phe-

nomenon in its natural field set-

ting

• Participatory—research design

is codeveloped with participants

• Descriptive—describes what actually exists, as well as its fre-

quency, and then categorizes the information

• Correlational—examines whether a change in a variable (no

manipulation) is related to change in another

• Comparative—measures variables that occur naturally in ex-

isting groups, then compares them to determine their influence

on the dependent variable

• Experimental—manipulates independent variables, measures

• Use qualitative methods to explain

quantitative data (words to explain

numbers)

• Use quantitative methods to further

explain qualitative data (numbers to

explain words)

• Use both methods to achieve trian-

gulation

SAGE

© 2018 by SAGE Publications, Inc.

SAGE Research Methods

Page 10 of 27 Understanding and Evaluating Research: A Critical Guide

Methods

The discussion now turns from the construct of research design to that of methods, which are understood

to include instrument development and apparatus, sampling, data collection, and data analysis, differing for

each of the three methodologies used to shape this book: qualitative, quantitative, and mixed methods. This

chapter provides a generic discussion of methods, followed with Chapter 9 (qualitative methods) and Chapter

10 (quantitative and mixed methods).

Methodology Versus Methods

Many disciplines use the word methodology to refer to methods (Schneider, 2014). This section explains how

this book uses these terms, respecting the lack of a consensus in the scholarly arena for their usage. This

book clearly distinguishes between methodology and methods (see Chapter 2). Methodology (ology) is fo-

cused on what is involved in creating new knowledge and refers to the branch of philosophy that analyzes the

principles and axioms of research. The word method refers to a system of strategies used to obtain informa-

tion for a study.

Many disciplines’ use of the word methodology to refer to methods (Schneider, 2014) most likely occurs be-

cause the empirical (quantitative) research paradigm is so prevalent. Given its dominance, authors tend to

• Illuminative—strategically focus-

es on one aspect of research de- </p

Our website has a team of professional writers who can help you write any of your homework. They will write your papers from scratch. We also have a team of editors just to make sure all papers are of HIGH QUALITY & PLAGIARISM FREE. To make an Order you only need to click Ask A Question and we will direct you to our Order Page at WriteDemy. Then fill Our Order Form with all your assignment instructions. Select your deadline and pay for your paper. You will get it few hours before your set deadline.

Fill in all the assignment paper details that are required in the order form with the standard information being the page count, deadline, academic level and type of paper. It is advisable to have this information at hand so that you can quickly fill in the necessary information needed in the form for the essay writer to be immediately assigned to your writing project. Make payment for the custom essay order to enable us to assign a suitable writer to your order. Payments are made through Paypal on a secured billing page. Finally, sit back and relax.

Do you need an answer to this or any other questions?

Order Plagiarism Free Answer Here

About Wridemy

We are a professional paper writing website. If you have searched a question and bumped into our website just know you are in the right place to get help in your coursework. We offer HIGH QUALITY & PLAGIARISM FREE Papers.

How It Works

To make an Order you only need to click on “Order Now” and we will direct you to our Order Page. Fill Our Order Form with all your assignment instructions. Select your deadline and pay for your paper. You will get it few hours before your set deadline.

Are there Discounts?

All new clients are eligible for 20% off in their first Order. Our payment method is safe and secure.

Hire a tutor today CLICK HERE to make your first order

Related Tags

Academic APA Writing College Course Discussion Management English Finance General Graduate History Information Justify Literature MLA