JUNE 1997. VOL 65, NO 6

R E S E A R C H C O R N E R

An overview of statistical significance in nursing

Editof's note: This column in /he Journal highlights reseorch issues related to peri- operative nursing practice. The outbors, AORNs codirectors of perioperative reseorch, provide practical advice for reading, conducting, and using reseorch in perioperative nursing pmctice.

esearchers talk about sturisri- cul signrficuizce, and clini- R cians talk about clirzical sig-

rz$cutzce. Both terms often are used in research reports, and it is important for perioperative nurses to understand how these terms are similar and different. Although these terms sound alike, they are not interchangeable concepts.

QUANTITATIVE RESEARCH DATA

researchers collect data that are numerical (eg, physiologic mea- sures, test scores, ratings). The people who provide this informa- tion are the subjects, and the group of study subjects makes up the study sample. The sample is a subset of the population of inter- est (ie, all possible subjects who meet the study criteria). After the researchers collect the data, they analyze them using a variety of statistical procedures (eg, t tests, analysis of variance, multiple regression). The researchers use the data analysis results to make inferences (ie, decisions) about the population of interest based on the results obtained from the study sample.

The basic premise of statisti- cal testing is that the study sam- ple is representative (ie, is a theo- retical distribution) of the popula-

In a quantitative study.

tion of interest. The researchers may make decision errors if the sample does not reflect the popu- lation of interest. Statistical tests are only as good as the data that are analyzed. If the data are flawed, the statistical test results also may be suspect.

During the planning phase of a study, the researchers must make two important decisions. The first decision is to formulate the null and research hypotheses. and the second is to establish the level of statistical significance (ie, the alpha [a] level). Both of these decisions are important for the process known as hypothesis test- ing, which is the basis of quanti- tative statistical analysis.

Hypothesis formulation. In research that is based on hypothe- sis testing, nothing is ever proven. This often is a point of confusion for novice researchers or new readers of research. In a quantitative study, the researchers formulate a null hypothesis and a research (ie, alternative) hypothe- sis. The null hypothesis states that there is no relationship between the variables of interest in the study, whereas the research hypothesis states that there is a relationship between these vari- ables. The research hypothesis is

SUZANNE C. BEYEA, RN, CS, PHD. is AORN codirec tor ofperioperutiv reseorrli.

LESLIE H. NICOLL, RN, MBA, PHD, is AORN codirector of'perinperutii7e research.

and clinical research based on previous research, sci- entific principles, and the researchers' knowledge and expertise. Based on the statistical test results, the researchers either accept or reject the null hypothe- sis (ie, if the results demonstrate that there is no difference between variables, the null hypothesis is accepted). If the null hypothesis is rejected, the researchers surmise that some- thing else is true (eg, the relation- ship between variables that is stated in the research hypothesis actually exists). Just because the null hypothesis is rejected, how- ever, does not mean that the research hypothesis is proven.

level of significance. The level of significance establishes the risk that the researchers are willing to take when testing the null hypothesis (ie, the probabili- ty of rejecting a null hypothesis that really is true). Researchers traditionally set the a level at .05, although some researchers use more stringent (ie, .Ol ) or more relaxed (ie, .lo) a levels. When setting the level of significance, the researchers are determining how crucial it is to have accurate results. When researchers set the a level at .05, it means that they are willing to be wrong five times out of 100 when rejecting the null hypothesis. If researchers set the a level at .01, it means that they are willing to be wrong only one time out of 100.

It is important to remember that all statistical testing is based on the concept of probability.

1128 AORY JOL'RhUAL

JUNE 1997, VOL 65, NO 6

When researchers set an a level at .05 and the study results indicate that the null hypothesis should be rejected, the researchers make the following conclusion: There is a 95% chance that these results do in fact reflect what is happening in the population of interest, and there is a 5% chance that these results occurred by chance alone and the null hypothesis actually is true.

Types of errors. In hypothesis testing, researchers can make two types of errors (Table 1). A type I error occurs when researchers reject a null hypothesis that actual- ly is true. A type II error is just the opposite (ie, researchers accept a null hypothesis that actually is false). Researchers are most con- cerned with making type I errors (ie, concluding that a difference exists when there is none).

STATISTICAL SIGNIFICANCE During the process of hypoth-

esis testing, researchers perform the statistical test and calculate a probability ( P value), and com- pare that value to the a level. If the P value is less than the a level, the researchers reject the null hypothesis. If the P value is greater than the a level, the researchers accept the null hypothesis. Researchers often write, “The probability was less than .05 (P < .05),” to describe a

Table 1

result in which the null hypothe- sis was rejected.

Researchers also may calcu- late an actual probability, which can be reported; however, they still compare this figure to the a for decision-making purposes. In the research article “Effect of sur- gical hand scrub time on subse- quent bacterial growth” that appears in this issue of the Jour- nal, the researchers calculated that the actual probability was .02, which is less than the level of .05 that they chose; thus, they rejected the null hypothesis.’

CLINICAL SIGNIFICANCE When researchers conclude

that statistical significance exists, this means that there is a very low probability that the findings occurred by chance alone. In sig- nificance testing, the word signif- cant does not mean that the results are important or that they have clinical or practical use. A study with nonsignificant statisti- cal results still may be helpful in understanding the lack of a rela- tionship between a group of vari- ables. Statistically nonsignificant results also may help researchers discover study design or measure- ment flaws. Regardless of what researchers conclude about the statistical tests used in a study, readers and consumers of research always must consider the question

POTENTIAL OUTCOMES OF STATISTICAL DECISION MAKING

“What is the practical or clinical Significance of this research?”

There are no statistical tests to help readers and consumers of research determine the clinical significance of study results. We must use our critical thinking skills and clinical expertise to study and interpret research results, and then we need to decide whether the statistical results have any relevance to our clinical practice settings.

Whereas statistical signifi- cance relates to group differences or effects, clinical significance is more important at the individual patient level. For example, researchers might design a study to evaluate a weight-loss inter- vention. If subjects in the inter- vention group lost an average of 20 lb (9 kg) and subjects in the nonintervention group lost an average of 15 lb (6.8 kg), the 5-lb (2.27-kg) difference between the two groups in this hypothetical study would not be statistically significant. The intervention, however, might be clinically sig- nificant to individual patients if the subjects in the intervention group reported an improved sense of well-being and quality of life.

In contrast, other researchers might study oxygen saturation differences related to patients’ body positions (ie, supine, sit-

ting). Mean oxygen satura- tion in the supine posi- tion might be

The actual situation in the population is that the null hypothesis is: g6%, com- true false pared to 99%

in the sitting position. When tested statistically,

The researcher concludes that the null hypothesis is:

true (accepted) correct decision lype I 1 error

false (rejected) lype I error correct decision this difference

1129 AORN JOURNAL

JUNE 1997. VOL 65. NO 6

might be statistically significant ( P < .05), but the difference might not be clinically significant for individual patients.

INTERPRETING RESEARCH When reading and interpreting

research reports, it is important to assess the relevance of the find- ings, regardless of whether they are statistically significant. Nurs- es can decide what constitutes clinically important research results by considering what dif- ference the findings make to indi- vidual patients and also the costs involved in achieving the out- comes. If the costs are high and the benefits are minimal, statisti- cally significant interventions may not be cost-effective or clini- cally significant.

The research article in this issue of the Jouriial illustrates some of the challenges in inter- preting statistical and clinical sig- nificance. The authors state,

Although the mean hacteri- al count differed signifi- cantly (P = .02) between the two-minute and three- minute surgical hand scrub times, it fell below 0.5 log, which is the threshold for practical and cliriicol significance.?

Some practitioners may be tempted to adopt this study’s findings in clinical practice. The authors, however, recommend

N O T E S

that future studies be conducted to test the clinical importance of bacterial reductions achieved with various surgical hand scrub times, using the 0.5 log reduction benchmark.’ Changes in surgical hand scrub times should not be

When reading and interpreting

research, it is important to assess the relevance of the

findings.

made based on one study. Although this study’s results

are statistically significant, their clinical significance is a complex issue. The study was conducted in one institution with a relative- ly small sample. The authors include evidence (ie, power analysis) that the sample is repre- sentative of the population of interest; however, the population in this case was the perioperative RNs and surgical technologists (STs) at the authors’ institution, not the entire US population of perioperative RNs and STs. The results, therefore, cannot be gen- eralized to another institution unless the study sample is similar

2. Ibid.

to the population of RNs and STs in the other institution. The authors suggest two practice changes, but they note that these changes should occur only if additional studies using other surgical hand scrub agents sup- port their finding^.^ These authors clearly are aware of the importance of differentiating between statistical and clinical significance.

This study presents new find- ings and raises interesting ques- tions. Statistical significance should never be the sole reason to change clinical practice; clini- cal significance also must be considered. Readers of research always should ask the question “So what?” For example, if there is a statistically significant differ- ence between patients’ oxygen saturations in the supine and sit- ting positions, so what? If this difference is only 3%, is it clini- cally significant? The same ques- tion applies to the surgical hand scrub time study results de- scribed previously. The mean bacterial count differed signifi- cantly between the two scrub- time groups, but the clinical sig- nificance of this difference is not clear. Such issues require further study to clarify the clinical sig- nificance of the findings. A Readers who have questions and ideas about perioperative nursing research are encouraged to caN AORN’s codirectors of perioperative research ot (800) 755- 2676 x 8277

1, S M Wheelock. S Lookinland, “Effect of surgical 3 . Ihid. 4. h i d . hand scrub time on subsequent bacterial growth,” AORN

Journal 65 (June 1997) 1087-1098.

1130 AORN JOURNAL

Our website has a team of professional writers who can help you write any of your homework. They will write your papers from scratch. We also have a team of editors just to make sure all papers are of HIGH QUALITY & PLAGIARISM FREE. To make an Order you only need to click Ask A Question and we will direct you to our Order Page at WriteDemy. Then fill Our Order Form with all your assignment instructions. Select your deadline and pay for your paper. You will get it few hours before your set deadline.