by John Demchak
It is hard in the information age to read a publication or listen to the news and not hear someone make reference to "a new study just published in the [insert name of journal] shows us that…" These statements are often cited as scientific fact. Science is a nice tool to make general observations that can then be used as a basis to establish general principles and guidelines. BUT, science generally does NOT provide us with scientific "facts".
The internet has provided many people with the ability to research out scientific studies. Unfortunately, all that is generally available is the abstract, a brief summary of the study. Many people (as well as the common media) use these abstracts as statements of fact. But, these abstracts do not provide the information necessary to necessarily permit valid interpretation by the individual reader. A review of the entire study is needed to determine this.
So, how does one tackle the task of reading through a scientific study? First, one should understand how a scientific paper is structured. Then, have an idea of what information each component is meant to convey. With this foundation, one can then look at a complete study with a critical eye.
The components (sections) of a typical scientific paper are:
The introduction gives a background on what is already known about the subject and why this particular study was undertaken. What were the limitations in past studies on the subject? What questions were raised by past research but not adequately answered? In some cases, there can be very limited 'known' information on the subject, just anecdotal evidence, and the study is undertaken to try to impart some scientific 'validity' to the theory that is under consideration.
The methods section will go into detail as to who are the participants of the study, how long was the study, what was measured and how, what factors were controlled, etc. It is this section of the paper that will give the reader some light as to whether the conclusions drawn are applicable to him or her. Usually there is a factor that is manipulated by the researchers (the independent variable) and an endpoint that is measured (the dependent variable). For example, suppose a study was comparing the effect of 3 sets of 8 repetitions versus 3 sets of 3 repetitions on the change in five rep maximum over a two month period for a group of retired, overweight individuals who had no weight training experience. The set and rep strategy would be the independent variable and the change in five repetition maximum would be the dependent variable. Now suppose further that the study concludes that 3 sets of 8 repetitions was more effective. This information may have little generalization to a one repetition maximum and would probably not be very useful to an experienced competitive powerlifter.
Next come the results section that presents the actual raw data. Every participant should be accounted for in the results. In addition to appearing in narrative form, data is generally presented in table and/or graph form as well. Accompanying the raw data will be the statistical analysis of the data using some accepted standard statistical method.
The final section is where the authors discuss the results and
offer their interpretations and the conclusions that they drew from the
data. Some readers find it useful to read the discussion section first,
then go back and see if the data in fact supports the conclusion.
***IMPORTANT POINT***
The authors' interpretations are based on statistical analysis of the data. Therefore, the conclusion drawn isn't even absolutely true for all participants of the study. How can one then say that the conclusion should apply to the population as a whole? Statistics can also skew the actual usefulness of the data. There may be a "statistical significance" but is it significant in real life? For example, suppose analysis of the data reveals that there was a significant difference is the effectiveness of method A vs method B to raise one's bench press. But on actual review of the raw data, it is seen that the average difference was only 5 pounds apart. Is this significant in real life? What if some of the participants actually showed a larger improvement with method A? Should this be discounted?
There is a lot of information being continuously presented in various forms and via various media. Some of it is good, useful information. But, unfortunately a fair amount is not. Hopefully, this brief overview will help you to sort through some of it. It is in no way meant to be an all-inclusive tutorial on reading scientific papers, but is intended rather to serve as a starting point for those who wish to read the literature themselves and not simply trust someone else's interpretation.