How Do You Know That?
A strategy to read and understand primary research papers
Throughout our lives, the public often relies on experts to provide us with the understanding of key issues in order to make decisions. This has overwhelmingly been evident during the SARS-CoV-2, the virus responsible for the COVID-19 pandemic. Although at times, experts and professionals are met with resistance which could be a result of a lack of communication and understanding from the general public.
Anecdotally, I get the impression that science (of any topic) is somehow off limits and too complicated for others to understand. However, this is problematic by 2-fold. First, it suggests that the general public has an inability to understand science. And second, induces the power of authority onto scientists.
What people don’t realize, whether you work at a standard 9 a.m. — 5 p.m. or any other occupation than a scientist, is that the public is fully capable of understanding the fundamental science and have tools available to evaluate the evidence for themselves.
How might you do this?
A great strategy is to go directly to the source. Fundamentally, science is attempting to answer a question or solve a problem, in which the conclusion/solution is supported by evidence provided through rigorous experimentation. This evidence is most commonly provided in academic papers that are published by the laboratories that conduct the experiments.
Here, I’ll provide insight into how you can go about identifying and reading academic papers and strategies to help interpret them.
What is an ‘academic paper’?
The papers that I describe are referred to as primary research articles. That is, a laboratory from a reputable institution conducted experiments to address a particular question or problem and wrote a paper with their interpretations on what they found. These papers are submitted to various journals to be considered for publication and are subject to peer-review.
Peer-review is simply the practice of other scientists in a similar field that are familiar with the subject material, critique the paper on its merits and whether or not it is suitable for publication in that journal. Once accepted, these articles are then available. Others have highlighted some issues with the availability of primary research which is a subject of intense debate, however beyond the scope of this discussion.
To find these primary research articles, useful search engines that are free to use are an excellent resource to get started. Specifically, websites like Google Scholar and Pubmed are great tools and user friendly. These search engines function similarly to basic google, in that generated results are based on keywords in the search bar. The only difference is that instead of various sources ranging from news media, blogs and random sites, the search results are academic papers published in academic journals.
Typically, these papers consists of key sections that form its overall structure.
- Abstract — A brief overview of what the study was about and the key findings. Simply put, it’s a summary.
- Introduction — This section provides some background and useful information about what the paper is trying to address.
- Methods — The methods describe how the experiments were conducted and often include various chemical reagents, software, experimental model, subjects studied, etc.
- Results — Simply put, the results are the actual data brought forth from the authors’ experiments. This is the evidence.
- Discussion — This section is typically reserved for the authors to expand on their results, provide a context as well as (and hopefully) highlight limitations in their work.
- References — This section lists the supporting evidence from other sources the authors identified in the paper. Typically you’ll see small superscript numbers or parentheses with names followed by years within the text.
Once you have a particular interest in a subject or are looking to answer a specific question, now comes time to search, read and interpret the evidence for yourself.
The Strategy
If you have never picked up a primary research article, chances are that reading one will be quite challenging. However, don’t get discouraged, the more you practice and develop your own reading style, the easier it will become. Nonetheless, this reading strategy is based on my own practices and can provide you with a starting point.
First, whatever you do, do not start with the abstract. I repeat, do not start with the abstract.
You likely chose a particular article based on it’s title on the subject you’re curious in learning more about or to answer a specific question. Assuming the title of the paper is not written in a ‘clickbait’ manner, abstaining from reading the abstract ensures that the reader does not begin the article knowing the key findings and more importantly, the authors’ potential interpretation of those findings. Thus, the reader can maintain a critical eye in their reading with limited bias.
Typically, you want to begin reading the introduction. Here you can be provided much of the background information on the topic or problem the paper seeks to answer. As such, the background will also provide with it a reference, either identified by a number or names/years depending on the citation style. If the subject is unfamiliar, often times referenced papers may require more reading to understand the paper you initially chose. Also, the papers cited in the reference section provide more breadth and sources to further reading.
Strategy #1: Circle the references in the introduction text you deem important as well as circle/highlight them in the reference section at the end of the article. This will help you identify papers that may support your understanding of the current paper, highlight conflictions, or more breadth on the subject later on.
As you continue to work through the introduction, highlight or underline key statements and scribble notes on the margins. This is a learning process and it will be easier to remember if you come back to the paper later on.
Toward the end of the section, you should have a good understanding of what the problem(s) and/or central research question is. Additionally, it’s important to identify what the hypothesis/hypotheses are. This is critical so that you understand the scope of what the paper is answering and determining whether or not the data will answer that question.
Strategy #2: Write down the key problem or research question the paper is attempting to answer. Along with it, identify the hypothesis (the testable statement on whether something is different or not).
Unlike the typical progression of the paper sections (introduction, materials & methods, results), after I read the introduction and have a thorough understanding of the research question(s), I immediately go the the results section.
The results are typically presented with data through the form of figures with tables and/or graphs. Because the written results section are the figures described through text, I evaluate all of the figures first. In doing so, I can remain critical of what is being presented and depending on what the data shows, make my own interpretation on that data prior to the author(s).
Each figure has a corresponding description of the dependent measures, groups, experimental manipulations, etc. allowing the reader to solely interpret the data without the written results. Often times, questions are generated about how the authors’ approached gathering the data and thus, I typically refer back to the Materials & Methods section. This practice, in my opinion, allows for an inner dialogue and conversation about how the data was gathered and achieve a greater understanding of what the numbers mean, rather than reading the Methods & Materials section independently.
Strategy #3: Read and interpret the figures in the results section, referring to the Materials & Methods section to understand the approach in which the data was generated.
Furthermore, within the paper’s figures, you may notice strategically placed symbols, identifying key groups/timepoints/etc. These symbols are often described in the the figure’s description and noted as representing statistical significance. Although statistics and statistical testing is a topic on its own, the highlighted statistical significance highlights the differences observed between control groups and our manipulated variable/group is not due to random chance.
Importantly, be weary of readily accepting statistical significance as meaningful. Again, it is up to you as the reader to determine whether that significant difference is practically significant. Just because the burden of statistical significance can be achieved, does not give it the practical significance.
Strategy #4: Take note of the statistically significant findings and determine whether the highlighted differences are meaningful in the particular field that paper resides.
As you navigate through all of the figures of the paper, be sure to make your own interpretations about what the numbers mean and what story all of the data form. Thus, you form your own interpretation about the data and are intimately familiar with the specific findings. After, read the written portion of the Results section which likely include the author(s) interpretations that may be similar or contrary to your own.
Strategy #5: Read the written portion of the Results section after forming your own interpretations of the presented data. Determine if you agree or disagree with the author(s) on what they found and what it means.
Lastly, the Discussion section of the paper is reserved for the author(s) to expand on the findings and how it fits into the general body of literature. As you read through the Discussion, remember to keep in mind your own interpretations of the data and the potential questions you may have that could be addressed or expanded on. This is also another area that will incorporate more papers listed in the Reference section that may not have otherwise been cited in other areas (i.e. Introduction). Utilizing the similar practice as before, highlight or circle those references in the text as well as the Reference section so that you can go back and find those papers to read later on.
Strategy #6: List your interpretations of the data as well as any questions you have that may be answered in the discussion. Be sure to note any references that are cited that may be of interest.
A good, well written discussion will also highlight the limitations of the paper and it’s experiments. There is no perfect study and without knowing the limitations of the findings/experiments disregards our ignorance. In doing so, determine if you agree with the noted limitations and if you also find limitations that the authors did not state.
Another important aspect of the discussion is to identify areas of further investigation or what is potentially to be done next, based on what the paper found. Again, this is an area to determine whether you may agree with the future direction or if there is another line of questioning, based on the data, that further work could be explored.
Strategy #7: Identify any limitations or weaknesses the paper has and next line of questions or routes of investigation, based on the paper’s findings.
Things to Remember
1. Read With A Purpose
Fundamentally, you chose a particular paper for some reason. Whether to gain more knowledge/understanding or to address a particular question. A good practice is to always remember why you are reading this paper.
2. A Paper Copy
We live in a digital age, but I find that having a paper copy that you will always have and having the ability to make notes/highlights in its margins makes a better connection with the reader and can be readily available later.
3. Read Critically and Creatively
Remember that we all have biases but you ultimately want to generate your own conclusions. Therefore, be skeptical of the author(s) interpretations as well as your own. What could be missing in the paper? Are the conclusions supported by the data? Was the research question answered?
Additionally, think creatively. It is much easier to break things down than to build them up. Try to extend what applications the paper’s findings have. What the possible improvements could be or what could be done next. This can help generate more questions and allow you to have a more thorough understanding of the subject material and be more critical in general.
4. A Reputable Source
Scientists and research personnel are as fallible as the general public. Most often, their work is financially supported through funding agencies either public (i.e. government) or private companies, that have invested interest in their work. Just before the Reference section of a paper, there are typically brief mentions of ‘funding support’, revealing the specific sources of funding. A good practice is to note these sources, and especially if they are from private companies whose business resides in the topic being researched (e.g. pharmaceutical company on paper researching a company’s drug, fossil fuel companies on a climate paper, tobacco company on lung health). Importantly, despite these funding sources, you should not excuse paper outright, rather the data and conclusions should be interpreted with greater scrutiny.
5. The Take Home Message
After you finish the paper, try to formulate no more than two sentences about what the overall conclusion is from the paper. This 30,000-foot view can help provide context and help solidify your understanding.
In conclusion, this reading strategy is a starting point to begin reading and interpreting primary research papers. More strategies will develop based on your own reading style. Know that sometimes it may take multiple reads of a single paper as well as reading other papers to fully grasp the content and it takes time! The general public can and should be able to generate their own conclusions and not solely rely on experts. Thus creating a scientifically literate populace and improving our society as a whole.
