Resources

The temptation is great. When you have a doubt about a health issue that worries you, with just one click you can access Dr. Google, Dr. TikTok, or Dr. Instagram. Nowadays, we can find many posts, reels, stories, videos, or even WhatsApp or Telegram messages that talk to us about a health topic and also assure us that they are based on a scientific article or study. This is often seen as a quality seal, and we tend to believe everything that is said. But, beware! Before taking these contents as valid information, we must ask ourselves: Does this study come from a prestigious scientific journal or from a predatory journal? Is it a study applied to animals or humans? Does it have enough evidence? In this article, we provide you with the keys to correctly interpret a scientific study.

Did you know that one of the most common problems is the misinterpretation of scientific studies? On social media, we find information about “miracle diets,” “superfoods that cure everything,” “secret medicines,” “miraculous treatments,” “harmful effects of a drug that are kept secret,” etc. In many cases, this misinformation or erroneous information is associated with incorrect or misinterpreted scientific articles, which creates confusion. Sometimes, this false or incorrect information can put our health at risk.

The well-known fake science includes false, misleading, or erroneous information related to science. Alexandre López Borrull places under the umbrella of fake science pseudoscience, predatory journals, and retracted articles. This professor of Information and Communication at the UOC states that fake science comes from both users and the scientific community itself and is, in his words, “the emerging consequence of the commercialization of science.”

From the GRECC-UPF research group in scientific communication, Professor Sergi Cortiñas defines pseudoscience as “a set of practices that try to appropriate the status of science and the scientific method, by exposing isolated, non-demonstrable, or purely subjective results, obtained in an uncritical, unsystematic, or biased manner, to generate knowledge that is neither cumulative nor scientifically valid, often operating in areas where science does not reach or has not provided satisfactory results, and is skilled at using the media and public emotions.”

In addition to pseudoscience, errors in retracted scientific articles or the misinterpretation of correct scientific articles can also lead to confusion, misinformation, and decisions about our health that are not based on evidence. Therefore, let’s go over some tips for correctly interpreting studies and news about health and science.

If it has been published in a predatory journal, you should be cautious. These are publications in which no review or validation has been conducted by at least two reputable experts in the field. Predatory journals charge scientists money to publish, but then they are not indexed to count towards their academic career. Therefore, we can say they scam scientists and deceive the public. If the study you are reading has been published in a reputable scientific journal, you should ask yourself the questions that you will find below.

We can classify scientific studies according to the phase of the editorial process they are in:

• Preprint: These are documents that have not been peer-reviewed or accepted for publication in a scientific journal. Therefore, their scientific validity and veracity are not guaranteed. They should not be considered valid articles, and their conclusions should be assessed with caution. To know if an article is a preprint, you can search for it in the MedRxiv tool.

• Retracted: These are documents that have been removed from the scientific literature due to issues with reproducibility, scientific validity, or even the veracity of the data and conclusions presented. Any article with this definition should not be considered. A useful tool to find out if an article has been retracted or corrected is Retraction Watch, where you can check the information in their database (Retraction Watch Database). A paradigmatic example of a retracted article is the study by Dr. Andrew Wakefield published in The Lancet, which claimed that the MMR vaccine increased the risk of autism. This invalid research, however, still fuels the anti-vaccine movement. The Lancet retracted the article for several reasons. One was errors in the sample: only 12 children were studied, and they did not come, as Wakefield claimed, from the gastroenterology department of Royal Free Hospital in London. They were instead selected non-randomly by a law firm handling a case against the vaccine manufacturers. Additionally, an investigative report discovered that Wakefield paid the children. The researcher lied about the results, used incorrect methods, and had a conflict of interest, as he wanted to patent another vaccine and discredit the MMR vaccine. The article was retracted, and Wakefield was banned from practicing in the UK, but the confusion still persists today.

• Published Article: If the article has been published following a peer-review system, it can be considered valid. However, it is still important to ask further questions to analyze its conclusions and interpret it correctly.

Determining the type of study design sometimes seems to be the most complicated, but it is important to know that researchers always indicate it in their articles. Saying this is a way for the rest of the scientific community to understand the strengths and weaknesses of their study.

But, even more importantly, it allows the public who learns how to search for and identify the type of design to quickly know whether this study is valid or not to support claims about human health. We can classify scientific studies into two main categories:

• Descriptive studies: They describe the characteristics or phenomena observed in a population or sample. There is no manipulation of variables. Causal relationships cannot be established. The results help to understand and contextualize the studied phenomena. For example, if a survey is conducted on a specific population about the hours spent playing video games and insomnia, the results that can be drawn are, at best, an association between these two elements. It could be said that people who play a lot of video games sleep little. But it could not be said, from this study, that playing video games causes insomnia because the study design does not allow for a cause-effect relationship to be established.
• Analytical studies: Their goal is to analyze the relationships between variables and establish causal or comparative inferences in a population or sample, using statistical methods. Data is collected through different methods, such as controlled experiments, surveys, or cohort studies, and analyzed using statistical methods. Biases and variables are controlled. The results of these studies allow for stronger conclusions about the relationships between variables. Here is an example of a news story about a drug used to treat Parkinson’s that has positive effects on ALS patients, associated with the cited study. There are many types of descriptive and analytical studies. We explain them here:

You need to know which type of study you are reading and interpret it correctly. Where can we find this information?

One of the most common mistakes is applying the conclusions of a study developed in cellular and/or animal models to humans. This is not correct. In fact, the percentage of drugs tested on animals/cell cultures that end up being suitable for therapeutic use in humans is about 1% of the molecules tested. Although these conclusions cannot be extrapolated, sometimes this is used with clickbait and generates confusion.

Once we know what the sample is—whether it’s animals, cells, or people—we need to ask another question: How was this sample obtained? In science, one of the most important parts is how the participants for the study are selected. There are two ways to select them:

• Randomly, known as probabilistic sampling methods.
• Non-randomly, known as non-probabilistic sampling methods.

As you can imagine, the most powerful method, and the one that will allow the results to be applied to the population, is studies that have selected participants randomly. If the study was conducted non-randomly, the conclusions of the study cannot be applied to the entire population, but only to the sample, meaning the people who participated in the study.

The applicability and relevance of scientific information is based on the level of evidence it provides. In this sense, if someone presents an observational study indicating that e-sports increase anxiety, but there is a controlled clinical trial that says the opposite, the “deciding” evidence is the controlled clinical trial. So, if we find various scientific articles on the same topic, how can we know which one is more important? Here, the hierarchy by level of evidence applies.

Finally, as a summary on how to approach articles that may appear in health news, here is a decision “algorithm” on how to decide if a document is really suitable to support what is said in the news we receive through different media.

Remember, to correctly interpret scientific or health information you receive through social networks, you should always go to the original source and ask yourself all these questions: Where was the study published? Is it scientifically validated or is it a pre-print or a publication from a predatory journal? What type of study is it? What is the sample? What evidence does it provide? It is important to stop and carefully analyze and think about these answers before making a health decision based on incorrect or confusing information.