Chapter 8: Evaluating and Interpreting Information

Learning how to get the most out of your information is so important that it needed its own chapter! Again, I will look at the role that critical thinking plays in the research process, while also assessing dependability in information sources and quality in evidence. You need to be able to interpret your findings accurately and without bias, and you must understand that "certainty" in research is extremely hard to find. You should also be aware of the common errors in reasoning and statistical analysis, and also know that research, by its nature, carries the potential for error.


It is true that not all information is equal-- the same is true for interpretations of that information. Collecting information is probably the easiest part of your research, it's sifting through that information that is the challenge. When evaluating your sources, there are several things to consider. First, you must determine the currency of your sources. Also, you should find the reputation of your printed sources-- see the publication's copyright page to find if the work was published by a university, professional society, museum or respected news organization. Also, check the perspective of your internet or database sources. You should also consider why a given study was funded-- what were there motives? It's also important to cross-check your sources against other similar sources.


You also need to evaluate the evidence that you obtained. Evidence is information that either supports or refutes your claim. First look at the sufficiency of the evidence-- do you have enough? Next, separate your hard and soft evidence. Hard evidence is facts, expert opinions, and statistics, and soft evidence is essentially evidence that cannot be verified. Also look at whether your information is balanced and reasonable, and consider how your facts are being framed.


Interpreting your findings is getting to the truth of the matter, whether or not your research adds to the conclusion you are trying to make. Research can yield three distinct and different levels of certainty. The first is the ultimate truth, which is the conclusive answer. These are the research outcomes that we want to find. The second is the probable answer, or research that stands a change of being accurate and true. The third is the inconclusive answer, which is not a good enough answer for your research!


You should also consider how underlying assumptions will affect your research. Also, watch out for personal biases and rationalization taking the place of reason. Try to look at all angles of interpretation.


Finding proof is normally a process of elimination. In order to rule out bad reasoning, you ought to ask yourself the following questions (Lannon, pg. 159):

• To what extent can these findings be generalized?
• Is Y really caused by X?
• To what extent can the numbers be trusted, and what do they mean?

There are three major reasoning errors that can distort our information. Faulty generalization is jumping form a limited observation to a sweeping conclusion-- even facts that are proven can lead to mistaken conclusions. Casual reasoning explains why something happens in complex situations, but this can go wrong when you ignore other causes, ignore other effects, invent a casual sequence, confuse correlation with causation, and rationalizing. Faulty statistical reasoning yields conclusions that are inaccurate and deceptive. Fallacies include the sanitized statistic ("How well are we doing?"), the meaningless statistic ("How many was that?"), and the undefined averaging (mean, medium, and mode are not the same as an average). There are also the distorted percentage figure ("Is 51 percent a majority?"), the bogus ranking (comparison based on poorly defined criteria), and the confusion of correlation with causation. Also noteworthy are the biased meta-analysis ("Who selected which studies to include?"), the fallible computer model (assumptions are programmed in computer models), and misleading terminology.


You should also not the there are several limitations to research. For example, there are obstacles to validity and reliability in surveys. And there can also be flaws in research studies, including epidemiological studies (studies of various populations), laboratory studies, and human exposure studies (studies that have a control group). The public can also get deceptive reporting on certain scientific findings, because it is true that bad science can make really good news.


To summarize, it is important to evaluate your sources, evaluate your evidence, interpret your findings, and check all of your research for weak spots. There is a lot of bogus information available the more technological advances increase, but this also means that we have greater access to awesome information from very trusted people and organizations. Just be extra careful to triple check your final information-- there is a lot of ways that facts and statistics can be misused and misleading.


Group Discussion Question: Have you ever made a mistake and reported inaccurate findings? How did you correct yourself and what were the consequences?