How to read research, understand statistics, and spot misinformation.
Why This Matters
Being scientifically literate helps you:
- Make informed health decisions
- Avoid being misled by false claims
- Understand news about science
- Evaluate product claims
- Discuss science-based topics intelligently
- Protect yourself and family from misinformation
Understanding Research
Types of Studies
| Study Type | Description | Strength |
|---|
| Case report | Single patient observation | Weakest; suggests hypotheses |
| Case series | Multiple similar cases | Still weak; patterns only |
| Cross-sectional | Snapshot of population at one time | Shows association, not causation |
| Case-control | Compare people with/without condition | Moderate; retrospective |
| Cohort | Follow groups over time | Strong; prospective |
| Randomized controlled trial (RCT) | Random assignment to treatment/control | Gold standard |
| Systematic review | Combines multiple studies | Very strong |
| Meta-analysis | Statistical combination of studies | Strongest evidence |
Hierarchy of Evidence
| Level | Evidence Type |
|---|
| Highest | Systematic reviews and meta-analyses |
| High | Randomized controlled trials |
| Moderate | Cohort studies |
| Lower | Case-control studies |
| Low | Case series, case reports |
| Lowest | Expert opinion, anecdotes |
Study Quality Indicators
| Good Sign | What It Means |
|---|
| Peer reviewed | Experts checked the work |
| Large sample size | More reliable results |
| Control group | Comparison baseline |
| Randomization | Reduces bias |
| Blinding | Prevents expectations from affecting results |
| Replication | Others got same results |
| Pre-registration | Methods declared before study |
Statistics Basics
Key Concepts
| Term | Meaning | Example |
|---|
| Mean | Average | Sum of values divided by count |
| Median | Middle value | Half above, half below |
| Standard deviation | Spread of data | How much values vary |
| Sample size (n) | Number of subjects | Larger = more reliable |
| Confidence interval | Range of likely true values | "True value is between X and Y" |
P-values and Significance
| Term | Meaning |
|---|
| p-value | Probability results happened by chance |
| p < 0.05 | Less than 5% chance of random occurrence |
| Statistically significant | p-value below threshold (usually 0.05) |
| Clinically significant | Effect large enough to matter |
Important: Statistical significance does not mean the effect is large or important.
Understanding Risk
| Term | Definition | Example |
|---|
| Absolute risk | Actual probability | 2 in 100 people affected |
| Relative risk | Comparison between groups | "Doubles your risk" |
| Number needed to treat (NNT) | How many need treatment for one to benefit | NNT of 100 = treat 100 for 1 benefit |
| Number needed to harm (NNH) | How many treated before one harmed | Risk-benefit ratio |
Misleading Statistics
| Tactic | The Problem | Example |
|---|
| Relative risk without context | Sounds dramatic | "Doubles risk" (from 1 to 2 in 1000) |
| Cherry-picking time frames | Choose data that supports claim | Start graph at convenient point |
| Confusing correlation/causation | Association isn't cause | Ice cream and drowning |
| Small sample sizes | Random variation | 3 out of 4 dentists |
| Missing confidence intervals | Uncertainty hidden | Single number sounds precise |
Reading Science News
Translation Problems
| What Headlines Say | What It Often Means |
|---|
| "Scientists say" | May be one small study |
| "Breakthrough" | Incremental progress |
| "Linked to" | Correlation, not causation |
| "May cause" | Possible but unproven |
| "Experts warn" | Some experts, maybe fringe |
| "Study proves" | Studies suggest, rarely prove |
Questions to Ask
| Question | Why It Matters |
|---|
| Who did the study? | Conflicts of interest? |
| Where was it published? | Reputable journal? |
| How big was the sample? | Small = less reliable |
| Was it in humans? | Mouse studies often don't translate |
| Was it replicated? | Single studies are preliminary |
| What do other experts say? | Consensus matters |
| What's the effect size? | Is the difference meaningful? |
Source Quality
| More Reliable | Less Reliable |
|---|
| Peer-reviewed journals | Press releases |
| Meta-analyses | Single studies |
| Expert consensus | Individual expert |
| Science reporters (specialists) | General news headlines |
| Primary sources | Social media summaries |
Warning Signs
| Red Flag | Why Suspicious |
|---|
| "Miracle cure" | Medicine doesn't work that way |
| "Scientists don't want you to know" | Real science is published |
| "Natural therefore safe" | Many natural things are dangerous |
| "Ancient wisdom" | Old doesn't mean true |
| Testimonials as evidence | Anecdotes aren't data |
| Appeal to celebrity | Fame isn't expertise |
| "Just asking questions" | Often hides agenda |
| "Do your own research" | Implies conspiracy |
| Extreme certainty | Real science acknowledges uncertainty |
| "Suppressed" information | Usually means rejected for good reason |
Logical Fallacies
| Fallacy | Description | Example |
|---|
| Ad hominem | Attack the person, not the argument | "He's funded by pharma" |
| Appeal to nature | Natural = good | "Chemicals are bad" |
| Appeal to authority | Famous person says so | Celebrity endorsement |
| Straw man | Misrepresent opponent's position | "Scientists think we came from monkeys" |
| False dilemma | Only two options presented | "Believe me or trust corrupt science" |
| Anecdotal evidence | Personal story as proof | "My aunt was cured by this" |
| Moving goalposts | Changing standards when met | Always demanding more proof |
Pseudoscience Markers
| Pseudoscience | Real Science |
|---|
| Unfalsifiable claims | Testable predictions |
| Immune to criticism | Welcomes scrutiny |
| Relies on testimonials | Relies on data |
| Cherry-picks evidence | Considers all evidence |
| Unchanging beliefs | Updates with new data |
| Conspiracy theories | Open publication |
| Personal attacks on critics | Addresses criticisms |
Specific Topics
Health Claims
| Claim Type | What to Look For |
|---|
| Supplement claims | FDA doesn't verify; "may support" is vague |
| Diet claims | Often based on weak evidence |
| "Detox" products | Your liver and kidneys already detox |
| "Boosts immunity" | Usually meaningless |
| Cancer cures | Be extremely skeptical |
| Good Source | Poor Source |
|---|
| Medical journals | Health blogs |
| Major health organizations | Social media posts |
| Your doctor | Celebrities |
| Cochrane reviews | Product websites |
Climate Science
| Claim | Evaluation |
|---|
| "Climate has changed before" | True, but current change is different |
| "Scientists disagree" | 97%+ consensus on human causation |
| "It's the sun" | Solar activity doesn't explain recent warming |
| "Models are wrong" | Models have been remarkably accurate |
Vaccines
| Common Claim | Reality |
|---|
| Cause autism | Large studies show no connection |
| "Too many, too soon" | Immune system handles far more |
| Natural immunity better | Risks of disease far outweigh vaccine risks |
| Contain dangerous ingredients | Ingredients are safe at given doses |
Practical Application
Before Sharing
| Step | Action |
|---|
| 1 | Check the source - is it reputable? |
| 2 | Read beyond the headline |
| 3 | Check the date - is it current? |
| 4 | Look for the original study |
| 5 | See what experts say |
| 6 | Check fact-checking sites |
Fact-Checking Resources
| Resource | What It Covers |
|---|
| Snopes | General claims |
| PolitiFact | Political claims |
| FactCheck.org | Political claims |
| Science-Based Medicine | Health claims |
| Climate Feedback | Climate claims |
| Full Fact | UK-focused |
Talking to Others
| Approach | Why It Works |
|---|
| Ask questions | Less confrontational |
| Acknowledge valid concerns | Shows respect |
| Focus on shared values | Find common ground |
| Provide context | Help them evaluate claims |
| Be patient | Beliefs change slowly |
Updating Your Own Beliefs
| Practice | Benefit |
|---|
| Seek disconfirming evidence | Counter confirmation bias |
| Notice when you're defensive | Signal to examine more closely |
| Change your mind publicly | Models intellectual honesty |
| Say "I don't know" | Honest about uncertainty |
| Consider the opposite | Challenge your assumptions |
The Scientific Consensus
What Consensus Means
| Aspect | Explanation |
|---|
| Expert agreement | Majority of qualified experts agree |
| Based on evidence | Conclusions follow from data |
| Can change | Updates with new evidence |
| Not a vote | Weight of evidence matters |
| Doesn't mean certainty | But reflects best current understanding |
When to Trust Consensus
| Trust Consensus When | Be More Skeptical When |
|---|
| Field has been studied extensively | Topic is new or controversial |
| Experts have relevant credentials | "Experts" are outside their field |
| Multiple lines of evidence converge | Based on limited data |
| No major competing theories | Legitimate scientific debate exists |
| Predictions have been validated | Predictions haven't been tested |
Key Takeaways
Not all studies are equal - Hierarchy of evidence matters; meta-analyses beat anecdotes
Correlation is not causation - Just because two things happen together doesn't mean one causes the other
Sample size matters - Larger samples are more reliable
Relative risk can mislead - Always ask for absolute numbers
Headlines exaggerate - Read the actual study or expert analysis
Extraordinary claims need extraordinary evidence - Be proportionally skeptical
Check multiple sources - No single source is always right
Consensus exists for good reasons - Widespread expert agreement is meaningful
Real scientists express uncertainty - Overconfidence is a red flag
You can update your beliefs - Changing your mind with new evidence is strength, not weakness