How We Grade Supplements: Our Methodology
Ever wondered why we give one supplement a glowing 8/10 while another barely scrapes by with a 2/10? You're not alone. As consumers ourselves, we believe you deserve complete transparency about how we evaluate the pills, powders, and potions flooding the supplement market. At Snake Oil Science, our scoring isn't based on marketing hype, popularity, or personal preference. Instead, we've developed a systematic, evidence-based approach that puts science front and center. Let's pull back the curtain on our 10-point grading system.
Our Three-Pillar Evaluation System
Each supplement we review is assessed across three key dimensions, with each receiving its own score out of 10. These scores are then averaged to create our final overall rating:
1. Efficacy in Clinical Trials (0-10 points)
This dimension evaluates how well a supplement performs in scientific research. We thoroughly analyze the available studies, with special attention to:
Randomized Controlled Trials (RCTs) — the gold standard of supplement research
Meta-analyses and systematic reviews that combine data from multiple studies
Study quality — sample size, methodology, placebo controls, and blinding
For example, when evaluating turmeric for joint pain, we examine all relevant clinical trials measuring pain reduction and improved mobility in humans—not just test tube studies or animal research.
A supplement backed by multiple high-quality trials showing significant benefits might score 8-10 here. Products with only theoretical benefits, contradictory results, or studies with serious methodological flaws will receive lower scores, perhaps 2-4 points.
2. Expert Consensus (0-10 points)
Science doesn't exist in isolation, so we also consider what the broader medical and scientific community says about each supplement.
We research:
Position statements from major health organizations
Clinical practice guidelines
Expert panel recommendations
Reviews in respected medical journals
For instance, omega-3 fatty acids score relatively well in this category because several medical organizations acknowledge their benefits for specific conditions like high triglycerides.
Sometimes the research is ahead of consensus—promising new studies might exist, but guidelines haven't caught up. We factor this in, giving middle-range scores when the scientific tide appears to be turning.
Scoring a perfect 10 here is rare and would require widespread acceptance among medical professionals—something few supplements achieve outside of essential nutrients and well-established therapies.
3. Benefit to Average Consumer (0-10 points)
Evidence and expert approval are crucial, but what about real-world impact? This dimension evaluates practical questions:
Is the effect size meaningful or negligible?
Does the supplement have a strong safety profile?
Are quality and consistency issues common?
Is it reasonable to use (dosing, cost, availability)?
Will most people notice a difference?
A weight loss supplement proven to work but only helping users lose an extra pound over six months would score low here despite having evidence. Similarly, a supplement with solid efficacy but serious side effects would receive fewer points in this category.
We also consider product quality issues. Some supplement categories are notorious for contamination or mislabeling. Others have established quality certification programs like USP Verification. These factors influence whether consumers can expect consistent, safe results.
Our Overall Score
After evaluating each dimension, we simply average the three scores to determine the final rating. This balanced approach ensures that a supplement must perform reasonably well across all areas to receive a high overall score.
Understanding Our Scale
Here's what those final numbers actually mean:
9-10/10: Exceptional — Rock-solid evidence, strong expert endorsement, and clear benefits with minimal downsides. Few supplements reach this level, but examples include folate for preventing birth defects in pregnancy or vitamin D for people with documented deficiency.
7-8/10: Strong — Good scientific support and generally positive expert opinion, though perhaps with some limitations or specific conditions for effectiveness. These supplements deliver meaningful benefits for their intended use.
5-6/10: Moderate — Mixed or preliminary evidence. These supplements might help some people under certain circumstances, but results aren't guaranteed. Often includes cases where further research is needed or where benefits exist but are modest.
3-4/10: Weak — Limited scientific support. Typically based on theoretical mechanisms, anecdotal reports, or a few low-quality studies. The jury is still out, leaning toward ineffective.
1-2/10: Poor — Virtually no scientific support or actively disproven claims. Often accompanied by safety concerns or quality issues. Your money is better spent elsewhere.
A Living Evaluation
Science evolves, and so do our ratings. When new research emerges, we revisit our scores. Many supplements have seen their fortunes rise or fall as better studies become available.
For instance, antioxidant supplements like high-dose vitamin E were once thought promising for heart health based on observational studies. But as rigorous clinical trials emerged showing no benefit (and potential harm at high doses), their scores dropped accordingly.
Why Our Approach Matters
The supplement industry operates with far less regulation than pharmaceuticals. Claims often outpace evidence, and consumers can find themselves navigating a sea of contradictory information.
Our methodology aims to be the compass you need—pointing toward supplements with legitimate benefits while steering you away from expensive disappointments or potential health risks.
We're not anti-supplement. In fact, many supplements score well in our system! But we believe consumers deserve to know when a product's marketing outshines its scientific support.
By understanding our grading system, you can make more informed decisions about which supplements deserve a place in your cabinet—and which ones are little more than modern snake oil.
References:
Natural Medicines Comprehensive Database. NMBER (Natural Medicines Brand Evidence-based Rating).
Weber W, et al. The Evolution of Science and Regulation of Dietary Supplements: Past, Present, and Future. Front Nutr. 2024.
U.S. Pharmacopeia (USP). USP Verified Dietary Supplements.
FDA Draft Guidance. Evidence-Based Review System for the Scientific Evaluation of Health Claims.