10 reasons supplement studies may be flawed, part I:

[Tominator]

Let’s face it, supplement research is complicated. Study results are thrown around with little discretion and can be found to support almost any theory. Attend most any lifting forum, and you’ll find yourself perplexed, confused, and frustrated by pseudo-scientists selectively using research to support their ideas or refute those of others. For some, this can become so trying that they will be tempted to begin ignoring research altogether and simply rely upon their personal experience or observation of others. However, the latter path is even more fraught with biases. I advocate another position: Learn to evaluate research and you can quickly dismiss the junk and understand the limitations. While mastery of research methods is a major undertaking, I’m writing a primer for you here to become an armchair research critic for supplement research. Below I list 10 common flaws in supplement studies. I briefly describe the problem and list a classic example from health research to show how the problem manifests.

1. Funding biases. It shouldn’t surprise you to learn that when a company funds a study for their own product, the results of that study are much more likely to support the benefits of that product. However, what some may not realize is that the supplement industry funds over 90% of supplement research, meaning most of this research is heavily biased. Point: Be very suspicious of research funded by the maker of the supplement in question.
Example: A recent review by Lexchin et al of antidepressants found that studies were more than 4 times more likely to report a positive effect for the drug when the study was funded by the drug maker.

2. Studied at wrong dosages. Supplements and drugs often have therapeutic levels at which they show benefits. Unless you are giving dosages at least at this minimum dose, it is silly to expect results, yet this is exactly what happens in many studies reporting no benefits. Point: Read what dose and frequency supplements were administered.
Example: Shockingly, a JAMA article from the 1970’s showed that steroids were ineffective for muscle growth, at the same time Arnold was winning multiple Mr. O’s, and studies from several years ago appearing in prestigious medical journals indicated that growth hormone also did not enhance athletic outcomes. The flaw? They used trivial dosages because, as the study authors reported ‘it would be dangerous and unethical to administer doses used by actual athletes’. These studies were designed to fail. Naturally, congress now cites these same worthless studies to support their laws against GH and steroid use.

3. Interactions. Our bodies are massive chemical soups of such complexity we are only beginning to understand how they work. Nothing in our bodies works in isolation. How any chemical or supplement affects us depends on what else we are taking and eating, our emotional states, sleep levels, and many other variables.
Example: Classic cases are vitamin interactions. Vitamin C, for example, is a well-known antioxidant, but combine it with iron supplementation and Vitamin C becomes a pro-oxidant (another reason not to take iron supplements if you’re a male). Not the result for which most of us are looking.

4. Studying in isolation. Many supplements don’t work at all, or even work in an opposite direction than expected if isolated. Yet most studies do exactly this; they try to study people like they are a Petri dish, isolating one supplement and studying it’s effects in the body. This is a poor reflection of reality, and means that much of this research tells you little about how the supplement will affect you.
Example: Recent studies of Vitamins C and E, known antioxidants, were actually shown to increase cardiovascular disase and cancer risk in several studies when supplemented. A seemingly shocking result until you realize the body doesn’t use these measly two antioxidants alone, but rather in combination with hundreds of other antioxidants (many of which we haven’t even identified) as they occur naturally in food.

5. Compliance. Studies often don’t even track whether people actually took the supplements as prescribed. If you didn’t know, people are lousy at taking drugs or supplements as they are supposed to, dreadful at keeping logs, and have abysmal memories for remembering what they did. If they don’t even take the supplement per the study protocol, how can you have any faith in the results?
Example: In the 1980’s and early 90’s ten of millions of tax payer dollars were spent to fund a massive interventional trial known as the Women’s Health Initiative to study the effects of a reduced fat diet on heart disease and cancer risk. Due to terrible compliance rates, the “reduced fat” group ended up only 3-5% percentage points lower in terms of their fat intake compared to the normal diet group. Of course the study failed to show any benefits, but the study investigators thank you for your financial support.

Part II tomorrow.....

[Joe]

Awesome!