What is Wrong with Testimonials and Results?
October 20 2010
Every trainer and fitness website has a testimonial page with a lot of impressive before and after pictures. And everyone claims it’s all about results. But what is blatantly wrong with these testimonials and results?
And this is the typical strength coach and trainer’s classic line: “I get results and results are all that matters”. Do you?
What is missing in testimonial and results?
Negative results: Have you ever seen a testimonial page with people looking the same in before -after pictures? Or anyone writing about how the program or the supplement didn’t do anything good for them?
For example, the testimonial page shows 10 people who got great results with the diet/workout program. But here might be 9 more people for whom the diet/workout didn’t do anything and 2 more people where the program/diet actually made them worse. These people are never seen of the testimonial page. Do you think the diet/ program still works?
Drop Outs: There are folks who dropped out from a diet or workout program and you will never find their names on a testimonial page. These are the people who usually think the program and supplement didn’t work for them. In drug trials of fatal diseases, like cancer, they usually say “dead men say no tales”.
What about research?
Reporting Bias/Outcome bias: In research this type of under-reporting is called publication bias or/and outcome bias.
In research, you have to report negative results, adverse outcomes, and the study drop outs /attrition rate. The effectiveness of a treatment/supplement is identified by taking into account BOTH the positive and the negative results. If you do not report negative results, you are in violation of the standard code of scientific conduct & ethical behavior and may end up in some legal trouble.
- This is the problem with results/ testimonials. Everyone seem to have results and it is hard to pick one workout/diet over the other based on it.
- Next time when someone talks about results/testimonials, ask them if they ever heard about reporting/outcome bias and send this article to them.
- That’s one more reason to stick with scientific studies.