A must-read post over at the science-based medicine website on how alternative medicine people try to change the rules in midstream:
Science is simply a set of rules of investigation. The biggest rule of science is that we have to test our ideas against reality. We can’t just make stuff up and then assume we are right – we have to subject our guesses to observations that have the potential of proving them wrong. Scientific observations must be recorded objectively so that we don’t have to rely upon flawed memories. Outcomes need to be specified ahead of time – we cannot decide at the end of an experiment which results prove our hypothesis. Outcomes should be quantified as much as possible, and as objectively and unambiguously as possible.
…
Today there is a political/ideological movement within medicine and health care to change the rules after the fact. The purveyors of many sectarian methods of treatment and unscientific belief systems of health and illness have not succeeded at the fair rules of science. So now they want to change those rules. They want anecdotes to not only count but to trump rigorously controlled observations (that is, when the anecdotes are in their favor). They was to reinterpret the placebo effect after the fact as if it were a real effect. They want to count only those experiments that confirm their beliefs and ignore or reject those studies that reject their beliefs.
Being educated adults they have much more sophisticated language to express their childish desire to alter the rules.
Andrew Weil wants to relabel anecdotes he favors as “uncontrolled clinical observations.” This is a way of getting to choose after the fact which observations count, rather than letting the rules of science decide.
Dr. David Katz from Yale’s “Integrative Medicine” Program wants to allow for “a more fluid concept of evidence.” This way modalities he favors, such as homeopathy, that have failed by the generally accepted rules of science can still win with his more “fluid” rules.
When studies of “alternative” modalities are negative, proponents want to change the rules after they see the results. They claim that the “sham” acupuncture was giving a real effect too, or that the numbers in the study were too small, or that homeopathy cannot be tested with the same methods as cookie cutter drugs, or that a statistically insignificant trend in their favor should count even though the rules say they shouldn’t. Of course, when the outcome is positive, then these same rules are just fine. Heads I win, tales you lose.