Posts Tagged ‘Flaws’

Recommended by HealthPAIN RELIEF from http://www.prevention.com/health/health-concerns/acupunctures-effect-knee-pain
How Researchers Reached The Flawed Conclusion That Acupuncture Doesn’t Help With Knee Pain By MARYGRACE TAYLOR SEPTEMBER 30, 2014
The effects of acupuncture on knee pain

Picture this scenario: An adult plagued with chronic headaches seeks relief by popping ibuprofen a few times a week. The meds help. Then she decides to stop taking them. And when she does, the pain creeps back.

Surprised? Not exactly. The last thing you’d deduce from this imaginary experiment is that ibuprofen doesn’t help with headaches. But that’s basically what researchers suggested about needle and laser acupuncture’s effect on chronic knee pain in a new JAMA study.

Prevention Recommends

How To: Lift Sagging Skin

If You’re One Of These Two Bra Sizes, You’re Probably Wearing The…

This Dog Lives Longer Than Any Other Dog

7 Ways To Fall Back Asleep In 10 Minutes Or Less

The Role That Kindness Plays In Healing From Addiction

6 Ways To Fight Inflammation
Recommended by
Prevention Newsletters
Exercise of the Week
Prevention Today
Recipe of the Day
Eat Up, Slim Down
Look Your Best Beauty
Flat Belly Diet
Outsmart Diabetes

Sign Up

You may unsubscribe at any time.
Your Privacy Rights | About Us
In the clinical trial, 282 adults age 50 and older with chronic knee pain were randomly assigned to needle or laser acupuncture treatments or a sham laser acupuncture treatment. After 12 weeks, participants who received the acupuncture reported modest improvements in pain. Then the treatments stopped, and nine months later, the participants had knee pain again. This, weirdly, led the researchers to conclude that acupuncture just doesn’t offer relief from chronic knee pain.

Sounds confusing, right? Save for undergoing surgery, most chronic pain problems can never really be permanently solved. Even for treatments that make the discomfort vanish, it tends to come back once said treatment stops. That’s sort of a given. “Acupuncture can be used as pain management, but it doesn’t necessarily heal the pain permanently,” says Michelle Goebel-Angel, licensed acupuncturist at Chicago’s Raby Institute for Integrative Medicine at Northwestern.

There’s more. The researchers of this small study posit that having a larger sample size might have yielded more significant results. Which is exactly what experts uncovered in 2012 meta-analysis of nearly 18,000 patients, which found that needle acupuncture does help with osteoarthritis, as well as other types of chronic pain.

Still, like many treatments, acupuncture doesn’t have the same effect on everyone. But it’s absolutely worth trying, and tends to be the type of thing where the benefits accumulate over time (as in, longer than 12 weeks). “When patients feel the relief, they believe it,” says Goebel-Angel. “And that opens a new level of healing—the spiritual aspect of healing.”

MORE: 12 Odd Pain Relief Tricks That Work



Read Full Post »


Can Acupuncture Treat Knee Pain?

Ge Nan, PhD, Yong Ming Li, MD, PhD

Recently an article in Journal of the American Medical Association concluded that “neither laser nor needle acupuncture conferred benefit over sham for pain or function” among older chronic knee pain patients. (JAMA, 2014;312:1313)

We, the practitioners of acupuncture and more broadly Traditional Chinese Medicine (TCM), believe this conclusion is premature. There has been several thousand years of acupuncture practice in history. Acupuncturists’ observation, patients’ feedback, many previous clinical trials, as well as basic researches, all suggest that acupuncture could effectively treat knee pain.

There are several flaws in the design of this article, which we will itemize and address below. In general, the key reasons are 1) the lack of an appropriate sham needle control in the trial, 2) poorly designed protocol and insufficient power of test, and 3) exaggerated data interpretation.

This study demonstrates better effectiveness of needle acupuncture over no treatment control.
This is consistent with observations in previous clinical trials and real clinical practice. Logically, this does not argue against needle acupuncture as a potential therapy for treating chronic knee pain.
The primary goal of this trial is to determine the efficacy of laser treatment, not needle acupuncture, in treating chronic knee pain, as clearly evidenced by authors own trial protocols and publications.
A negative result is concluded from the data, that is, laser treatment was shown not to be better than sham laser treatment or needle acupuncture.
The final conclusion of this study states: both needle acupuncture and laser treatment are not better than sham laser treatment, thus acupuncture should not be recommended for patients with chronic knee pain.
This conclusion is ungrounded, and quite frankly does not make sense. Sham laser treatment is not a valid control for needle acupuncture, thus the comparison is invalid.
The acupuncture protocol in this trial is poorly designed and does not reflect real clinical practice and management of patients with knee pain.
Acupuncture point selection, acupuncture dose and frequency, time course, and evaluation points was not optimal.
Among 282 patients participated in this trial (about 70 per group), with variable health conditions, only 54 patients completed needle acupuncture treatment (less than 1 treatment per week, for 12 weeks).
This small sample of patients does not provide enough statistical power to test the difference between acupuncture and sham treatments.
Twelve months is too long a time-peirod for a second point of follow-up, considering the treatment concluded at the end of 12-week.
Moderate or severe chronic knee pain occurs naturally among older patients. A more appropriate follow up regimen would be frequent survey over a shorter time period.
The author over emphasized the strength of Zelen design and ignored its limitation.
Despite randomization in the beginning, the usage of Zelen design in this study is not appropriate. For example, different treatment groups are not equally blinded during the trial, which will exert variable placebo or even nocebo effects and eventually lead to unfair among-group comparisons. Furthermore, 19% of the patients that refused needle acupuncture were analyzed as receiving the treatment. This is not an accurate comparison by any means.
This trial includes patients older than 50 years with moderate or severe chromic knee pain and the potential benefit effect of acupuncture on younger patients or older patients with mild knee pain cannot be ruled out.
Since publication, there have been a lot of debates around the efficacy of needle acupuncture. The investigators of this trial appeared on several public interviews to promote their negative findings on acupuncture and recommend their physical therapies to the patients. However, based on above analysis, we think the conclusion of this trial is premature and editors of JAMA should re-evaluate this report.

Summary of 2014 Australian Acupuncture Trial
Figure 1: Summary of 2014 Australian Acupuncture Trial for Knee Pain: The trial was originally designed to test laser treatment using needle acupuncture as a positive control (reference treatment). However, after collecting undesired results that the needle did not work as they expected, the authors changed their research aims to test both needle and laser treatments in final report.

NIH Summary
Figure 2: A recent summary by National Center for Complementary and Integrative Health (NCCIH) of NIH states acupuncture may help to reduce pain and improve joint mobility for osteoarthritis. NCCIH analysis was based on data published prior to 2010. The quality of acupuncture trials is crucial, because flawed negative data may change this balance in the future.

Acupuncture Today is a leading provider of acupuncture and oriental medicine news, info and research information in the world

Read Full Post »

Chinese Doctors Poke Holes in Australian Acupuncture Trial for Chronic Knee Pain by Rana Hinman- By Bill Reddy, LAc, Dipl. Ac..

Read Full Post »

Controlled trials, systematic reviews and meta-analyses: acupuncture effective for patients with knee osteoarthritis.

Read Full Post »

Controlled trials, systematic reviews and meta-analyses: acupuncture effective for patients with knee osteoarthritis.

Read Full Post »

The methodology flaws in Hinman’s acupuncture clinical trial, Part II: Zelen design and effectiveness dilutions.

Click to access S2095-4964(15)60172-8.pdf

Read Full Post »

The methodology flaws in Hinman’s acupuncture clinical trial, Part I: Design and results interpretation. J Integr Med 2015; 13 (2) : 65–68.

Read Full Post »

The methodology flaws in Hinman’s acupuncture clinical trial, Part I: Design and results interpretation. J Integr Med 2015; 13 (2) : 65–68.

Read Full Post »

Click to access S2095-4964%2815%2960172-8.pdf

In the October 2014 publication of JAMA, Dr. Hinman and colleagues published the study “Acupuncture for Chronic Knee Pain: A Randomized Clinical Trial,” which concluded that “in patients older than 50 years with moderate or severe chronic knee pain, neither laser nor needle acupuncture conferred benefit over sham for pain or function. Our findings do not support acupuncture for these patients[1].” As pointed out in my former article, Part I[2], there were serious flaws in the trial design and statistics, as well as in the interpretation of the results. This article attempts to address problems in the Zelen design used by Hinman et al[1]. There are some advantages to using a Zelen design for a randomized controlled trial (RCT). First, a Zelen design has a post-randomization consent design, which means that consent is only sought for one treatment each time, without the uncertainty of randomization. Researchers can be more comfortable knowing that they have the participants’ consent each time they undergo a treatment. Patients can also be more comfortable with this design because they know which type of treatment they are receiving; unlike traditional RCTs, patients are not ignorant of whether they are receiving the placebo or experimental treatment. Effects such as resentful demoralization and what is known as the “Hawthorne effect” (altered behavior or performance resulting from awareness of being a part of an experimental study) become less of an issue as patients are not weary of being part of a new alternative group, only the “standard” therapy will applies to them. However, it does have some disadvantages, and therefore can cause biases.

1 High drop-out rate:

The drop-out rates were 2.82% (2/71) in the control group; 22.86% (16/70) in the acupuncture group; 18.31% (13/71) in the laser acupuncture group; and 22.86% (16/70) for the sham laser acupuncture group. According to the acceptable standards for an RCT, dropout rates less than 10% are acceptable, drop-out rates between 10% and 20% mean that the resulting data quality is poor, and drop-out rates of more than 20% mean that the data quality is considered very poor and should not be used in analysis. In this trial analysis, the data quality in the acupuncture and sham laser acupuncture groups are very poor as the drop-out rates are over 20%; the authors should not have directly used them in any statistical analysis, unless they had re-adjusted and re-balanced the sample among the groups during the study. As outlined by the National Institutes of Health, if there is a differential drop-out rate of 15% or higher between study arms, such as between the control group and the treatment group in this clinical trial, then there is a very high potential for bias. This is a flaw that can decrease the quality of the study results.

2 The effectiveness in intervention groups was diluted by various factors

The dilution rates should then be 21.87% in the laser acupuncture group, 13.80% in the sham laser acupuncture group, and 31.27% in the acupuncture group (the dilution rate calculations were shown in Tables 1–3). The dilution rate was very significant in the acupuncture group, which causes the effectiveness to be undervalued in the acupuncture group, by almost 1/3.

The effective significance was masked by limited sample size due to the Zelen design of this study.

3.The sample size calculation in this study is questionable.

4 Conclusion The effectiveness of the acupuncture group was diluted 31.27%, and its drop-out rate was 22.86%, much higher than that of the other groups in Hinman’s clinical trial, which constitutes major flaws in how this study is analyzed and interpreted[8]. Based on the bias of Zelen design used in the study, and incorrect sample size calculation, the conclusions drawn from this study are of poor quality, inaccurate, and invalid.

Click to access S2095-4964%2815%2960172-8.pdf


1 Hinman RS, McCrory P, Pirotta M, Relf I, Forbes A, Crossley KM, Williamson E, Kyriakides M, Novy K, Metcalf BR, Harris A, Reddy P, Conaghan PG, Bennell KL. Acupuncture for chronic knee pain: a randomized clinical trial. JAMA. 2014; 312(13): 1313–1322.

2 Fan AY. The methodology flaws in Hinman’s acupuncture clinical trial, Part I: Design and results interpretation. J Integr Med. 2015; 13(2): 65–68.

Read Full Post »

Rana Hinman’s Acupuncture Clinical Trial has too many methodology flaws (IV)-The sham laser acupuncture is not a valid negative control for acupuncture.

Read Full Post »

The methodology flaws in Hinman’s acupuncture clinical trial, Part I: Design and results interpretation. J Integr Med 2015; 13 (2) : 65–68.

Read Full Post »

The sham laser acupuncture is not a valid negative control for acupuncture

In the October 2014 edition of JAMA, Dr. Hinman and her colleagues published an acupuncture clinical trial entitled “Acupuncture for Chronic Knee Pain: A Randomized Clinical Trial” and concluded that “in patients older than 50 years with moderate or severe chronic knee pain, neither laser nor needle acupuncture conferred benefit over sham for pain or function. Our findings do not support acupuncture for these patients”(Hinman RS,et al. Acupuncture for chronic knee pain: a randomized clinical trial. JAMA. 2014; 312(13): 1313–1322.).

I strongly disagrees with such a conclusion, as there were serious flaws in the trial design, the statistical analysis of the data and in the interpretation of the results of this study.

I do agree acupuncture should have a real sham control in a vigorous RCT; however, in Hinman’s acupuncture RCT, the sham laser acupuncture is only fit to the laser acupuncture, not to real acupuncture. Because Acupuncture and Sham laser acupuncture, these two interventions do not have comparability in both characteristics and form (i.e., not matched). Furthermore, there was no blinding method performed between these two groups-both the patients and the administrators who performed the interventions knew the difference between the groups, such as needling acupuncture and sham laser acupuncture.

Read Full Post »

There is a crucial mistake in interpreting the Hypothesis testing – What means? if P>0.05.

Hinman said :”in……chronic knee pain, neither laser nor needle acupuncture conferred benefit over sham for pain or function (Dr. Fan notes: Her statement was based on P>0.05). Our findings do not support acupuncture for these patients”

From the perspective of hypothesis testing in Statistics, if acupuncture has better results and with significant difference over the primary control (no-treatment group), p<0.05, we can conclude that “acupuncture is effective”- no matter what the result get from the comparing to the secondary control, such as “sham laser acupuncture”, but Hinman intentionally does not report this effectiveness in her conclusion; if acupuncture has better results over “laser acupuncture” and “sham laser acupuncture”, without significant in statistics, p>0.05, we can conclude that “acupuncture is better than the laser acupuncture, and sham laser acupuncture, but need more studies to confirm”. We can’t conclude that “acupuncture is not effective” because that there are no significant difference in statistics between acupuncture and “laser acupuncture”, or between acupuncture and “sham acupuncture” does not mean there is no difference between these treatments clinically. Hinman et al mis-interpreter the results and violates the basic principle of Statistics.

Read Full Post »

Letters to JAMA Exposing Acupuncture Research Flaws Applauded by TCMAAA


Published: Feb 19, 2015 8:01 a.m. ET

TCMAAA calls for stricter adherence to research ethics and well-designed acupuncture studies among the integrative medicine community

TAMPA, Fla., Feb 19, 2015 (BUSINESS WIRE) — In five letters to the editor published in the latest issue of JAMA, the Journal of the American Medical Association, acupuncture clinicians and researchers around the world point to key flaws that call into question the validity and research methods used in a randomized clinical trial published in JAMA in October of 2014. The Australian study, Acupuncture for chronic knee pain: a randomized clinical trial, by Hinman, et al., concluded, “In patients older than 50 years with moderate or severe chronic knee pain, neither laser nor needle acupuncture conferred benefit over sham for pain or function. Our findings do not support acupuncture for these patients.” Many American acupuncturists were outraged when the October 2014 article was published in JAMA and have called for a review of the study’s design and protocols.

Yong Ming Li, MD, Ph.D., of New Jersey challenges that the researchers altered the aims and hypotheses of the study after the data was collected and the trial was closed. According to the original aims and hypotheses submitted to the official clinical trials registry in 2009 the objective of the study was not to evaluate the effectiveness of traditional needle acupuncture against sham laser acupuncture, but to evaluate laser acupuncture against sham laser acupuncture with needle acupuncture serving as a positive control for laser acupuncture. Protocols originally filed with the registry as well as the authors’ baseline publication do not describe sham laser acupuncture as being a control for needle acupuncture. Dr. Li’s letter furthermore debates the validity of using sham laser acupuncture as a control for needle acupuncture, as it is not generally accepted as a valid control for needle acupuncture.

Hongjian He, AP, Ph.D., of Florida also questions design choices: she specifically points to the use of non-standardized point selection for chronic knee pain. Also some patients received treatments once a week, while others got treated twice a week. This lack of consistency throws into question the validity of the statistics extrapolated from the data collected during the study.

David Baxter, TD, DPhil, MBA, and Steve Tumilty, Ph.D., questioned in their letters why the researchers chose to use laser dosages below the threshold necessary to have a therapeutic effect and why they failed to specify wavelength used in the study and why those levels were chosen.

Lixing Lao, Ph.D., MB, and Dr. Wing-Fai Yeung, BCM, Ph.D., point out in their letter that patients were assessed after 12 weeks and then again after one year, but that without treatment for chronic knee pain after one year, the condition naturally will deteriorate, so that the findings after a year are irrelevant. With these key flaws revealed the conclusion of this randomized clinical trial is clearly undermined.

No group has been more involved in this issue than the Traditional Chinese Medicine American Alumni Association (TCMAAA). Through its broad social media in the USA and around the world, TCMAAA has orchestrated a series of professional forums and discussions on research ethics and design for acupuncture studies after the Australian study was published in JAMA.

“This collection of letters represents a merging of licensed acupuncturists and integrative medicine practitioners who demand the same gold standards of ethics and design quality for clinical acupuncture research as conventional medical studies,” stated Haihe Tian, Ph.D., AP., the President of TCMAAA.

Even with the challenges acupuncture poses in gold-standard randomized clinical trials this valuable treatment method should not be overlooked. With properly designed and well-thought-out studies acupuncture can be evaluated fairly and thoroughly, with conclusions founded upon careful reasoning, accepted controls, and irrefutable evidence.


Registered in Florida, TCMAAA (website: http://www.tcmaaa.org) is a nonprofit organization with one thousand members of licensed acupuncture practitioners formally trained in accredited medical education institutions in China. As a leading organization among Chinese Medicine practitioners, TCMAAA continues to support its members’ professional growth across the United States.


Selene Hausman, L.Ac., 480-510-2259

Read Full Post »