subject: Why Wax Paper Doesn't Work (OR) Liars, Damn Liars, and Statisticians [print this page] Why Wax Paper Doesn't Work (OR) Liars, Damn Liars, and Statisticians
In July 2010, a very controversial study by Joseph Wax was published in the American Journal of Obstetrics and Gynecology. The paper asserted through a meta-analysis of 12 home v hospital birth studies that children born outside the hospital setting are 2-3 times more likely to die within 28 days than those born in a hospital. The backlash in the birth community has been huge.
The controversy surrounding this journal article is not so much in the statement it purports, but in the statistical methods utilized in the research. A recently published Scientific American article, "Home-birth Study Investigated," highlights some of the article's issues as presented by various academics and statisticians. "Divergent methods" appears to be the academic world's biggest complaint. Wax's paper was a meta-analysis of 12 studies, which means that rather than designing and implementing his own research study, the Wax was analyzing 12 prior studies and reporting with inferential statistics that home birth is bad.
Scientific American mentions that Wax selected from his 12-study data pool a subset of only 4 studies from which to draw his inferential data. Maybe he was a slacker and only wanted to do just a little number crunching, but it's more than likely that he was selecting from his data pool the sample population best suited to make his anti-homebirth agenda a more convincing one. That's a statistician's job. But this one really blew it by eliminating from his data pool a certain 2009 Dutch study of 300,000 home births. This study found no increased risk of death after home birth. That's a very large population of excellent home birth outcomes to exclude from an ethical investigation. Given such dramatic number errors, I personally feel it's safe to say the Dutch study would have annihilated Wax's thesis statement completely.
Dutch birth is a different league of birth than we see here in the United States. American Midwifery is based on the Dutch model of birth, in which well-trained, licensed midwives are the de facto care providers of gestating women. When birth outside the hospital setting becomes the norm, the providers are so familiar with and well-trained in natural home birth that their aggregate results are inevitably far greater than a country that is still hot to incarcerate midwives in a large percentage of its states. Statisticians investigating the Wax paper are calling this study omission the biggest fundamental flaw in the research findings. Diana Petitti, Arizona State University Center for Health Information and Research epidemiologist mentions, "The problem of excluding the Netherlands study dwarfs any problem related to software or the statistical methods."
Oh that's right: there was a software issue. The journal that published Wax's article, the American Journal of Obstetrics & Gynecology, launched an investigation on the study, though they are not ready to retract the article. Carl Michal, a physicist at the University of British Columbia recognized a flaw in the study's calculations that lead the investigation to identify an error in the online meta-analysis calculator used by Wax's research team. No one can deny the numbers are off. That calculator's developers have warned other potential users of the problem, but it remains online as-is. And Karin Michels, a Harvard Medical School epidemiologist, points out that research methods "did not provide measurements of the variations between the studies." It's a problem because studies with data concentrated toward one result should not be combined with studies concentrated with opposing data, because "readers can't know how much the studies included may differ and whether it was appropriate to combine them."
American Journal of Obstetrics & Gynecology reportedly mandated Wax to publish online full summary graphs for each of the study's outcomes. They've been posted. But the results posted online are now different from the original results published in the journal article. Risk of newborn death and postmaturity among home birth babies is now higher than originally asserted; and the risk of prematurity is now lower. How come, Joseph Wax? The results are different now, but he's never reported whether it was due to recalculations, statistical errors, or any other reason.
The critics are not happy. Home birth is an endangered piece of our American culture, and the big boys playing hardball to eliminate it completely doesn't help. Especially when they use their power under the big titles like ACOG to produce inaccurate assessments of what is safe and what is not safe. The Wax paper's researchers did not see or touch a single woman in their research to blacklist home birth. Bad methodology bridged with tertiary information from the meta-analysis recreated an image of statistical lying made famous by the Bush administration. And now on top of that, Wax is refusing to comment until further investigation of the study is underway.
