Glow, a data science company that makes an eponymous mobile app designed to help women get pregnant (or avoid pregnancies), claims it has helped over 150,000 couples conceive.

That, at least, is the latest milestone data point that the company came to WIRED with an exclusive on. And last week, at the annual meeting for the American Society of Reproductive Medicine in Baltimore, the company presented a study to back up its app’s performance. Comparing Glow users based on how frequently they tracked ovulation cycles in the app, the researchers found that regular loggers were more likely to conceive—and conceive faster—than those who didn’t.

‘The app itself may not be the reason women who use it more are more likely to conceive.’ Jennifer Kawwass, fertility specialist, Emory University

“We very much care about how we can leverage data to change health, and improve health,” Glow CEO Mike Huang says. And this study is a step in that direction: “It showed that [active Glow users] have at least a 40 percent greater likelihood of getting pregnant compared to if you weren’t as actively tracking and using Glow,” explains Glow’s vice-president of marketing and partnerships, Jennifer Tye.

That rhetoric, though, doesn’t match up with the data Glow is presenting. Can a fertility tracking app really help women get pregnant? Maybe. But the study the company presented doesn’t prove it—and can’t, unless Glow gets serious about turning a critical eye on the valuable data it’s collecting from users.

To be sure, it’s laudable that Glow has been so involved in women’s health metrics, an area that has been sorely neglected by other tech companies (and medicine) at times. But “the study does not suggest causation,” says Jennifer Kawwass, a fertility specialist at Emory University. “The app itself may not be the reason women who use it more are more likely to conceive.”

Women’s Health in Tech

About two and a half years ago, Glow launched its mobile app to help women deal with fertility issues. The company followed that up with a host of other apps designed to help tackle all sorts of facets of women’s health, including pregnancy and sexual health. But Glow’s flagship app remains its most popular, and in August 2014, it took credit for 25,000 pregnancies.

Now, Glow’s latest ASRM study suggests that the app not only helps women get pregnant, but helps them conceive faster than they would otherwise. The company looked at more than 100,000 female Glow users, split into two populations: those who used the app “frequently,” completing health logs at least once every two days on average, and “infrequent” users who entered data less than once every two days.

The results: Those who rigorously adhered to a program of ovulation tracking conceived more quickly than those who did not. “This study suggests that fertility tracking with the mobile app Glow is effective in helping users get pregnant faster,” the presentation says.

Screen-Shot-2015-10-27-at-8.08.30-PM Glow

But according to researchers not associated with Glow, the study has a chicken-and-egg problem. Is frequent app use driving behavior that supports fertility, or are people who are already aware of fertility facts more inclined to use Glow’s app regularly?

“The study is very crude in nature—very similar to an online survey, without controlling for many key factors,” says Kutluk Oktay, a doctor and reproductive endocrinologist at New York Medical College. Perhaps women who use the app more frequently are more actively engaged in trying to conceive, while casual Glow users are less deeply invested in conceiving right now—so they’re not having sex as often, or aren’t doing other things to improve their chances at conception. “It may just be measuring the motivation to conceive, rather than helping to conceive,” says Oktay.

According to Emory University’s Kawwass, it would have been helpful if the authors controlled for variables like age, race, pregnancy intent, history of infertility, duration of unprotected intercourse without contraception, and prior or current use of fertility treatment—all of which could have an impact on the likelihood of conception.

The study’s results themselves support the idea that the observed fertility gains were caused by more than app usage. Vitaly Kushnir, a fertility doctor at the Center for Human Reproductive in New York, notes that the study reported improvement in fertility after just one ovulatory cycle—unlikely to be an effect of short-term tracking. “This is more likely to reflect selection bias than a change in behavior learned from fertility tracking,” Kushnir says.

The Potential for Better Data

Here’s the frustrating part: Glow has the potential to design a truly excellent study based on the detailed fertility data it collects from its users. Because it has a lot of it.

Not every person fills out every blank in the app, but Glow allows users to aggregate daily information about everything from whether they had sex to basal body temperature to sleep duration. If enough of the 100,000-plus users in this study tracked data about their alcohol consumption or weight or cervical mucus, perhaps Glow would be able to control for those differences to pull out the real impact of app usage. (A perfect study would compare two groups that were biologically and behaviorally identical—except for how often they used the Glow app.) Glow’s executives do say the company is still in its early days of data collection. But its data scientists also don’t seem to be looking into the tough questions so far.

Asking those questions—and asking them the right way—is remarkably important given how difficult it is to collect clean data about fertility. The best data the scientific community has, Kawwass says, is from patients who have sought care from fertility clinics—which still isn’t a whole lot of people. Some landmark studies are decades old, conducted on the Hutterites, a population that believes in large families and abstains from contraception.

Here’s the frustrating part: Glow has the potential to design a truly excellent study based on the detailed fertility data it collects from its users.

Now, Glow’s user base isn’t a perfect test ground for fertility research either. “The Glow population is not currently representative of the general United States population,” the study says, because, by definition, the app’s users are concerned about pregnancy.

But their sheer numbers could allow researchers to ask novel questions about fertility, ones that are difficult to answer within the confines of academic clinical trials and fertility clinics. And the data could increase confidence in the one result Glow and its users are truly interested in: the app’s ability to improve chances of conceiving.

At last year’s ASRM meeting, Glow said it had tested the assumption that the moon affected women’s menstrual cycles—and it found a correlation. At the time, the company rightly said that it needed “to do more studies to determine causality.” The same logic applies here: Correlation does not imply causation. The fact that women’s health is getting more attention among makers of apps is a good thing, as is helping women proactively manage fertility. But as Kawwass says: “We just have to be careful about what kind of conclusions we draw from the available data.”

Go Back to Top. Skip To: Start of Article.

Continue reading: 

Does Glow’s App Help Women Conceive? The Data’s Tricky