Opinion: Conflicting results: Simultaneous studies often yield conflicting outcomes

Key Points

Dermatology Times wondered how dermatologists handle conflicting results. When do they decide the studies offer benefits for their patients? How do they decide which studies may be more reliable or how to weigh the results of one against the other?

On Call talked to a number of dermatologists around the country and asked how they judge studies, and especially those that offer conflicting information that could affect their patients.

It's an important issue, according to Larisa J. Geskin, M.D., director of the University of Pittsburgh Cutaneous Oncology Center and Photopheresis Unit, because of some of the unique factors in dermatology.

"Unfortunately, dermatology is a relatively small specialty. There are not as many patients or academic dermatologists to conduct huge clinical trials with thousands of patients like, for example, cardiologists do.

"By and large, our clinical trials, with some exceptions, have smaller numbers of patients because we are a small field. Sometimes we have to sacrifice the placebo-controlled arms, because we need to have enough patient information in the treatment arms."

That being the case, Dr. Geskin says, "Not all studies are created equal. It is extremely important to look at the quality of the study, including size and overall design, whether it was prospective or retrospective, and whether it was cross-sectional."

Each study has some kind of inherent flaw. "Having prospective studies that include many patients, are randomized and placebo-controlled are optimal for any question. Unfortunately, that option is not always available," she says.

Study design

In St. Cloud, Minn., Daniel K. Elieff, M.D., agrees that study design plays a critical role in evaluating study results.

"One thing we probably all do is look at whether the information comes from a larger, at least somewhat nonbiased study, or if it is from a small, maybe a pharmaceutical-company- funded, individual study.

"In that case, if I had data from a larger, multicenter study that contradicted a small pharmaceutical study, I would definitely tend to favor the data and information coming from the larger project.

"Placebo-controlled studies are important when you're dealing with oral and topical medications," Dr. Elieff says.

Consider the source

Dr. Elieff considers the source of the study design and data, as well.

"The source makes a huge difference, even the credentials of the individuals who produced the data or information. Getting data from the pharmaceutical company or getting it from a study performed by a large educational institution - I would be in favor of the information I received from the institution that did that research," he says.

Primarily a Mohs surgeon, Dr. Elieff tends to think educational institutions may be a bit more reliable in how they handle study data.

"You've got a better likelihood that if it's early or preliminary data, if it's from a large multicenter trial, it won't be released early, and you won't be dealing with a lot of wrong assumptions. Whereas, with some of the small pharmaceutical-sponsored studies, you'll get preliminary data that is contradicted later on, and you might find yourself treating something that was not appropriate," Dr. Elieff says.

Follow-up research

Several of the dermatologists told On Call that conflicting studies often prompt them to do follow-up.

Kenneth A. Mark, M.D., clinical assistant professor at New York University, has practices in the Hamptons, Aspen and Manhattan. He says even a sentence in a study can spark research on his part.

"When I read articles, I'm the kind of person who likes to go back, even when I'm in the middle of the article, and see the original reference. I wouldn't say I'm skeptical, but would say I'm intellectually curious to see who the source is.

"I remember one product, which I don't want to name, but when I would read the studies, they were always done by the same guy. It struck me as kind of weird that there was only one guy in this one group endorsing this particular product.

"On another product - a cancer drug - one person in Pittsburgh was doing all of the clinical trials for that company.

"That kind of situation strikes me as 'fishy,' so I like to see more than one person or group involved in studies, then I can make my decisions. Obviously it's good to see consistency, not conflicting reports, when you're trying to treat patients," Dr. Mark says.

Holding back

Dr. Mark's expertise is in Mohs surgery and cosmetic surgical dermatology. All things being equal in conflicting studies, he is likely to take a wait-and-see position.

"If it's a brand new treatment and the feedback is conflicting, I will hold back. I think time will tell. I'll let other people comment on its use and see what they come up with," Dr. Mark says.

Experience

A couple of the dermatologists say conflicting studies generate their interest in finding out for themselves whether new treatments are effective.

In Newport Beach, Calif., Vince F. Afsahi, M.D., a clinical instructor at the University of California, makes his determination on conflicting studies based on his experience with patients.

"Patients will tell you if they don't like something. They don't hesitate. If something isn't tolerated by the patient, or they don't like the consistency of the cream, they don't use it. It won't be effective if patients leave it on the shelf. Although, when they see a treatment work, their compliance goes up.

"So I will look at active ingredients and see if the treatment makes sense from the biological standpoint, or from a safety profile. Then I will hand select a few patients who have tried other modalities, and perhaps failed, and see if they respond to this new treatment.

"I have to tell you, patients are very perceptive; they know what they like, know their own body, and know if they are responding to a treatment," Dr. Afsahi says.