Monday, February 1, 2010

How does pointless research get published?

In light of my many posts on obvious and/or stupid research, quite a few of you have written in with that question.

There are several answers, but the most common one is busy work. And I am my own best example.

I am not an academic/research person. I have nothing against those who are, it's just not my thing. One of my career goals was to die unpublished. I didn't ever want my name in any journal, anywhere.

But when I was doing my fellowship, the chairman was of the opinion that it was critically important that everyone get published at some point, regardless of the quality of the research involved. So he came up with an absolutely bullshit project for me. And I was faced with the options of doing it or failing the fellowship.

So I did the project. It was remarkably stupid and pointless. it consisted of me reading through MOUNTAINS of old charts, going back several years, and making notes. For the record, a lot of BS research is done this way. Some poor sucker in training is forced to tediously analyze endless piles of old charts or videos or patient forms or something, to come up with worthless information, under the threat of failing out of their program.

Let's face it. You can get pointless data out of anything: "Our chart review found that people who saw the original release of The Wizard of Oz in 1939 were more likely to have Alzheimer's disease in 2009 then those who'd seen the original release of Star Wars in 1977. This suggests an unidentified risk factor for dementia in seeing MGM films vs. those made by 20th Century Fox."

And these studies are generally cheap to do, because you're already paying the salary of the resident or fellow involved (even cheaper for med students, since they work for brownie points).

And there's always a crappy journal out there, trying to get advertising dollars and willing to publish anything to get readers.

So I found some meaningless data, and at a weekly division meeting I presented it. There were 4 attending physicians and 2 fellows in my subspecialty at the time. 3 of the attendings, and both of the fellows (including me) agreed the paper and it's findings were meaningless drivel.

Unfortunately, the only person who disagreed was the chairman. And since he was editor at the time of some desperate medical journal, he got my paper published there.

To make matters worse, he then got me a poster spot at the annual neurology meeting that year. So I had to go to this meeting, set up a poster with my worthless data on it and then STAND BY IT wearing a badge that identified me as the author.

So for the required 2 hours I stood there, trying to smile at all the big league academics going by. Most looked at my poster and politely didn't say a word. A few gave me sympathetic looks. 3 made comments about how worthless it was (I silently agreed). Only one said something kind.

I left the poster hanging in the meeting hall. I think I was the only person who didn't take theirs home. I assume it's in a landfill by now.

My shitty article got published a few months later, and several intelligent neurologists (who I assume were reading in a hot tub) found my paper to be such absolute garbage that they felt the need to write to the journal to complain. And the journal editor, my chairman, forwarded the letters to me to write a rebuttal.

How do you defend the indefensible? Hell, I agreed with them.

But by this point I'd completed the fellowship, and was an attending physician. And I didn't care. So I just tossed his requests in the trash.

So my sole contribution to the medical literature is out there. Fortunately, as the years go by, it will continually be buried under newer (though equally worthless) data.

And that's where at least some crappy research comes from. And I suspect most of it has similar origins. Some poor sap who's under pressure to publish something, anything, regardless of how stupid or obvious it is, or people trying to pad their resume, or someone with way too much time on their hands and absolutely no life (if you're in the last category, get a dog. Or join Facebook. Or do ANYTHING to waste your time in a more useful way), and crappy journals willing to publish anything.

And that's the way it is.


Julie said...

now that explains a lot of things ...

Old MD Girl said...

My med school has a 3 month long "scholarly pursuit" program that really lasts about a year. Students pass their "positions" with relevant faculty on to the next year's suckers after they match.

It's brilliant really. My institution gets to say that they support students doing research, and look at this unique program they have! And 100 faculty members get a med student/slave to make do their research for them (or else).

The master's program affiliated with my department does something similar for its fellows. I won't go into the particulars here, but suffice to say, nobody really cares if the fellows do any research after they leave.

Anyway, that's my rant for the morning. I actually like doing research, and hopefully not all of it will be pointless when I'm done!

Sara said...

You have just described my thesis, exactly. As I recently said, it's one of the things I'm least proud of in my entire life.

Anonymous said...

I earned a master's degree in art history and my thesis sought to explain that a certain type of ancient Greek jewelry was primarily worn by upper-class prostitutes. And you know something? I actually feel like I contributed to the world's knowledge - I really did study a topic no one had worked on before, and it was kind of a controversial study, too. As pitifully as people portray art historians, you gotta give us credit for doing more original work than some doctors are apparently forced into. Sorry, Dr. Grumpy!

mean_owen said...

I seldom read the minor medical journals, for much the same reason. Most of the time, the people writing them (and apparently, the reviewers), aren't scientists at all, and don't understand scientific method, statistics, or anything else. You see lots of case studies and stuff like that, which boil down to mere anecdotes.

On the other hand, even very worthwhile science can often be difficult to publish. Reviewers tend to be very demanding. I review for a few different journals, and I'd say that overall about 20% or less of the articles I've referreed were ultimately accepted (usually, there are 3 reviewers). With that, I'd say most of the manuscripts described worthwhile studies, it's just that there were flaws in either the design or interpretation of the science.

There are of course also some journals of last resort for the more basic sciences just like there is for medical journals. Sure, you could publish almost anything is some of those, but you could never show your face in decent company again.

axl said...

There remains the possibility, no matter how slim, that someone doing one of these research projects may accidentally have a profound insight. Such an insight would be ignored a silliness by the publisher of learned papers.

Anonymous said...

It is never too late to write a rebuttal of your own article. That should count as canceling out your original paper. If you really want to make a screed attacking lazy research, given your field you could target it at one of the higher tier philosophy journals like Mind or Nous.

Margaret/Heather said...

SO, then a question. Responsible docs, it seems to me, continue to read to stay up on research pertinent to their practice. If these are crap (and you've got to know just from reading summaries) do you still read them in case there's some small nugget of something useful to you? Or are they invariable pretty useless? Inquiring minds

Grumpy, M.D. said...

I skim through the journals, usually reading just the summaries, and then reading the whole article if it's something of value. This crap was all in the summaries.

Margaret/Heather said...

Ah ha! I suspected as much. I confess that when I do research, I do the same thing.

In those journals that offer a summary. :/ My hobby field's journals most frequently do NOT include summaries so potentially fascinating articles turn out to be duds with no warning. It's pretty frustrating.

Anonymous said...

What a waste -- your advisor could have you doing something real to contribute to the field.

It reminds me of my own profession in the professor world: loads of people putting together crap so that they can add another publication line to their vitas and get tenure out of it. Journals that no one reads and that exist solely so people can have that line on their vitas. One of my colleagues recently published a literary analysis of spam e-mails.

Ladyk73 said...

In my Master's program, we did program evaluations for our local non-profits. My group did a really good program evaluation and we could of published our findings However, it was still alone the lines of ...if someone takes a class in Topic A, they would understand More about topic A. Kind of a stupid thing to publish, but is exactly what a program evaluation is....

Amanda said...

Curious question: what did you do your fellowship in?

Anonymous said...

@6:19 PM: I'd actually like to read that - the humor value sounds priceless! Have you considered nominating your colleague for an Ig Noble?

Word verification: lonator. A robot loanshark from the future?

Hopalong Ginsberg

Anonymous said...

Generally, research projects our PharmD was assigned were presented in program-ending poster presentations, not aimed at getting anything 'published' deal, but practical, useful information. We spent enough time in the fourth year drug information rotation analysing crap studies put out by drug companies, that some of us approached anything where researchers reported company affiliation as automatically suspect.

Thanks for the honesty. There's some real stuff out there, but the take-home message is that we cannot believe everything in print, even published in supposedly reputable journals (JAMA, JPharmSci, Lancet, etc.), never mind specialty publishers. Heck, I've even seen stuff that people read like a Bible that have misspelled words, incorrect grammar, never mind the statistics.

student dr. blaze said...

Amanda: fellowship had to have been in neurology, since he's a neurologist. :-)

And a minor correction, Dr. Grumpy: med students PAY to work. <--actually, that might make a good study topic: wtf would anyone with a drop of intelligence PAY to be worked to near-death?

Aren't you going to at least give us a hint as to what you studied? And why didn't you publish under a pseudonym?

Grumpy, M.D. said...

My fellowship is classified. I gave up trying to subspecialize out of boredom, and hide those credentials. They came off my letterhead years ago.

Anonymous said...

I once, as a lowly Research Associate, was added to a paper solely so that the primary author could list our address first. Definitely contributing valuable info to the body of knowledge there!

Grumpy, M.D. said...

In retrospect, I now feel my research time was far better spent.

It was much more valuable than, say, getting my penis shocked to measure it's electrical conductance.

Wendy said...

I would have replied to those letters pointing out that the chairman and the editor were the same, thus explaining the humiliation of having to publish drivel.

Locations of visitors to this page