Wednesday, April 11, 2007

More on Reviews

Recently Lab Lemming started a good discussion on some of the issues involved with scientific reviews, specifically reviewer anonymity. This, and related posts throughout the geoblogosphere were especially timely for me. In the past two weeks I have reached two milestones in my scientific career. First, I wrote and submitted my first official review of a paper submitted to an honest to god reputable scientific journal. I was pretty excited about this to begin with. I have heard many of my bosses and older sciencey folk talk in exasperated tones about all of the papers they have to review and what a pain it was. I suppose after I've done this a hundred times I'll be sick of it, but for now it made me feel like a real professional geologist person. Anyways, I spent a lot of time on this review, wrote and re-wrote my comments, slept on it, read the paper a fourth time, and finally sent in the final comments. In general I was positive in my review, although I did raise some excellent points, which I hope to be thanked for generously in the acknowledgements paragraph of the final paper. How will I be thanked? Well, as an anonymous reviewer of course. This was a tough call for me. Theoretically I do not like anonymous reviews. I stand behind my comments and evaulation, so why don't I attach my name to them? I wasn't overly harsh or negative, and I even pointed out lots of good things. But why do I still fear retribution because I recommended acceptance only with revisions? Truth is, I think as a post-doc who will in the not too distant future be on the job market, I am not sure how much I want to risk souring potential search committee member's opinions of me. I do not yet have the position or name recognition to feel insulated at all from negative vibes. Even though I know I gave a fair and honest assessment, I left my name off the final review. Maybe Thermochronic is just a big old chicken?

And, let's add smarmy hypocrite to big old chicken, because I have also spent the past week grumbling about the second milestone of the past two weeks that I have achieved, namely getting an off-the-mark and anonymous negative review of a paper. This is not a paper I am first author on, but that has not stopped me from cursing anonymous reviewer #n under my breath. I don't know if anonymity played much of a role in making this review crappy, but I would at least like the name of a geologist to google and curse at while I take breaks from revisions. What kills me about this review is that (1) all of the other reviews were very positive, and (2) the comments actually didn't raise any significant issues, just the reviewer's unfamiliarity with some of our standard laboratory techniques.

I like the suggestion of Lab Lemming to include signed copies of reviews with tenure packages. I actually think we could take this a little farther and make reviewer anonymity temporary. If a paper gets published, and a year later the names of the reviewers are made available, then as long as the reviews were honest I think many of the fears of pay-back would be gone. Right now I am all hopped up, stomping around when I think of anonymous reviewer #n and his/her goofy comments, but when the paper is published, a year from now, the emotion of the rejection would be largely gone, and I'd be able to see the review in a much clearer light. Come to think of it, as I sit at the "Scientific Paper of the Year" awards banquet with my co-authors we will probably laugh about the whole incident. So, how's that, a statute of limitations on reviewer anonymity.

11 comments:

Brian said...

oh, 'more on reviews'...i thought you meant 'moron reviews'

I was just talking about this topic with some people at a conference a couple weeks ago. I definitely prefer reviewers to not be anonymous...but after reading your reasons from the perspective as a junior reviewer (i haven't been asked yet as i'm not yet graduated) I guess I can see your point.

Hmmm...the "Scientific Paper of the Year" banquet? I've never been invited to that....oh...right.

Thermochronic said...

That's the thing, anonymity means something completely different to "You can't fire me unless I kill someone" famous tenured prof than it does to "Pleeeeease Hire Me" soon-to-be-grad/ post-doc/ adjunct/ soft-money-peon.

Thermochronic said...

A-ha, I was amazed at the time you posted your comment until I read your blog and realized you were blasting zircons all night long. Perhaps you should post on "tips for staying awake on the ion probe."

"Ain't From Around Here" said...

Ha! I am in the same boat! However, since my reviews have only been positive (though certainly the reviews themselves were not excellent), I did not go anonymous. I think if you have a point then you have a point and any reviewee should recognize that, but we are talking about insulated academecians here... so I see your cause for reservation. If you ever review a paper of mine, I promise only to curse your name only and not your progeny if it is unfavorable.

Chuck said...

TC says:
"I actually think we could take this a little farther and make reviewer anonymity temporary. If a paper gets published..."

Doesn't this give the reviewer who wishes to remain anonymous an incentive to trash the paper so severely that it is rejected (thus preserving anonymoty) instead of published?

Thermochronic said...

Good point, I suppose even articles that get rejected can fall under this clause.
Mayve we should all be forced to read our reviews out loud at AGU or GSA, then th author can be given a box of rotting fruit. That way they can take out their aggression and anger and the reviewer can get their vitamins.

Dr. Lemming said...

When weighing up the risks in deciding on anonymity, did you also consider the potential benefits?

Thermochronic said...

I did, and that just made it a tougher call, because as I said I spent a lot of time with the review and I think my comments would help the paper a lot. Hmmmm. This was the first one I have written, so I shouldn't beat myself up too much, but but I am still not sure if I made the best decision. I supppose it is out of my hands now.

Chuck said...

What are the potential benefits?

EliRabett said...

Sticks and stones
Can break your bones
But grant reviews can harm you.

Vibrato said...

Just reading a review of a book by Susan Hough - Richter's Scale: Measure of an earthquake, measure of a man. Seems he was very grouchy and gloomy at the end. I think that's the fate of all free-thinkers: they think the whole world has gone to pot.