Log in

No account? Create an account

Peer review quandry - F*cking with Clusters — LiveJournal

About Peer review quandry

Previous Entry Peer review quandry Jul. 18th, 2027 @ 04:33 pm Next Entry
I just got asked to peer review a paper for a fairly big machine learning journal...

I'm in a bit of a moral bind, unfortunately. The paper has a simple, but somewhat cool application of machine learning to a field that could really use it (I have no idea how much one is allowed to say about papers for review, so I'm erring on the safe side).

However, the paper is 24 pages long, and it took me much longer than it should have to extract the page worth of actual meat. Its as if the author was afraid that their idea was too simple, and so they had to throw as much jargon and convolution into the paper as possible.

While this sort of mess is common in machine learning, I feel like I'd be endorsing poor writing if I were to just let it pass. The only other sizable complaint I have is that they used simulation data to check the validity of their assumptions (and did nothing at all to explain how their simulation isn't based on those same assumptions). Bad use of simulation data alone isn't enough to get you dropped in the ML world, so I'd really be forcing a value judgement of "I think you should write your papers to convey information instead of to impress people" onto them if I said "no".

Have any of you ever hit this?
Tags: ,
take a penny
[User Picture Icon]
Date:July 19th, 2007 12:54 am (UTC)
Recommend rejection (though maybe "weak reject" if you have that option), and give detailed reasons. That's the only way the authors will learn. ("Bad scientist! No biscuit!")
(Deleted comment)
Date:July 19th, 2007 01:27 am (UTC)
I think that is what "rejection" means in peer review.
[User Picture Icon]
Date:July 19th, 2007 06:40 am (UTC)
Indeed. However, if a paper is rejected, you do have to fight the deadlines to resubmit.
Date:July 19th, 2007 02:17 am (UTC)
I'm in favor of rejection with comments. If need be, print out a copy of the paper and bleed some red ink on it. If things like this didn't get rejected, then there wouldn't be any value in trying to publish to a peer-reviewed journal, would there?
[User Picture Icon]
Date:July 19th, 2007 02:23 am (UTC)
It is perfectly acceptable to ding them for being unable to get their point across effectively. Do it for all us poor suckers who have to read papers by similar people.
[User Picture Icon]
Date:July 19th, 2007 07:31 pm (UTC)
Journals, from my understanding, have a few options: accept, accept with minor revisions, accept with major revisions, reject. Most papers supposedly fall in the middle two categories, usually the major revisions one. Usually a paper accepted with major revisions will have to be reworked three or four times until the reviewers or who ever are satisfied.

It sounds like this paper could fall within that category. Check with the journal to find out. Conferences usually have weak reject and weak accept options, for 'good idea' but paper doesn't have enough meat or there are some technical flaws. The simulation problem sounds like one a the major technical flaw.

The other thing to consider is whether it is really journal material. There are several good ML conferences out there. And if they don't have enough meat for a full journal paper, then probably it is really a conference paper (or two conference papers).

I've only had to review conference papers and there was a fairly clear divide within the papers I read.
(take a penny)
Top of Page Powered by LiveJournal.com