In this piece, for example, the author pats himself on the back for his “paper output” and high h-index. The h-index is a count of papers published that factors in how many times those papers have been cited. It’s a better metric than simply a paper count because it attempts to assess quality under the assumption that quality can be measured by citation. Good papers are cited more than bad ones, in theory. That’s fine, but it’s still a measure inherently based on total output.
Judging the quality of a scientist by publication volume has always bugged me. Lets say I do a thorough, solid experiment that involves several steps. Maybe first I do some tests to figure out how to control for something. Then next I do some to figure out the best conditions to use. Then, finally, I do my experiment under the conditions and controls I determined earlier. Because I did the first two steps, the third should give me a very nice, definitive data set. If I publish it all, it will make a great paper. If I did my job right, after you read my paper you shouldn’t be left with any doubts along the lines of “well, maybe, but you didn’t account for so and so…” because I already thought of possible problems and did the initial steps to solve them, and included that data.
But I’m being judged on number of publications, so rather than write one thorough paper I’ll split it up into three, none of which can really stand on their own. That doesn’t help me (you’ll read my paper and say “wow what a piece of shit this experiment is, they didn’t even take so and so into consideration” and so you’ll think I’m an idiot for it) and it certainly doesn’t help anyone who reads the paper.
The h-index weights a straight paper count by incorporating how often those papers get cited, so its a bit better, but still problematic. For one thing, most papers start out in the Introduction section with some perfectly obvious, banal fact. “As a result of increasing oil prices, alternative forms of energy are attracting significant interest”. People feel the need to cite that All. The. Time. so a great many citations to a paper don’t necessarily cite the results, or some new method, but just some obvious statement that sets up the intro.
Further, lets say I want to raise my h-index. I’ll start by writing lots of short partial papers like I mentioned, and I’ll get all my friends together. We all agree to cite each others papers every time. Or I could cite myself. Or, as an anonymous reviewer, I could demand others cite me In other words, game the system with a bunch of useless, incestuous citations.
Scientists naturally want to develop a system to quantify everything. That makes sense. But when assessing research, counting papers and citations just doesn’t cut it. You have to read papers, and decide whether or not those papers are good. Output based standards are lazy methods of evaluation that incentivize bad, lazy science.
520 Chestnut 
Interesting!