A Review of Peer Review: What’s wrong with scholarly publishing?

Part 1 The Way We Were

Peer review was first institutionalized by the Royalty Society of London in 1665.[1] It’s basically a good idea. Disinterested experts in a field vet a scholarly contribution and assess the worth of a paper before announcing it worthy. Peer review encourages quality and helps authors to sharpen their work. But peer review is not mandatory to assure quality. Plato’s Republic, Euclid’s Elements, and Darwin’s Origin of the Species had no peer review.

Moreover, the assumption that today’s peer-reviewed paper has been vetted by experts and has consequently earned its blue ribbon for excellence is far from the truth. Peer review is not doing its job. Today’s collection of scholarly literature is exploding in quantity and deteriorating in quality. The institution of peer review is, itself, sick with shortcomings. But perhaps if we can examine the ailing patient and diagnose the disease, we can prescribe some medicine to make it better.

Part 2 The Sausage Factory

I’ve served on and off as an Associate Editor for a number of journals in my career. Last year, I received a paper from the Editor-in-Chief of an IEEE journal for which I served this position. IEEE stands for the Institute of Electronic and Electrical Engineers. The world’s largest professional society, with over 440,000 members, IEEE publishes 180 transactions, journals, and magazines. As an Associate Editor of one of these publications, it has been my duty to locate reviews for submitted papers that are referred to me by the journal’s Editor-in-Chief.

I looked over the most recent paper I was assigned and saw it was only peripherally in my area. That’s okay. It’s the reviewers’ job to inform me. And it’s my job to find reviewers.

I’m an old hand at recruiting reviewers (which is not to say that I’m good at it). I send requests to 15 potential reviewers. How does one choose 15 reviewers for a paper that isn’t in your specialty? One helpful resource is the authors themselves. On the electronic submissions page, authors can list suggested reviewers for their papers. When I submit papers, I list all my buddies as potential reviewers. These are colleagues who, at worst, will paint my contribution in the most positive light possible. And my cronies expect the same of me when they list my name. (So much for peer review by disinterested professionals). So I go to the list of reviewers suggested by the authors and put them on my list.

In a vain attempt to avoid cronyism and with the goal of letting a paper stand on its own merit, some favor double-blind reviews. When the reviewer receives a paper, the authors’ names have been stripped from the copy. Double-blind review is motivated by the double-blind testing method required by the FDA when drugs and placebos are administered to human subjects. The practice doesn’t translate well to the review of papers. As a repeat participant in this process, I estimate I can as a reviewer identify the author or authors of the double-blind review paper about 90% of the time. Although the identity of the authors is stripped from the paper, their prose and the references remain. The easiest cases of identification come from reading something like Our previous work focused on…. At the end of the sentence there are references that immediately identify the author. If one’s goal is to have a paper reviewed without revealing the identity of the authors, double-blind review is probably the best approach, in spite of the fact that it doesn’t work very well.

For the paper assigned to me, I so far have only the names of reviewers suggested by the authors. To assure outside reviews, I Google the names and email addresses connected to some of the references listed in the paper. All in all, I build up a list of 15 names and send out the invitations. This is done for me on the journal’s website. The email content is a form letter making the review invitation in a polite albeit slightly forceful fashion.

Have you ever tried to get kindergarteners to rank the taste of boiled vegetables? Recruiting top tier peer reviewers to assess the quality of a submitted journal paper is like that.

Only one of the 15 that I’ve contacted responds positively.

And the positive response was not from someone on the authors’ list of buddies. It may seem curious that the authors’ cronies didn’t respond. But due to the anonymity guaranteed by the reviewers to the author, the authors might never know for certain. Perhaps the authors in question would benefit from more loyal friends.

Some of the reviewers I invite give me the courtesy of declining the invitation. The rest don’t even bother to respond. So one week later, I invite 15 more potential reviewers to take a look at the paper. At this point I’m getting desperate. I use Google Scholar to identify papers cited in the paper’s references. Of the 15 new invitations to review, I get three positive responses. But two of these responses turn out to be lies. After they say yes, I never hear from them again. To be safe, I send out five more invitations one week later. As the list gets longer, my attention to the quality of the reviewer diminishes. Like I said, I’m getting desperate.

I eventually yield three reviewers out of 35 invitations. That’s a pathetic .120 batting average. The final reviewers are (1) a postdoc from California, (2) a Professor from Croatia and (3) a Professor from Malaysia. I have never heard of any of them. After a couple of months, I get their reviews and recommendations. The three reviews fall across the spectrum:

• Accept the paper for publication,
• Revise the paper according to comments made in the review and resubmit for additional reviewing, and
• Reject the paper

So much for consensus.

An Associate Editor like me hopes for some uniformity of recommendations. Uniformity not only constitutes a consensus on the quality of the paper, but more importantly, makes the job of the Associate Editor easier. We don’t need to spend a lot of time looking at the paper. Being an Associate Editor is like a judge waiting for a jury’s verdict. For this paper, the jury was hung.

So I’d like to tell you that I took a day out of my life to sit down and read the paper carefully and make my own independent assessment. But I didn’t. I scanned the reviews, began reading the paper and dozed off. It was so boring! I find that subjects outside of my field often are. When I awoke, I decided to do the right thing and follow the Golden Rule. If I was an author of the paper, how would I like the Associate Editor to respond? After the paper was revised according to the reviews, I accepted it for publication.

Recall Physicist Frank Tipler’s claim that those who decide the fate of a paper are quite often not as intellectually able as the author whose work he judges. For the paper I was assigned, I can’t vouch for the academic ranking of the authors in their field. However, I must confess that in terms of expertise needed for this paper, I was one of Tipler’s less able judges.

Today, I try to make a policy of declining invitations to review papers that I don’t find interesting. So yes, I am normally in the group of the 32 of 35 invitees who decline to review a paper. Last year I resigned all my Associate Editorships. Although it looks good to those who count my beans, I am no longer interested in the institutionalized assembly line review process employed by my professional societies.

All this begs the question of how things can be improved. A sausage lover can’t rightfully complain about the blood on the floor in the sausage factory without offering a better way to make sausage. I just accepted the EIC responsibilities for the journal Bio–Complexity, which allows the publication of papers supporting intelligent design. I have published a number of papers there myself and am sympathetic to the journal’s mission and goals.

Stay tuned and I’ll let you know if I find any new recipes for bloodless sausage.

Part 3 Towers of Mostly Babble

There are two activities that rank high in academic currency: publications and attracting external funding to support research or academic programs (particularly those which supply lots of overhead money for your university). The law of supply and demand has spurned a glut of journals. There are also conferences where papers are published in conference records. Both the scholarly journals and the conferences post peer-review guards at the gate with the supposed goal of blocking the entry of unworthy papers. The collection of these papers is often referred to as the literature.

The number of authored papers often becomes a glittering merit badge worn by a researcher. Quantity is easier to assess and communicate than quality. Two decades ago, I declared Pure publication quantity today has become a meaningless metric. One can publish almost anything. I have since revised my opinion. Today I would remove the word almost. But no one listened to me and paper counting is now more deeply entrenched in academia than ever.

Authors are often asked to write short autobiographies in the third person at the end of their papers. In these biographies, we often read self-congratulatory phrases like Dr. Pythagoras is the author of over 500 journal and conference papers. This is like saying Dr. Pythagoras pounded 500 nails into various types of lumber. The pounding of the nails is unimportant. It’s what you’ve built that counts.

I recently found myself in a discussion about publication count with a newly-minted acquaintance at a neural network conference in Japan. He asked me how many publications I had. I told him. He looked at me with near disgust.

Geesh, he muttered. That’s nothing!

He then told me how many publications he had. It was a lot more than me. This was 20 years ago. I have lost track of my friend, the publishing machine. I can tell you, though, that none of his papers has made any impact in the field of neural networks. Such worthless papers are called write-only articles, which is funny if you know what ROM stands for. I’m unsure whether my friend continues to publish today. Chances are his well-endowed paper count caused him to be promoted to an administrative position where he now evaluates faculty by counting beans. Final faculty promotion and tenure decisions are more often than not made by those who lean heavily on bean tallies. The Dean can’t read, but the Dean can count.

And I confess that, yes, I report my publication count in my short third-person autobiographies, but usually with a hedge. In a recent paper, my bio reads [Dr. Marks] is the author of hundreds of journal and conference papers. Some of them are good. And it’s true! In fact, a few of them are very good.

So what is the impact of ranking paper quantity over paper quality? Unsurprisingly, the answer is: more papers. In the December 1994 issue of Scientific American, a figure labeled Tower of journals shows a stack of papers from the MEDLINE bibliographic database in the area of biomedical research. The stack of papers is placed next to the Washington Monument. The stack of papers is taller.

I looked at the trend since 1994 from an updated online list of MEDLINE data. As you can see from my plot in Figure 1, the stack gets taller and taller each year and looks to be literally increasing exponentially. To place the sheer number of publications in perspective, some familiar landmarks are pictured in the background. The numbers are astonishing. Over 1.1 million scholarly publications in 2013! On average, MEDLINE lists two biomedical scholarly papers every minute, 24/7. As you can see in the figure above, the stack of papers at the time of writing is almost as tall as the Empire State Building shown on the right. And MEDLINE only lists the journals it considers reputable. So the true numbers are larger.

But maybe we’re being unfair. The biomedical field is broad and there is no way one can keep up on all areas of the research. So I did a tally of papers in computational intelligence. This is a favorite research area of mine and I have published a lot of papers in the field. (Some of them are good). Using the Scopus database, I generated the figure below with the giraffe in it. At first, this graph looks a lot better than the MEDLINE graph. Although buildings are too tall to put in the figure, we can include a six-foot man and a giraffe for perspective. But closer consideration reveals the problem has not gone away. In 2012, there were 53,446 papers published in the area of computational intelligence. That’s 146 papers each day or six papers per hour. That’s a lot less than MEDLINE’s two papers per minute, but is still unmanageable. No wonder I find it harder and harder to keep up with the current developments in my field.

As shown in Figure 3, I performed an exponential regression on the MEDLINE data and got a fit of 4 × 10-27 e0.0371xx where x is the date (e.g. x = 1990). The R² factor is 0.9834, which is decent. Exponential increases can be alarming. Notice that during recent years, the number of publications lie significantly above the exponential regression curve. Things look like they are getting worse at an ever increasing rate.

If paper hard copies of all of today’s publications were necessary, university libraries could not contain them. Many of today’s publications are somewhere in the cloud and available only on the web.

And thank goodness! We have avoided an environmental calamity! George Kaub published a classic satirical paper in the Journal of Irreproducible Results entitled National Geographic, the Doomsday Machine prophesying global chaos because of the increase of print publications. In 1974, Kaub was concerned that accumulation of heavy glossy paper used by National Geographic Magazines was amassing so fast that catastrophe was imminent. He wrote:

These threats to our environment, our health and our mental wellbeing are real and with us, but not nearly as immediately catastrophic or totally destructive as the disaster which imminently faces this nation…. [This] menace of monstrous proportions can be likened only to the entire country resting on a gargantuan San Andreas fault. Earthquakes, hurricanes, mud slides, fire, famine, and atomic war all rolled into one hold no greater destructive power than this incipient horror which will engulf the country in the immediate and predictable future.

This continent is in the gravest danger of following legendary Atlantis to the bottom of the sea. No natural disaster, no overpowering compounding of pollutions or cataclysmic nuclear war will cause the end. Instead, a seemingly innocent monster created by man, nurtured by man, however as yet unheeded by man, will doom this continent to the watery grave of oblivion.

But there is yet time to save ourselves if this warning is heeded.

PUBLICATION AND DISTRIBUTION OF THE NATIONAL GEOGRAPHIC MAGAZINE MUST BE IMMEDIATELY STOPPED AT ALL COSTS! This beautiful, educational, erudite, and thoroughly appreciated publication is the heretofore unrecognized instrument of doom which must be erased if we as a country or continent will survive. It is NOT TOO LATE if this warning is heeded!

Kaub’s inspired doomsday prediction reminds me of the dire warnings we have today about manmade global warming and the self-sacrifice we must all endure. The difference of course is that today’s warnings are serious.

The accumulation of National Geographic Magazines would be dwarfed by the mountainous pile of paper copies formed from all of today’s so-called scholarly publications. We are all thankful to the cloud for making paper copies no longer necessary. It has allowed us professors to publish an ever increasing number of papers while successfully derailing Kalb’s forecast of impending environmental doom.

Part 4 How to Publish Your Scholarly Paper

Let’s suppose you have written a scholarly paper of which you are proud and want to get it published. The easiest way is the sequential journal list approach: if one journal doesn’t accept your paper, send it to another. There are often dozens of journals or more in a given field. It would be most efficient to send the same paper to a glut of journals at the same time. Buffalos come in herds, whales in pods, and journals in gluts. The practice of parallel submission is considered verboten because you are using too much of the valuable time of those who review your papers. But sequential submission is within the rules and it works.

I once had a paper rejected from Applied Optics. The reviews were unfair. (Authors always think negative reviews are unfair). I let the project sit for a month or so when I learned there was a special issue of Optical Engineering in the area of my paper, so I made some queries. I was invited to submit the paper, and did. It was published. And since I was asked, I could list the contribution as an invited paper on my résumé. A presentation, paper, or book chapter pays higher wages in academic currency when accompanied by the word invited. It says you are highly respected by your peers. My ugly rejected paper metamorphosed into a beautiful invited paper.

If your paper is outright rejected, shake the dust from your sandals and walk away. But even for negative reviews where the paper is not outright rejected, stick your foot (sandal and all) in the door. Initiate a back and forth with the journal. You become a salesman pushing your wares. It shouldn’t have to be like this. Your work should stand on its own merit and your contribution should be clearly seen by reviewers with any intelligence. But this is typically not the case. Many reviewers don’t have any intelligence.

Over the years I have developed a system that maximizes the chance of getting your paper published in a scholarly journal when the door is left open. The method is a combination of principles I learned from Dale Carnegie, Mark Mathis and from my experience across a six-year stint as Editor-in-Chief (EIC) of the IEEE Transactions on Neural Networks.

Let me begin with Mark Mathis. I was in the documentary titled Expelled: No Intelligence Allowed because of an ideological conflict with Baylor University over a website I administrated. (We’ll leave the details about this to another time.) I first met Mark Mathis, the Associate Producer of Expelled, when he came to my house with a camera crew to record me for the documentary. Mark specializes in media relations and has a background in journalism. One of Mark’s jobs was appearing as a guest on talk shows and writing op-ed pieces that promoted Expelled.

Mark taught me that most journalists were either overworked or lazy. He suggested that a good way to get journalists to write about your agenda is to do their work for them. In his book Feeding the Media Beast, Mark says to always be ready with prepared spontaneity for any media interview. These are memorable points and phrases a journalist will quote verbatim because it’s easier to use your clever prose than it is to make up their own clever prose. Everybody likes it when you help them do their job.

And it works. Here’s a personal example.

The same administration that gave me problems with my website at Baylor later decided to deny tenure to a large number of deserving faculty. The ill-conceived motivation, I believe, was to send a message to untenured junior faculty that the tenure bar had been raised and they had better get off their butts, get busy writing papers, and start attracting research funds. Some of my colleagues who I felt were strongly deserving of tenure were victims of this massacre. I got an email from a reporter I had befriended during the Expelled fiasco asking if we could talk about the mass tenure denial. I said yes. But I needed some prepared spontaneity for the interview. After some thought, here’s the quote I came up with:

The administration’s decision about this year’s tenure swept over Baylor like a tsunami. It was totally unexpected and left a significant body count.”

For an engineer, this is a pretty good quote! I wrote it down.

I got the call from the reporter and gave him permission to record our conversation. We chatted at length about the problem. I likened the mass tenure denial to a massacre. And when I felt the time was right, I looked down at my notes and read the quote. In the morning, the newspaper article came out and there was my quote almost word for word:

Marks called the tenure denials a massacre and said the administration’s decisions about this year’s tenure swept over Baylor like a tsunami. It was totally unexpected and had an entirely significant body count.

Prepared spontaneity works! Plus, it was a help to the reporter and it sure made me sound clever.

On another occasion, I made the list of The 20 Most Brilliant Christian Professors on the website CollegeCrunch.com. I got an email from a reporter and agreed to an interview about the list. Following Mark Mathis’s advice, I again worked on my prepared spontaneity and decided on a variation of an old Jack Benny joke. I worked the quote into the interview and, sure enough, there it was verbatim in the next day’s newspaper story.

Marks said he was humbled by his inclusion on the list. For the record, I don’t deserve this, he said. But I have lower back pain and don’t deserve that, either.

Not only did my prepared spontaneity appear in the article, it is now listed on my Wikiquote page.

Like journalists, Editors of scholarly journals are often overworked. Others are lazy. The legacy of Editors lives on, nearly independent of their performance. Like many bureaucrats, Editors want to make sure they are perceived as doing a good job while minimizing their effort. The line stays on the résumé whether or not they put in overtime. There are exceptions, of course. And there are many claims of exception. But in my experience, the warranted exceptions are few.

One of the reasons I know this about Editors is because I was guilty of these crimes when I was EIC of the IEEE Transactions on Neural Networks. During this time, I had a full-time position as a professor with a full teaching load on top of directing the research of a gaggle of graduate students. (Yes, graduate students come in gaggles). I also had three pre-teen children in homeschool. Like most Editors, my EIC duties were an overload and, frankly, not my tip top priority.

I recall being impressed on one occasion with the response of a team of authors to a paper that my Associate Editor considered borderline. I asked the authors to respond to the reviewer’s criticism of their paper. Besides the revised paper, the authors submitted a response letter that went on and on. Each criticism of their paper was addressed in detail. The response letter was almost as long as the revised paper. I read the first few pages of the letter noting that the author’s responses were both polite and seemed to directly address the criticisms. After three pages I thumbed through the rest of the response letter to see how long it was. Twenty pages. Wow, I thought. This guy really makes a good case and really knows his stuff. So without further reading I accepted the paper for publication.

I decided to adopt this practice myself and now write long detailed response letter to Editors. And it works.

Besides being long, response letters should be extraordinarily polite. Angry letters from rejected authors polarize. The author is stupidly posturing as an enemy of the Editor and then asking for a favor. Editors are only human and have a natural tendency to push back against angry authors. I once received a letter from a rejected author who promised that if his revised paper was not accepted, he would publicize the injustice of my journal and assure that my reputation and that of my journal were damaged beyond repair. After muttering some Milquetoast expletives (I’m a Christian), I read the revised paper looking for all of the reasons it should be rejected. Ultimately, it was.

To be effective, long detailed response letters must fully embrace the teachings of Dale Carnegie. Here are some Dale Carnegie principles from his great book, How to Win Friends and Influence People, followed by responses taken verbatim from letters I have written. All of the responses here ultimately resulted in acceptance and publication of the paper.

1. Dale Carnegie Principle: Begin with praise and honest appreciation.
   
   Compliments must be honest. Otherwise they are perceived as flattery. Here is an example of the beginning of one of my response letters to an Editor.

   First, thank you for coordinating a thorough and in-depth review. We have noticed deterioration in peer review quality in the last few decades and it is refreshing to see quality. We have revised the paper in accordance to the reviews and, as a result, the paper is of higher quality. Below is our detailed response to the comments of the three reviewers….

2. Dale Carnegie Principle: Show respect for the other person’s opinions. Never say ‘You’re Wrong.’
   
   One way to do this is to shift the blame to yourself and then claim you corrected the problem. Here’s me taking the blame for some negative reviews without admitting the paper itself was bad.

   There were of course some [reviewer] comments made which were negative. Upon reflection, we believe that our first draft did not make some of the content as clear as it needed to be. If you and the reviewers, who are obvious experts in the field, misunderstood our paper, then we must take the burden of blame. The good news is the comments were helpful in us generating the attached revised edition which makes points more clearly and addresses every comment made in the review.

3. Dale Carnegie Principle: Call attention to people’s mistakes indirectly.
   
   This can be done by claiming that a misunderstanding is your fault. Here’s an example.

   Sometimes when one is so entrenched in doing research, the degree of familiarity of others becomes obscured by your closeness to the topic. I suspect you have felt something like this yourself. Although in retrospect all of the information needed was made available in the paper, we did not present it as clearly as we should. I think you’ll agree that in the revision these problems have been taken care of. Please read our response below. Although we believe you will agree with this, we would be highly appreciative if you don’t that we can have as feedback a critique. You’ll see that all of the negative comments were a result of misunderstanding. We have taken care to alleviate these problems and to explain things more thoroughly.

   Note that if the Editor didn’t agree with us, we asked him to do more work. Under the assumption the Editor wants to avoid spending more time on our paper, this is a smart request.
4. Dale Carnegie Principle: Try honestly to see things from the other person’s point of view.
   
   Salesmen frequently do a phony version of this to manipulate customers. As a customer, you say I really don’t like this used car. I’ve heard the Yugo is cheaply made and falls apart when you drive across the first speed bump. The veteran used car salesman pauses, looks up, rubs his chin, and pretends to think. After rumination, he nods in agreement and responds. Yes. Absolutely, he agrees, still nodding. You know, I used to feel that way too. He looks you in the eye and points his finger. And then I realized…. The salesman agreed with you. Now he will skillfully deflect the criticism with some reason or another. And the sales pitch continues.
   
   Here is my attempt using such a response. The Editor wanted the Appendix of my paper shortened and I felt it couldn’t be shortened any more. I wrote:

   A great Albert Einstein quote regarding explanations is:
   
   Make things as simple as possible, but not simpler.
   
   We are very appreciative of the comments of the reviewer because we were also troubled by the length of the Appendices. They are, indeed, longer than is the case for the average paper. Be assured the Appendices have been shortened significantly as we drafted and redrafted the original submission. We now believe they have reached the Einstein limit. Our attempt to provide a rigorous foundation for presenting….

   The rest is the technical justification for not shortening the Appendix. Note that, like the used car salesman, my response is Yes. The Appendices are too long. I agree. But then I realized….
5. Dale Carnegie Principle: Appeal to the nobler motives.
   
   At times I receive reviews of my papers that are so inane, I wonder if the reviewers have even read the paper. The stupider the review, the stupider the reviewer. The stupider the reviewer, the less chance there is at having a reasonable exchange. In such cases, I often appeal to the Editor to do the right thing and dump the reviewers. Here’s an example of appealing to an Editor’s nobler self after receiving some pretty stupid reviews:

   I served as EIC for The IEEE Transactions on Neural Networks for six years, and insufficiently in-depth reviews slipped through my fingers in more cases than I care to admit. Reviews were assigned by lazy professors to graduate students. Other reviews were made by those inept in the field who would not confess their ignorance. Grammar and nit-picky comments on the structure of the paper often are the fruit of reviewer ineptitude. I always felt, and I’m sure you agree, that the enormous amount of work that authors give to a paper deserves thorough review by true peers. I have had papers rejected, but always with good reason such as literature oversight. Even when I did not agree with the decision, the reviews were authoritative, in-depth and well documented.
   
   The papers we submit to [your journal] contain some of the best work I have ever done. [Your journal] is the top journal in its area. We are looking forward to seeing substantive peer reviews.
   
   Thanks for listening.

Here are the takeaways to getting your journal paper accepted after a review:

• First, remember that journal Editors are people who want to be treated nicely or even praised. Everyone does. A good response letter should adhere to the best teachings of Dale Carnegie.
• Secondly, Editors are typically snowed with work. You, as an author, can take advantage of this by writing long, detailed response letters. Doing so without appearing to ramble is an art. The Editor will look at your meticulously-written tome and think I don’t have time to read this. But …  flipping through the pages,  … it looks like a comprehensive and well-written response. Besides, I have a research grant to write. And your paper will be accepted.
  
  At least, this strategy has worked for me.

Part 5 Computer-Generated Beans

We know that real artificial intelligence has arrived when a computer can write soul-stirring poems, compose grand music, pen engrossing novels and, yes, write scholarly journal papers. We’re halfway there. There are free sites on the web that automatically write scholarly papers for you. The first, I believe, was SCIgen. Not surprisingly, SCIgen was written by computer science students at MIT. Go to the site. Enter up to five authors and hit Generate. A nice web-formatted paper appears. Click on PDF and there’s your paper ready to submit to a scholarly computer science journal. SCIgen automatically draws figures and references for your paper. The references typically contain fictitious papers where you are assigned authorship along with other notables such as Albert Einstein and Charles Darwin.

The computer-generated SCIgen papers remind me of the pod copies in the 1956 black and white science-fiction classic Invasion of the Body Snatchers. Pods from outer space are placed next to sleeping humans. The pods take on human form and, during the copying process, the human dies. To an outsider at a distance, the finished pod copies look like humans. Interaction and closer inspection reveals they are phonies. Close relatives are able to notice the difference more quickly. The SCIgen papers are like this. They initially look pretty good. But a closer inspection reveals they are phony. And if computer science is your field, the phoniness is recognized more quickly.

Should we be surprised that phony papers generated by SCIgen have been accepted by conferences and journals? The pressure to publish has been applied to professors almost everywhere. Supply and demand dictates that journals and conferences be created to meet the demand. Many of these conferences and journals, motivated by profit, are not picky about the quality of the papers they accept. They are more interested in collecting fees. Although I’m not a big fan of peer review as it is currently practiced, there always needs to be a gatekeeper to bar entrance of garbage trucks.

A phony paper written by the computer program SCIgen was accepted at the ninth World Conference on Systematics, Cybernetics, and Informatics (WCSCI) in Orlando, Florida. After accepting the paper unreviewed, the WCSCI organizers found the SCIgen webpage where the paper’s authors announced their triumph and were soliciting donations to allow them to travel to the conference and present the paper. After the discovery, the conference wrote …since you gave the information in your web page that the paper was a fake one, we think we should not accept your registration even if you have total responsibility on the content of your paper (as a non-reviewed one). Whatever that means.

There are quality controls on good journals and on journals that are trying to be good. Some journals do a better job than others. Every journal in my professional society, the IEEE, is considered quality and is good academic currency for promotion, tenure, and raises. For biomedical research, The National Library of Medicine controls the PubMed journals indexed by MEDLINE. The government bureaucracies have a tendency to maintain inertia and adding new journals to PubMed is a slow process. Harvard researcher Mike Shrime notes If you want to find a reputable journal, you’d turn to PubMed, but the problem is that there are also many reputable journals that are not on PubMed. Another source of papers is Google Scholar. Google Scholar includes all journals in its database. But there are so many junk journals, you often must search through a lot of trash to find a treasure.

An apparent rebel, Shrime submitted an obviously phony computer-generated journal article to 37 junk journals. Seventeen of them accepted the paper for publication. The article, titled Cuckoo for Cocoa Puffs, didn’t even go through an automatic paper generator like SCIgen. SCIgen at least makes a lame attempt at sounding legit. Shrime simply used a random word generator available on the web that usually fails to even write complete sentences. His paper begins:

In an intention dependent on questions on elsewhere, we betrayed possible jointure in throwing cocoa. Any rapid event rapid shall become green. Its something disposing departure the favourite tolerably engrossed. Truth short folly court why she their balls. Excellence put unaffected reasonable introduced conviction she.

Even though the word cocoa looks thrown in, it’s all random gibberish. Shrime’s designated authors were twentieth century media icon Orson Welles who scared the world with his War of the Worlds radio broadcast (1938) and then went on to write and star in Citizen Kane (1941), a movie most movie critics include on their all-time top ten lists. Welles died in 1985. The other author of Shrime’s paper was Pinkerton A. LeBrain. Although the paper was accepted, it never went to press because Shrime refused to pay the required $500 processing fee.

Like Shrime, engineer Alex Smolyanitsky generated a phony journal paper. He chose Edna Krabappel, Kim Jong Fun and Maggie Simpson as the authors for his random text-generated paper titled Fuzzy Homogeneous Configurations. The middle author looks to be heir to a North Korean dictatorship and the other two are cartoon characters from The Simpsons. The paper was accepted by both the Journal of Computational Intelligence & Electronic Systems and The Aperito Journal of NanoScience Technology. Smolyanitsky continues to receive invoices from the journals for a $459 publishing fee.

SCIgen generates phony computer science papers. Another paper generator, Mathgen, specializes in phony mathematics. Fictitious author Professor Marcie Rathke had Mathgen produce the paper “Independent, Negative, Canonically Turing Arrows of Equations and Problems in Applied Formal PDE”. It was accepted for publication by the impressive sounding journal Advances in Pure Mathematics. The editor’s acceptance letter begins:

Thank you for your contribution to the Advances in Pure Mathematics (APM). We are pleased to inform you that your manuscript … has been accepted. Congratulations!

I’m not sure how a computer-generated paper can be effectively reviewed, but the editors of this junk journal actually gave it a try. Remember while you read, the paper has been accepted. All it needs is some fine-tuning. In the review, the editor writes We can’t catch the main thought from this abstract. This cracked me up! For some reason, the editor could not catch the main point of pure gibberish. He continues. In this paper, we may find that there are so many mathematical expressions and notations. But the author doesn’t give any introduction for them. That’s because no explanation for the paper exists.

Publication of Rathke’s paper was again marred by the requirement of a $500 processing fee.

Phony SCIgen papers have also made it into reputable conferences. Springer, a large German based publishing house, has published a number of SCIgen papers. After signing a contract, Springer also recently banned publication of a legitimate project of mine involving intelligent design without even reading a word of the work. Yet Springer reviews phony SCIgen papers and then publishes them. The strange story of my rejection illustrates the deeply entrenched ideology of some publishers. Here’s the story.

In 2011, there was a Biological Information: New Perspectives symposium held at conference facilities on the campus of Cornell University. Most of the participants were intelligent design advocates. John Sanford, William Dembski, Michael Behe, Bruce Gordon and I edited a book from the conference. As editors, we decided the papers would not deal with theology or philosophy. Only science, mathematics, and engineering were allowed. We were invited by Springer to publish the collected works. Co-Editor William Dembski brokered the deal with Springer. I liked the idea. Two other of my previous books were published by Springer. Both of my books were recently reissued in paperback.

So far so good.

After hours of reviewing and editing manuscripts, we were a day or so away from bringing the collected papers to press. The book was listed on both the Amazon and Barnes & Noble websites under the Springer logo with a to be released notice. But an anti-ID zealot noticed the book listed on Amazon and contacted some top brass at Springer claiming our book would besmirch Springer’s reputation because the book’s editors were closely identified with intelligent design. Even though neither Springer’s top brass nor the complainer read a single page of the book, Springer pulled the plug.

We tried appealing to Springer’s better side, but were unable to find it. Our contract with Springer was solidly on our side, but lawyers told us taking a German company to German court would be a long, difficult, and expensive process. Plus, life is too short. So we put thumbs to noses and waved goodbye to Springer. The book was ultimately published by World Scientific. They did a great job. The book is now available online for free. For those who enjoy Reader’s Digest condensed versions, see John Sanford’s well-written and concise synopsis, available at Amazon. There is also a series of videos from Loma Linda University discussing the book. For those so inclined, take a look and let me know if there is anything but solid science and math there. If something slipped through, I want to know.

Speaking of besmirched reputations and SCIgen, Springer was recently informed that a number of papers published in its edited volumes were generated by SCIgen. When I first heard this, I confess to experiencing sinful albeit gleeful schadenfreude about my old friend, Springer. The authors of the phony journal papers had a different motivation than did undercover pranksters Shrime and Smolyanitsky. The authors were legitimately trying to make their piles of beans higher. Springer contacted the authors of the SCIgen papers, who confirmed that their submissions were not intended as a hoax. The intention seems to have been to increase their publication numbers and to increase their standing in their respective disciplines and at their institutions. Springer has retracted the papers, but you can still reference them in your papers if so inclined. An example of a Springer published SCIgen paper is:

Sun Ping. Application of Amphibious Technology in the ReutoMail. In Proceedings of the 2012 International Conference on Communication, Electronics and Automation Engineering, pp. 409-413. Springer Berlin Heidelberg, 2013.

By simply listing the paper here, the citation count of Ping’s paper as tallied by Google Scholar will increase by one. From the perspective of pure bean counting, Springer’s publication impact factor will correspondingly increase a bit, and Ping’s h-index may also be helped. I’m uncertain whether or not Ping’s paper is still included in Springer’s book or not. You can find out by buying a used copy on Amazon for $525.53 or a new version for $319.00 at the Springer website. If you find out whether or not Ping’s paper is still in the book, let me know. I’d love to see a copy.

Compared to the large number of papers published by Springer, the number of SCIgen papers is small. To its credit, Springer is trying to scrub off some of the mud by taking steps to make sure publication of phony papers doesn’t happen again. But I suspect Springer will always appear muddy to me.

One of supply side academic’s prime currencies remains publication count. In promotion and tenure cases, The Dean can’t read, but the Dean can count. One would think the clear case of bean abuse would enlighten academic administrators. It hasn’t. They have instead decided to abuse different beans.

Robert J. Marks II is a Distinguished Professor of Electrical & Computer Engineering at Baylor University in Waco, Texas. The material in this column, though, does not necessarily represent the views of and has not been reviewed or approved by Baylor University.

Notes