Loomio
Mon 3 Aug 2015 9:59PM

PeerJ possibility report, discussion

AH Alex Holcombe Public Seen by 198

for background on PeerJ see here https://www.loomio.org/d/VIgtdbXP/call-for-questions-for-peerj

I spoke to PeerJ founders Pete Binfield (who was formerly the executive editor of PLoS ONE) and Jason Hoyt.

PeerJ is a mega-journal in the style of PLoS ONE or Scientific Reports. For computer science, they created essentially a new journal, PeerJ Computer Science, running on the same infrastructure and tightly linked to PeerJ. They are not interested in doing this for perception/vision however. The numbers of potential authors are too low for them to have any confidence it would yield a vibrant community. The pool of potential authors in computer science is >>100,000, much bigger than the few thousand (?) in perception science.

But they are interested in creating sub-sections of their journal that are sort of like sub-journals but not as autonomous. For PeerJ, these provide several advantages over an undifferentiated mega-journal. They mentioned two such options:

Collections

So far collections have been created for papers on a very specific topic, or for those associated with a conference. They can be normal, peer-reviewed manuscripts accepted by PeerJ, or PeerJ preprints, including posters. An example is "The biology of the Hawaiian Archipelago" collection, an example of one for a conference is https://peerj.com/collections/15-msw15/ . The way this works currently (they seem open to potential modifications) is that an editor who proposes the collection decides which already-accepted articles are part of the collection and which aren't. This could be a good way for the community to dip our toe into the waters at PeerJ.

For example, ECVP might decide they'd like to highlight the award-winning posters by creating a PeerJ collection of them. The authors of said posters, should they like the idea of being included, would submit their posters to PeerJ Preprints. A person from ECVP would then approve them for inclusion in the collection. (I don't know whether ECVP is giving poster awards this year, and perhaps if they do only give a few, not worth a collection, but one idea is to include an entire "longlist" of posters that were in the running for the award). An alternative is to create a Collection that is essentially a special issue, if we can think of a topic that would attract good authors.

This is a purely informational post, I won't advocate here for my preference- a PeerJ perception subject page :)

Subject pages

A subject page is more like a journal within a journal. A subject page points to all the articles accepted into PeerJ on a particular topic. There is one for Psychiatry and Psychology, which is mostly automated and has a "Latest articles and preprints" section and feed, also "Widely read", etc., and "Editors' picks" which I believe by default is those articles highly rated by the accepting editors. Along the right is a list of "Editors and advisors".
There is no editorial content in the sense of stuff (editorials) written by editors but PeerJ is open to that, I think they said they're planning on adding functionality for that. The idea is to create a community feel, and I think we / the editors of Perception/vision within PeerJ would want to be able to write editorials.
Notice the ways that a PeerJ Perception or Vision subject page would differ from a separate journal

  • Collections and Subject Pages don’t assign their own ISSN nor citation. The citation for each article would be to PeerJ (the journal) or PeerJ PrePrints
  • Same editorial standards as the rest of PeerJ, which is essentially just like PLoS ONE, importance not a criterion, just good methodology. It is quite subjective ultimately what counts as good methodology though, and basically the editors in the area decide that. However there currently is no facility for a chief editor of a subject, so there is no built-in across-editor organization.

Other issues

  • Institutional funding to reduce the (already-low) author fees? It's possible. #1, institutional memberships exist, e.g. Jon Peirce said his uni is a member, making publishing at PeerJ free. Also, bulk buying already exists for handing out to authors of your choice.
  • How do we know they won't sell out to Elsevier? They said they unfortunately can't guarantee anything. One of the co-founders, Jason Hoyt, came from Mendeley and from his post at the time you can see he was very disappointed (somewhat guarded, he was talking about his colleagues) in them selling out to Elsevier http://enjoythedisruption.com/post/47527556151/my-thoughts-on-mendeleyelsevier-why-i-left-to
  • They don't publish traditional review articles where an author surveys the field and writes their opinion. They do publish however things that look like meta-analyses or that result in plots of evidence mined from the literature, e.g. https://peerj.com/preprints/921/
  • They have nice support for embedded movies which they host themselves, and create both small download and large download versions.
LD

Lee de-Wit Tue 4 Aug 2015 8:34AM

Thanks for this Alex. It is logical they would not see PeerJ Perception/Vision as a possibility, but obviously it is a pity for us. Not being able to set our own editorial standards would be a big loss, I think.

That said, the formats within PeerJ could be worth trying, particularly the 'Collections', which could indeed 'be a good way for the community to dip our toe into the waters at PeerJ', with a special issue.

I'm going to try and contact http://www.ubiquitypress.com/ to hold a similar information gathering discussion with them. Their costs are also very low, and they seem to offer a much more straightforward platform for creating a traditional journal, so I think it would be good to know more from them to compare.

AH

Alex Holcombe Tue 4 Aug 2015 9:15AM

Ubiquity would be a good choice too, I spoke with Brian Hole (their CEO) three years ago when Dan Simons and I were thinking of starting a registered replications journal before APS got on board with the idea.. I also joined the editorial board of Journal of Open Psychology Data that Jelte Wicherts started with them a few years ago http://openpsychologydata.metajnl.com/ ( http://openpsychologydata.metajnl.com/ ), although it hasn't caught on much. They use customised versions of Open Journal Systems to keep costs low. At the time they were expecting an annual contribution of 1,000 GBP, usually paid for by the society backing the journal. Then there's also the author fees, but they keep those very low I think.

They may not have the nice layout, submission system, preprints, optional open peer review workflow, and science-related ecosystem that PeerJ has created, but still a good option especially because our community would have total control of editorials, editorial policy, structure, and process.

TW

Tom Wallis Tue 4 Aug 2015 9:41AM

Thanks for gathering valuable info, Alex. While it's disappointing that PeerJ aren't interested in a separate Perception journal, I'm not too surprised. We're just not a big enough field for them.

Collections and Subject Pages don’t assign their own ISSN nor citation. The citation for each article would be to PeerJ (the journal) or PeerJ PrePrints.

In terms of placating administration-led focus on journal impact factors this would probably be a good thing, no? Since impact factors are largely a measure of the size of a field, PeerJ vision articles would benefit from the huge size of the biology field and associated citation rates.

As for subject pages: I see that on "psychiatry and psychology" there are a number of "editors and advisors". How are these editors chosen?

Perhaps one way forward would be to create a Perception Science (or Vision Science) subject page at PeerJ with a team of editors our community elects. These editors could agree on an editorial standard for the subject page, and the review of all articles appearing under the subject page would be supervised by one of those editors (who would also solicit reviews). In that way we could set editorial standards we feel are appropriate for our community. Do you think this would be feasible, or do you think these articles would get lost?

Do you think there would be scope to consider different review systems (e.g., the eLife system I detailed on another page)?

AH

Alex Holcombe Tue 4 Aug 2015 7:59PM

Tom yes, I suspect the PeerJ impact factor will continue to be 2 or higher, whereas an entirely new journal would not have an impact factor for a few years - as we've said a major obstacle to attracting submissions.
About how the editors are chosen, I think it's ultimately by the publisher Pete, but probably there's a role for the advisers and academic editors, I'll ask Pete. We would want this new perception area's editors to have say-so in appointing new editors, wouldn't we?
About the editorial standards, my feeling from talking to PeerJ is that they are not inclined to modify their written evaluation standards (which I have pasted below), they apply to all fields and I doubt they would allow us to write a distinct set.
However, note that what counts as their "high technical standard" is somewhat subjective, and probably they'd encourage communication among the editors in an area about what precisely that means, and maybe we could give more explanation of what it means in our subject's editorial content.
Everyone should also note their stricture that "The data on which the conclusions are based must be provided or made available in an acceptable discipline-specific repository", quite different from all perception-specific journals of which I am currently aware.

  • The investigation must have been conducted rigorously and to a high technical standard.
  • Methods should be described with sufficient information to be reproducible by another investigator.
  • The data should be robust, statistically sound, and controlled.
  • The data on which the conclusions are based must be provided or made available in an acceptable discipline-specific repository.
AH

Alex Holcombe Tue 4 Aug 2015 9:48PM

For editorials, Pete wrote back to say they don't have any facility for that but are interested in developing it. An editor would write it and submit it as a preprint and then it would go on the subject page as highlighted editorial content.
They don't know what the process would be for deciding who gets to write the editorials, but seem open to discussing it.

Currently Academic Editors on PeerJ need to be at PI level normally with >=25 journal articles on their CV. Unfortunately seems that would exclude some of the people who are most enthusiastic about this initiative, but perhaps something could be arranged. So far there's been no formal editor-level decisionmaking or organisation at each subject, but we could propose something for handling our putative subject page and editorials.

TW

Tom Wallis Wed 5 Aug 2015 1:28PM

I like the idea that the editorial group could have an additional document for field-specific notes defining "high technical standard". I think the main thing would be to convince the community that rigorous standards will be upheld, and that papers will be rejected from the journal (unlike the perception of e.g. Frontiers).

Further to the subject page: do you think the editorial group for that subject page would have control over what keywords / tags could be used by PeerJ articles? That is, could someone just tag their paper as "perception science" (or whatever the subject page gets called) without going through one of the community's approved editors? If editors don't have control over the tags, I'd be worried that sub-par research slips in by being reviewed by non-experts but tagged as "perception science".

As for the Academic Editor requirements: that's a personal bummer for me, but I think overall it's better for the new journal to focus on appointing a well known editorial team rather than stocking it full of unknown postdocs. Getting a couple of high profile figures on board would help to boost submissions (and the enthusiastic postdocs can submit the first articles!).

NS

Nick Scott-Samuel Wed 5 Aug 2015 3:09PM

I'm strongly in favour of a balanced editorial team, with both established types and up-and-coming postdocs. One reason (not the only one, for sure) is that the latter group are, as Tom points out, enthusiastic. I think that this counts for rather a lot, if my experience at Perception is anything to go by. Furthermore, the whole drive for this idea has come predominantly from younger researchers. It doesn't feel right that they wouldn't have a big say in how things develop.

AH

Alex Holcombe Wed 5 Aug 2015 7:07PM

Tom I think one would tag their paper with "perception" upon submission and that should send it to one of the corresponding editors. That's really important to check though. Also note that while the subject page has an automatic feed of all such papers, there's also the featured content which could be hand-picked by the editors.

LD

Lee de-Wit Wed 5 Aug 2015 9:28PM

Even if we could not set separate editorial standards, I must say, I'm very happy with the criteria Alex lists. I'm also happy with the PI +>=25 paper criteria for editors. I think it's more important people respect the editorial process, than that younger enthusiasts get to be editors. Personally I'd be very happy to just have a journal I am actually happy submitting to - I don't particularly care about being an editor of it. The bigger problem for me is the 'chief editor(/s)' role, someone needs to make a subjective decision after the criteria have been meet, and that surely has to be an expert in the field, not administrators at PeerJ. But again, sounds like they might be up for discussing that if we set up a subject page.

AH

Alex Holcombe Thu 6 Aug 2015 12:23AM

I'm hoping to hear more opinions and also the outcomes of deeper dives into other possibilities like Ubiquity Press, but if people continue to really like the PeerJ idea, we should be thinking about moving toward a concrete proposal for dipping the community's toes in the waters of PeerJ, something like publishing abstracts there from ECVP or a symposium or satellite, e.g. of ECVP or VSS. Would be great to have something concrete for people at the ECVP forum to sign on and commit to. Maybe there should be a fleshed-out proposal both for a subject page and the less-commitment test the waters thing. I suppose such proposals should be worked out on a separate thread.

One thing we haven't detailed much is the pricing structure at PeerJ and its consequences. I believe that, absent an institutional membership(?), all authors of a paper need to be PeerJ members for a paper to be published, at a minimum of $99/per author https://peerj.com/pricing/. Once you've paid those $99, you can publish one paper per year for life (or unlimited papers, if you instead pay $199). You can see how this unusual scheme is designed to rapidly grow the number of PeerJ members. Once I become a member, my incentive is to get my colleagues to sign up because although it costs my colleagues money to buy in, it won't cost me anything to publish additional papers.

By the way, this part of the discussion forum is currently public - anyone can read it, in case you didn't realize :)

NS

Nick Scott-Samuel Thu 6 Aug 2015 7:24AM

Lee, you say you "think it’s more important people respect the editorial process, than that younger enthusiasts get to be editors". Fair enough, but have a look at who's driving this discussion. I think you need both, because my experience is that the <25 types are more likely to get things done.

MB

Marco Bertamini Thu 6 Aug 2015 9:23AM

Alex, thanks for all the work on this.

I prefer the option of 'subject pages' rather than a collection. It would be good (a point already raised) if only papers that have been dealt by a specific set of editors would be on these pages, or have a specific tag, avoiding the risk that something edited by people outside what we feel is the vision/perception area gets mixed in. The reason this is important is not so much to control standards but to avoid a diluting of the scope.

The creation of such journal within a journal could be a specific proposal for the discussion at ECVP. Ideally the proposal can be circulated before and some people will be on board by then and take the lead.

As for publishing ECVP abstracts, this year we have an arrangement with Perception, they have always published the abstracts and we were keen to maintain the continuity. The arrangement is complex because although it costs money to publish the abstracts they are also sponsors of the conference (so it's a two-way deal).

There has been talk of special issues related to satellite meetings (we have two, and they are not covered by the Perception arrangement) and the illusions parade (but for that we are well into discussion with another journal).

The future may see developments because various things have changed this year. First of all we are now operating an entirely electronic abstract publication, secondly the publisher of Perception has changed from a small company (Pion) to a large one (Sage).

TW

Tom Wallis Fri 7 Aug 2015 12:20PM

Alex: in your outline of the PeerJ membership system, you neglected to mention what I think is the coolest feature (and also a slight catch, for those not reading carefully). The $99 lifetime membership is contingent on you contributing to the community via a review or comment (post-pub review) once per year:

We aim to make PeerJ a community, and no one is forced to provide a review if they choose not to do so. To help the community though, we are incentivizing participation by inviting those with paid publishing plans to submit a review at least once per year (and we consider a 'review' to be an informal comment on a submission to PeerJ PrePrints; a formally requested peer review* of a paper submitted to PeerJ; or an informal comment on a published paper). If you choose not to perform at least one review every 12 months, then at our discretion your publishing plan will lapse and you will need to pay $99 to reactivate your plan the next time you want to publish with PeerJ. We think this give-and-take is fair to the community as it incentivizes participation in the ongoing task of peer review and will collectively reduce everyone's burden.

I think this is a large part of the reason that authors report such positive and fast reviews -- because members of PeerJ are motivated to do at least one review per year. The fact that comments on papers also counts is one way that I feel the platform might help increase post-publication reviews in a way that e.g. PLoS's comment system has failed to do.

TW

Tom Wallis Fri 7 Aug 2015 12:21PM

(the above quote is taken from the FAQ page, under "What if I don't do a review when invited?"...