Pre-registration & Earth Science?

I'm VERY interested in this, having followed the evolution of pre-registrations in areas of psychology. Indeed, Earth Science would definitely benefit from this sort of approach, at least partly to stop the equivalent of p-hacking (https://bitesizebio.com/31497/guilty-p-hacking/). We've all done it, to some extent...

If you'd like to modify an existing template, make a copy of this one and change it to be more relevant to Earth sciences: https://docs.google.com/document/d/1DaNmJEtBy04bq1l5OxS4JAscdZEkUGATURWwnBKLYxk/edit?pli=1 You can then post it on an OSF project (like this one https://osf.io/x4gzt/) and try out some studies using it to see what works or how to improve it. If there is interest it could be a standard option on the OSF registry. (Other templates that you could consider modifying are listed here: https://osf.io/zab38/wiki/home/).

Here https://www.frontiersin.org/articles/10.3389/fpsyg.2018.01831/full an interesting reading about pre-registration.

Thanks, @Domenico Chiarella! This is very interesting, and it has got me wondering how this might relate to the types of stuff we do in geosciences. For example, I guess you could pre-register a hypotheses and set of tests for, for example, structural controls on sedimentation in a rift? Or for some petrological and geochemical analysis of rocks thought to record a specific geological event/condition?

Yes, and I think for experimental geochemists this certainly works. We do experiments in the lab to explicitly test certain hypotheses.

@Christopher Jackson, I though to it as well but dont have a clear answer yet. There are definetely area/questions where it is possible to pre-register an hypotheses - especially if we want to test a specific idea like in the two examples you mentioned - but sometimes we start to work in an area with no clear in mind where we will end. Just as example, I remember when strated my PhD project, my supervisor sent me to an area with as main info the stratigraphic interval to focus on and nothign else. Obviously, this example doesn't represent the normality but for sure after we start to study an area we can come out with much more (and different?) information respect what we had originally in mind. Does this means that pre-registration works for some type of project and not for everyone? Is my view too limited and I am not seeing the full spectrum of pre-registration use and value? For sure, I think the use of pre-registration is (still?) not so common in geoscience, and probably it could be interesting to investigate more.

Thanks all for the interesting conversation. Other fields have approached pre-registration to deal with p-hacking--I'm not sure what/if there's an equivalent problem in earth sciences. Another aspect of pre-registration is the time stamp and I think for many, a preprint itself is enough of a time stamp. There could be value in registering hypotheses in order to make the distinction between confirmatory & exploratory work as in @Domenico Chiarella 's example, or to empower folks to publish the equivalent of "null" results if they're beholden to registered hypotheses. Anyway, I'd be happy to revisit this when the dust settles after the announcement!

I have never come across a pre-registered study in Earth Science. (I am in geophysics, mostly earthquake research). Among my own papers, I can think of one that could have been suitable for pre-registration; it involved a "blind" statistical analysis of data with very low signal-to-noise ratio. In my community, a more common practice that addresses some issues targeted by pre-registration is cross-validation: test a model on data that was left out during the model construction. It's even more systematically done now that Machine Learning is booming (training dataset vs testing dataset).

Ditto. I am struggling a little to see how it might work with some of the stuff we do...although I am aware that there has been times when we've found, by chance, something interest, rather than posing a specific, testable hypothesis. This is what @Domenico Chiarella is thinking about. Those chance encounters with something cool! My career has been about that...:-)

I am not aware of an study using pre-registration in climatology. So if we find more I would be interested.

However, with a group I am working on one. In the ISTI we are generating a dataset that will be used to test data processing tools. We wanted to write a paper about how the dataset is generated and one on how the processing tools will be assessed. Reading this question I realize that this would be a pre-registration of the analysis. In this case the pre-registration would not be blind, as would be normally the case, and we mainly do it so that the other participants know how they will be judged, not to get complaints afterwards that the analysis was not fair for a specific tool.

Pre-registering climate studies will be hard. On the one hand, in climatology you normally already know how the data will look, even if you gather new data for a more precise estimate, how the climate changes is known. On the other hand the analysis tends to be unique. As the data is not particularly new, you tend to make a paper worthwhile by doing new types of analysis and then it is hard to know in advance what to expect and accurately write up how you will do this.

If EarthArXiv would do blind pre-registrations, we would need a mechanism to upload a document that will be published later at a fixed date (or earlier if the authors want). But maybe we best leave this to COS, especially as long as pre-registrations are rare.

Such a mechanism would also be handy in case of embargoes. Then someone could upload their post-print when the article is accepted and it would be automatically made public when the embargo of the publisher ends. Being able to do this when publishing and not having to do this one year later by hand likely increases the number of uploads.