July 20-24, 2015

Background

Schloss Dagstuhl is an informatics retreat supported by the Liebniz-Centrum for Informatik and located just outside Wadern Germany in the German state of Saarland.  Each week, an invited computer science seminar is hosted based on applications made by organizers.  Organizers in turn invite participants.  The seminar I attended was 15302, Open Science in Psychology and the Behavioral Sciences.  The attendees were Xavier Aime, France, Dietrich Albert, Germany, Bjorn Brembs, Germany, Mike Conlon, US, Oscar Corcho, Spain, Alex Garcia Castro, Paul Groth, Netherlands, William Gunn, US, Susann Fiedler, Germany, Janna Hastings, UK, Carline Jay, UK, Iris-Tatjiana Kolasssa, Germany, Silvia Koller, Brazil, Christoph Lange, Maryann Maertone, US, Russell Poldrack, US, Alec Schmecher, Canada, Daniel Staemmler, Germany, Robert Stevens, UK, Gary vandenBos, US, Hal Warren, US, Eric Weichselgarter, Germany.

Purpose

The purpose of the seminar was to bring together a diverse group of scientists and practitioners to understand the current conduct of Pyschology and the Behavioral Sciences, and to make recommendations regarding the application of computer science to the betterment of the displines.

Format

Invitees presented on their projects and background – see my intro and VIVO intro presentations.  The presentation topics and presenters were:

1. Psychology.  Aime, Albert, Jay, Kolassa, Koller, Poldrack, Weichselgartner
2. Data practices and projects.  Alter, Brembs, Conlon, Fiedler, Garcia Castro, Martone
3. Publishing.  Groth, Gunn, Schmecher, Warren, Staemmler, vandenBos
4. Ontology and information representation.  Hastings, Stevens
5. Computer Science.  Lange, Corcho

Many of the presentations were multi-disciplinary and covered a number of these areas.  Most presentations were 20-30 minutes long and grouped into clusters of 5 or 6.

Mid morning and mid-afternoon breaks created approximately two-hour sessions.  Some sessions consisted of presentations, some of joint discussion, some of workgroup breakouts and some of breakout read backs.

Meals with assigned seating, randomly determined by algorithm facilitated workshop attendees meeting and discussing topics of common interest at lunch and dinner.  All meals and accommodations were held at Schloss Dagstuhl.

Wednesday afternoon, the seminar attendees visited Trier, an ancient Roman city on the Luxembourg border.  The Roman emporer Constantine built a spectacular amphitheater in Trier that was partially destroyed in the second world war and then restored.

Evenings were an opportunity for additional conversation regarding the topics of the day.  The surrounding forests and many rooms of the castle and new additions provided ample space.  The seminar ended mid-day Friday.  Friday evening, remaining guests biked to nearby Wadern for dinner.

Themes

Over the course of the week, the participants sketched out a vision for the future of eScience consistent with a previous Dagstuhl seminar in 2011, which led to the formation of Force11.  Force11 advocates for open science according to a set of principles known as “FAIR” – Findable, Interoperable, Accessible, and Reproducible.  Two large issues in the conduct of psychology and the behavorial sciences were identified:

1. Legitimacy – is the science good science?  Publication bias, misuse of statistical reasoning, sampling bias, poor technique, inappropriate measures and other issues lead to a crisis in reproducibility and cast doubt on the field.
2. Agility – are investigators able to get work done?  The growing regulatory burdens, grant and financial issues, training gaps, inability to assemble teams, recruiting limitations and  other barriers create concern for the productivity of investigators in the disciplines.

FAIR principles, Research Objects, ontologies, persistent identifiers (including ORCID), credit systems, and repositories (such as Dspace and VIVO), were seen as potential elements of remedies.

Outcomes

The group will joint write and edit a manifesto regarding the current state of eScience in Psychology and the Behavioral Sciences, including strategies for creating new opportunities for scholarship.  Pilot projects will be proposed.  The manifesto will appear as a publication of the Dagstuhl series and will be completed in January 2016.