2019-07-18 DSpace 7 Working Group Meeting

Date

18 Jul 2019 from 14:00-15:00 UTC

Location: https://duraspace.zoom.us/my/dspace (Meeting ID: 502 527 3040).

• More connection options available at DSpace Meeting Room

Agenda

• (15 mins) Developer Stand Up - Developers give brief updates on their effort (or their team's effort).

  • Update/see "Current Work" section below based on your status. Please feel free to update prior to meeting.
  • Please highlight any new work (needing reviews/testing), any blockers (for you), and any discussion topics you may have.
• (30 mins) General Discussion Topics
  
  1. Overview/Discussion of initial DSpace 7 Performance Testing by Chris Wilper : https://cwilper.github.io/dspace-perftest/
  2. Redesign of Community/Collection homepages by Laura Henze: https://github.com/DSpace/dspace-angular/issues/365
     1.  See: Community page mockups
  3. Improved Estimation Strategy for Beta release. (Estimates should include time for discussion / code reviews, also should be done by multiple developers)
     1. Development Planning Spreadsheet needs to be updated, so that we can determine what (outstanding) tasks need estimation.
     2. Will be using Wideband Delphi estimation technique
• (15 mins) Planning for next week
  
  • Assigning PRs for Review
  • Next tasks from Development Planning Spreadsheet

Attendees

• Art Lowel (Atmire) 

• Andrea Bollini (4Science) 

• Tim Donohue 

• Heather Greer Klein

• Lieven Droogmans

• Mark H. Wood
• Giuseppe Digilio (4Science)
• Ben Bosman
• Pascal-Nicolas Becker
• Chris Wilper
• Terrence W Brady
• Pablo Prieto
• Paulo Graça 
• Bill Tantzen

Current Work

Tickets / PRs In Progress

1. (Angular) Adding Accessibility via Travis CI  https://github.com/DSpace/dspace-angular/pull/356 (work in progress) (Lower priority)
2.  (Angular Bug) https://github.com/DSpace/dspace-angular/issues/368 ( Art Lowel (Atmire) )
3. (REST Contract) Edit Homepage news: https://github.com/DSpace/Rest7Contract/pull/45 (Ben Bosman  - has outstanding questions/comments) (Lower priority)
4. (REST) DS-4043: Revisit the security layer of the submission  (work in progress) Andrea Bollini (4Science)
5.  (REST Contract) Group and eperson management: https://github.com/DSpace/Rest7Contract/pull/41 (Waiting on updates fromBen Bosman )
6.  (REST) Pagination issues on Items findAll - https://github.com/DSpace/DSpace/pull/2406 (Waiting for confirmation of proposed approach by Andrea Bollini (4Science) from  Kevin Van de Velde (Atmire)  and any other interested/available )
7. (Backend) DS-4257: Renaming "dspace-spring-rest" to "dspace-server" (Tim Donohue).  (Timeline: Not likely to be completed until after OR2019)
8. (Angular) Transfer to .po message format - Initial PR: https://github.com/DSpace/dspace-angular/pull/366 (Paulo Graça,   Tim Donohue , Art Lowel (Atmire) )

PRs Needing Review

1. (REST Contract) Collecting statistics - https://github.com/DSpace/Rest7Contract/pull/63 (Dimitris Pierrakos, Andrea Bollini (4Science) -  changes requested, Ben Bosman )
2. (NEW) (REST Contract) OAI Harvesting configuration - https://github.com/DSpace/Rest7Contract/pull/66 (Paulo Graça, Tim Donohue)
3. (REST) Item Mapper functionality: https://github.com/DSpace/DSpace/pull/2282  ( Tim Donohue , Ben Bosman - merge conflicts resolved)
   
4. (Higher priority) (REST) [DS-4266] bitstream format registry https://github.com/DSpace/DSpace/pull/2442 (Ben Bosman, Andrea Bollini (4Science) -  ), merged during the meeting
5. (REST) Pagination bug with withdrawn items: https://github.com/DSpace/DSpace/pull/2406 (Dimitris Pierrakos , Ben Bosman - Feedback provided)
6. (Angular) (Entities) Deleting relationships: https://github.com/DSpace/dspace-angular/pull/402 (Paulo Graça - will test again, Tim Donohue )
7. (Angular) Move Item Component: https://github.com/DSpace/dspace-angular/pull/335 (Giuseppe Digilio (4Science) reviewed and provided feedback, Tim Donohue )
8. (Angular) Item-Collection Mapper:  https://github.com/DSpace/dspace-angular/pull/348 (NEEDS REVIEWERS)
9. (Angular) Add missing ResourcePolicyService: https://github.com/DSpace/dspace-angular/pull/427 (Giuseppe Digilio (4Science)approved, Art Lowel (Atmire))
10. (Angular) Shibboleth integration support (WORK IN PROGRESS): https://github.com/DSpace/dspace-angular/pull/429  (Julius has updated the PR. Ready for RE-REVIEW) (Giuseppe Digilio (4Science), Fernando FCT/FCCN, Paulo Graça )
    1. 4Science will enable Shibboleth on public demo.
11. (NEW) (Angular) Submission Miscellaneous fixes: https://github.com/DSpace/dspace-angular/pull/432 (Art Lowel (Atmire), Julius Gruber)
12. (NEW) (Angular) Replace model factories https://github.com/DSpace/dspace-angular/pull/434 (Tim Donohue, Giuseppe Digilio (4Science))
13. (Backend) Upgrading to Handle Server v9: https://github.com/DSpace/DSpace/pull/2394 (Mark H. Wood, Ben Bosman )
14. (Backend) Update email templates to use Velocity (for richer, powerful templates): https://github.com/DSpace/DSpace/pull/1992 (Chris Wilper , Tim Donohue, Art Lowel (Atmire) - reviewed, provided feedback)
15. (Week of July 22) (Backend) One Webapp Phase 2: Rename "dspace-spring-rest" to "dspace-server-webapp": https://github.com/DSpace/DSpace/pull/2459 (Mark H. Wood, Alexander Sulfrian, Terrence W Brady)

PRs Merged this week!

1.  (REST) [DS-4298] changed the PathVariable for the upload method: https://github.com/DSpace/DSpace/pull/2467
2.  (REST) (Entities) DS-4223 Metadata Schemas for configurable entities https://github.com/DSpace/DSpace/pull/2443
3.  (REST) [DS-4266] bitstream format registry https://github.com/DSpace/DSpace/pull/2442
4.  (Angular) (Entities) DS-4223 Metadata Schemas for configurable entities https://github.com/DSpace/dspace-angular/pull/420
5.  (Angular) Configurable browse-by https://github.com/DSpace/dspace-angular/pull/421 (closes #361)
6.  (Angular) Selectable Themes. https://github.com/DSpace/dspace-angular/pull/419

Blocked

1. (Blocked PRs go here)

Delayed / Needs Discussion

1. Initial Performance Testing from Chris.
   • https://cwilper.github.io/dspace-perftest/
2. (REST Contract) Edit Homepage News: https://github.com/DSpace/Rest7Contract/pull/45
   
   1. Delayed until after Preview release. General agreement (in meeting on March 21, 2019) that storing HTML in metadata fields is not really ideal behavior.  Metadata (from a librarian standpoint) tends to be free of format-related markup (as that allows for easier sharing, understanding of metadata.  Currently Community & Collection homepage information is HTML-based and is stored in metadata that is appropriate for a minor subset of information (like the title) but it is better to move large/rich text to bitstreams.  
   2. Proposal here is to consider storing HTML-based markup (for Site, Community & Collection homepages) in Bitstream(s) associated with the object in question.  May allow for more CMS-lite behavior in the future
   3. Timeline for this is uncertain.  Possibly in 7 or 8. May depend on how/whether it can be scoped.
3. (REST) Scripts & Processes endpoint: https://github.com/DSpace/Rest7Contract/pull/17
4. Concurrency in DSpace 7 (or 8).  What do we want to do when multiple editors are editing the same object?  Needs further analysis regarding implementation details
   1. We've decided (in meeting on March 7, 2019) to use ETags to implement concurrency. REST Contract notes on ETags: https://github.com/DSpace/Rest7Contract#etags--conditional-headers
   2. ETags only update of the two fields match. If someone edits first, your edit would fail and you would get a fail response (422?)
   3. ETags seems to have broader support in other REST APIs.  Recommended also by both Art and Andrea.

5. Not discussed much, but could there be opportunities to use Travis CI + Docker Compose for testing of Angular??

   • Docs: https://docs.travis-ci.com/user/docker/#using-docker-compose
   • Blogpost on how to do it: http://elliot.land/post/using-docker-compose-on-travis-ci
   • An example travis.yml file: https://github.com/Ortus-Solutions/docker-buildfiles/blob/master/.travis.example.yml

Notes

• Chris walked us through early performance testing results at https://cwilper.github.io/dspace-perftest/
  • Test data set & more info is at: DSpace 7 Performance Testing
  • A few areas that need work were called out:
    • Top Communities list on homepage takes over 2 seconds, see this table: https://cwilper.github.io/dspace-perftest/report/2019-06-04/7.0-preview-1/details-big#big-a-home-requests
    • Browse By Title with lots of larger bitstreams takes forever (26+ seconds), see this table: https://cwilper.github.io/dspace-perftest/report/2019-06-04/7.0-preview-1/details-browse
    • TODO: CREATE TICKETS for both of these specific performance issues, as they seem high priority.
    • UPDATE : Tickets were created to log these performance bugs:
      • Unable to locate Jira server for this macro. It may be due to Application Link configuration.
      • Unable to locate Jira server for this macro. It may be due to Application Link configuration.
  • Ideally, all REST queries should be <2 seconds (at slowest)
  • Andrea points out overlaps here with Projection discussions, see  Unable to locate Jira server for this macro. It may be due to Application Link configuration.
    • Projections would let the UI limit what info is returned per request.  This could speed up individual queries significantly, as often the UI doesn't need all metadata or all bitstreams... but currently all queries return a lot of embedded content, etc.
    • Need to revisit Projections in light of these results
  • Lots of suggestions for future performance testing areas (to enhance future data sets, testing points)
    • Testing performance from an SEO perspective (tests that act as a web crawler)
    • Testing performance of Entities (e.g. an article with many authors, or an person with many articles).  Paulo mentioned he has some real life data that could be used here
  • Additional suggestions should be added to Wiki page at DSpace 7 Performance Testing
• Laura briefly overviewed her design proposal for Community pages:
  •  See: Community page mockups
  • Designs were based on local user feedback at NYU
  • Discussion here was limited (as first topic ran long).  However, all are encouraged to bring these designs back to your institution for feedback/analysis.  Also looking for additional "Design objectives" (based on other local user testing).
  • REVISIT in meeting on August 1 (in two weeks, because Laura is out next week).  This topic will be first on the agenda that week.