2019-09-26 DSpace 7 Working Group Meeting

Date

26 Sep 2019 from 14:00-15:00 UTC

Location: https://lyrasis.zoom.us/my/dspace (Meeting ID: 502 527 3040).

• More connection options available at DSpace Meeting Room

Agenda

• (15 mins) Developer Stand Up - Developers give brief updates on their effort (or their team's effort).

  • Update/see "Current Work" section below based on your status. Please feel free to update prior to meeting.
  • Please highlight any new work (needing reviews/testing), any blockers (for you), and any discussion topics you may have.
• (30 mins) General Discussion Topics
  
  1. Changing the collection in the submission doesn't change the item template 
     1. How should this work, if you change the collection after you've already filled out some metadata?
• (15 mins) Planning for next week
  
  • Assigning PRs for Review
  • Next tasks from Development Planning Spreadsheet

Attendees

• Art Lowel (Atmire) 

• Andrea Bollini (4Science)  

• Tim Donohue

• Heather Greer Klein

• Lieven Droogmans 

• Mark H. Wood
• Giuseppe Digilio (4Science) 
• Ben Bosman
• Chris Wilper
• Terrence W Brady
• Paulo Graça 
• Dimitris Pierrakos 
• Julius Gruber
• Laura Henze

Current Work

Tickets / PRs In Progress

1. (Angular) Adding Accessibility via Travis CI  https://github.com/DSpace/dspace-angular/pull/356 (work in progress) (Lower priority)
2.  (Angular Bug) https://github.com/DSpace/dspace-angular/issues/368 ( Art Lowel (Atmire) )
3. (REST Contract) Edit Homepage news: https://github.com/DSpace/Rest7Contract/pull/45 (Ben Bosman  - has outstanding questions/comments) (Lower priority)
4. (REST) DS-4043: Revisit the security layer of the submission  (work in progress) Andrea Bollini (4Science)
5. (REST) Pagination bug with withdrawn items: https://github.com/DSpace/DSpace/pull/2406 (Dimitris Pierrakos , Ben Bosman - Feedback provided)

PRs Needing Review

1. (NEW) (REST Contract) CRUD on Collection & Community logo https://github.com/DSpace/Rest7Contract/pull/79 (Kevin Van de Velde (Atmire), Tim Donohue , Dimitris Pierrakos ) 
2. (NEW) (REST Contract) CRUD on Collection Item template https://github.com/DSpace/Rest7Contract/pull/80 (Kevin Van de Velde (Atmire), Tim Donohue , Dimitris Pierrakos ) 
3. (UPDATED) (REST Contract) Group and eperson management: https://github.com/DSpace/Rest7Contract/pull/41 (Tim Donohue, Andrea Bollini (4Science))
4. (REST) Issue when community has multiple dc.title values https://github.com/DSpace/DSpace/pull/2486 UPDATED ( Tim Donohue - feedback added, Andrea Bollini (4Science))
5. (REST) Oai harvesting setup https://github.com/DSpace/DSpace/pull/2491 (Tim Donohue -  Minor improvements requested, Andrea Bollini (4Science), Paulo Graça )
6. (REST) Spring security for createAndReturn with parent id https://github.com/DSpace/DSpace/pull/2489 (Tim Donohue -  Minor improvements requested, Chris Wilper ) 
7. (NEW) (REST) Authority control bugfixes https://github.com/DSpace/DSpace/pull/2528 (Tim Donohue, Giuseppe Digilio (4Science), Andrea Bollini (4Science))
8. (REST) (Entities) CSV Import fixes: https://github.com/DSpace/DSpace/pull/2522 (Ben Bosman, Paulo Graça)
9. (Angular) Item-Collection Mapper:  https://github.com/DSpace/dspace-angular/pull/348 ( Tim Donohue -  REREVIEW, Art Lowel (Atmire) - provided feedback)
10. (Angular) Shibboleth integration support: https://github.com/DSpace/dspace-angular/pull/429  (Giuseppe Digilio (4Science) reviewed again fixed error with yarn start, Fernando FCT/FCCN, Paulo Graça)
11. (Angular) Submission Miscellaneous fixes: https://github.com/DSpace/dspace-angular/pull/432 (Art Lowel (Atmire), Tim Donohue -  Noted performance issues)
12. (Angular) forceBypassCache should be removed from the RequestService: https://github.com/DSpace/dspace-angular/pull/468 (Art Lowel (Atmire), Giuseppe Digilio (4Science))
13. (Angular) Collection pages WIP: https://github.com/DSpace/dspace-angular/pull/472 (Art Lowel (Atmire) - feedback provided, Tim Donohue)

PRs Merged this week!

1.  (REST Contract) Scripts & Processes endpoint: https://github.com/DSpace/Rest7Contract/pull/17
2.  (Angular) Redirecting user to same page after login https://github.com/DSpace/dspafixed error with yarn startce-angular/pull/467
3.  (REST) Endpoints to collect statistics https://github.com/DSpace/DSpace/pull/2502

Blocked

1. (Blocked PRs go here)

Delayed / Needs Discussion

1. (REST) Scripts & Processes endpoint: https://github.com/DSpace/Rest7Contract/pull/17
   1. How do we move this forward? In our Jan 24 meeting, Tim noted a risk of "scope creep" with this feature suggestion. This idea has been tabled since then.
   2. In July 25 meeting, Atmire said they'd come back with notes on the proposed backend implementation.
2. Managing Authorization info in Angular UI: How to pass Authorization rights (i.e. logged in user's access rights) from REST API to Angular?  See for example: https://github.com/DSpace/dspace-angular/issues/393
   1. Can this be achieved via passed HAL "_links" (e.g. the existence of an "edit" link in REST response means you must have Edit rights)?
   2. In July 25 meeting, we noted this probably cannot be resolved with just one simple solution. May need to look at different options for different scenarios
      1. Also likely to need to store/cache a user's Groups in UI layer, as some areas (e.g. Administrative) require knowledge of user group membership
3. REST API Projections:  Unable to locate Jira server for this macro. It may be due to Application Link configuration.
   1. Early work begun at https://github.com/DSpace/DSpace/pull/1847.  Discussed in more detail in our Aug 22 meeting.  Overall, this approach seems like a good direction, need volunteers to move it forward.
      1. https://github.com/DSpace/Rest7Contract/issues/2 (discussion resumed by Andrea Bollini (4Science) could be relevant for the projection)
4. Initial Performance Testing from Chris.
   1. https://cwilper.github.io/dspace-perftest/
5. (REST Contract) Edit Homepage News: https://github.com/DSpace/Rest7Contract/pull/45
   
   1. Delayed until after Preview release. General agreement (in meeting on March 21, 2019) that storing HTML in metadata fields is not really ideal behavior.  Metadata (from a librarian standpoint) tends to be free of format-related markup (as that allows for easier sharing, understanding of metadata.  Currently Community & Collection homepage information is HTML-based and is stored in metadata that is appropriate for a minor subset of information (like the title) but it is better to move large/rich text to bitstreams.  
   2. Proposal here is to consider storing HTML-based markup (for Site, Community & Collection homepages) in Bitstream(s) associated with the object in question.  May allow for more CMS-lite behavior in the future
   3. Timeline for this is uncertain.  Possibly in 7 or 8. May depend on how/whether it can be scoped.
6. Concurrency in DSpace 7 (or 8).  What do we want to do when multiple editors are editing the same object?  Needs further analysis regarding implementation details
   1. We've decided (in meeting on March 7, 2019) to use ETags to implement concurrency. REST Contract notes on ETags: https://github.com/DSpace/Rest7Contract#etags--conditional-headers
   2. ETags only update of the two fields match. If someone edits first, your edit would fail and you would get a fail response (422?)
   3. ETags seems to have broader support in other REST APIs.  Recommended also by both Art and Andrea.

7. Improve/Re-enable End To End (e2e) Testing. Could there be opportunities to use Travis CI + Docker Compose for testing of Angular?? https://github.com/DSpace/dspace-angular/issues/453#issuecomment-519672141

   • Docs: https://docs.travis-ci.com/user/docker/#using-docker-compose
   • Blogpost on how to do it: http://elliot.land/post/using-docker-compose-on-travis-ci
   • An example travis.yml file: https://github.com/Ortus-Solutions/docker-buildfiles/blob/master/.travis.example.yml

Notes

• North American User Group was a success (~50 attendees).  DSpace 7 talks/workshop were all recorded & video will be posted in coming week(s).  Some slides already available at 2019 DSpace North American User Group Meeting
• Discussion: Changing the collection in the submission doesn't change the item template 
  • Summary: While it is a nice feature, there are a few known issues with how changing collection in submission currently works.
  • First, Item Templates are only applied when a submission is created.  Currently, there is also no way to kick off a submission in a specific Collection... instead all submissions currently start from the configured "default" Collection.  This means that Item Templates are not currently working properly....only the "default" Collection template would ever be applied (if any).
    • This may be (partially) solvable by allowing a way to select a Collection to start the submission in before the submission is created.  However, users must be informed that if they decide to switch Collections later on, then the new Collection's Item Template (if any) will not be applied 
    • Perhaps for DSpace 8 it'd be worth thinking about whether there's a way to switch Item Templates?
  • Second, if you change Collections after already entering metadata, it's possible some of your metadata entries will be removed (if they don't exist in the Submission form configuration for the Collection you are switching to).  So, for example, if you enter in a "dc.contributor.editor" of "Smith, Jane", and switch to a Collection that doesn't include that field in the forms, then that field entry is deleted without notifying the submitter
    • This may be (partially) solvable by warning the submitter before they change Collections that entered metadata may be lost.  Ideally, it'd be useful to tell the submitter which fields will be lost (if that's possible with minimal effort) as there also may be many scenarios where no fields will be lost (so the warning is unnecessary).
    • We also could pre-warn users by adding a text note to the Collection dropdown – note something like "switching the collection may result in changes to the submission form.  Therefore, we recommend switching the Collection prior to filling out the form (as much as possible)"
    • Perhaps for DSpace 8 it'd be worth thinking about whether there's a way to "map" or "move" entered metadata to a different field (where necessary)?
  • We need more information to determine which route to take to solve these problems.  It's currently unclear if these suggested (partial) solutions are easily doable, or if they add a ton of additional complexity.  Another option under consideration should be whether to temporarily remove  this new change Collection feature, and reschedule it for DSpace 8 (once we've had the opportunity to discuss these scenarios and provide more user-friendly behavior).
• Discussion: Code reviews are taking a long time over the last month
  • Should we change code review practices.  Have one person do a review, and they can suggest a second reviewer if they feel it needs one?  Or set a deadline for a second review (if it's 2 weeks old and has one +1, then merge it)
  • Ideally, obviously we want two sets of eyes on everything, as it results in better code overall.  But, finding two code reviewers for everything has proven to be a challenge recently – we need more reviewers/testers!
  • Tim will bring this internally to LYRASIS to discuss more.