Page History

Tim Donohue [9:00 AM]
Reminder that in one hour (15UTC) is our weekly DevMtg in #dev-mtg .  I have a meeting just before that (starting now), but will see you in an hour.  Agenda at: https://wiki.duraspace.org/display/DSPACE/DevMtg+2018-09-05

Tim Donohue [10:00 AM]
@here: It's DevMtg time. The agenda is listed above.  Let's do a quick roll call to see who is able to join discussion today.

Terry Brady [10:00 AM]
hello

Alexander Sulfrian [10:00 AM]
hi

Mark Wood [10:01 AM]
Hi.

Tim Donohue [10:01 AM]
Hello Terry, Alexander and Mark. Looks like we have a small quorum of folks, welcome
So, to start off the agenda with some notes from other meetings.  I don't have much to report out of DSpace 7 Dev meetings (work is progressing, but Aug was slow. Sept looks like it should start to ramp up)
But, we had our first DSpace 7 Entities WG meeting yesterday (several of you were there)
The notes/video from that meeting is at: https://wiki.duraspace.org/display/DSPACE/2018-09-04+DSpace+7+Entities+WG+Meeting

Pascal Becker [10:03 AM]
Hi.
has to excuse himself for at least parts of the meeting. will be here one eyed for the other parts.

Tim Donohue [10:04 AM]
And as a reminder, even if you were unable to attend the DSpace 7 Entities WG meeting, you are still welcome to provide feedback on the "early prototype" of configurable entities
If you wanted to give feedback, we ask that you review the "early prototype" documentation and add inline comments/questions/concerns: https://docs.google.com/document/d/1wEmHirFzrY3qgGtRr2YBQwGOvH1IuTVGmxDIdnqvwxM/edit
Those comments/questions/concerns will help drive conversation in the coming meetings...next meeting we'll concentrate more on the proposed Data Model changes
(The next DSpace 7 Entities WG meeting will be next Tues, Sept 11 at 15UTC)
I think that's it for quick updates/notes from other meetings...any questions/comments on any of that (or anything I forgot?)
Ok, not hearing anything
Moving along, DSpace 6.x -- no updates to speak of.
However, DSpace 5.x has a topic (#3 on the agenda).... whehter we should consider a 5.10 release in the near future
I believe the main call for this is because the REST API was accidentally broken in 5.9.  It was a small fix, but an important one to anyone using it in 5.x
Here's the fix/problem: https://github.com/DSpace/DSpace/pull/2197

Terry Brady [10:08 AM]
REST was definitely broken. RDF might also have been broken.

Tim Donohue [10:09 AM]
Is RDF fixed yet?  I hadn't seen any reports of RDF being broken in 5.9, yet

Terry Brady [10:09 AM]
If we do decide to expedite a release, it would be good to find folks who could agree to validate each of the webapps.
Let me find the other PR...

Tim Donohue [10:10 AM]
So, I'll say I'd approve pushing out a quick 5.10 -- I agree though that we want to ensure we aren't immediately turning around and pushing out a 5.11 for other small issues.  So, finding a list of the small issues to fix would be good

Terry Brady [10:10 AM]
https://github.com/DSpace/DSpace/pull/2104

Tim Donohue [10:10 AM]
That said, I won't have time myself to push out 5.10 -- I'll help support it, but I'd need a different release manager/coordinator

Terry Brady [10:10 AM]
I think the wrong postgres jar made it into RDF.
I'll be glad to take that on.
I would just want to find someone familiar with Sword and with RDF to help validate those webapps.

Tim Donohue [10:11 AM]
PR#2104 affected RDF?  Oh, I didn't realize that.  I thought it was just a PostgreSQL library issue if you wanted to run DSpace 5.x + Java 7

Terry Brady [10:11 AM]
I should be able to verify the others.

Pascal Becker [10:11 AM]
@terrywbrady We should setup a Docker image with a Triple Store to make it easy to test RDF. Would you like to work with me on that?

Terry Brady [10:12 AM]
Sure @pbecker

Pascal Becker [10:12 AM]
Great.
BTW I think dspace-rdf does not need Postrgres at all. But it’s bin some time I developed that.

Terry Brady [10:13 AM]
If you ran 5x with Java7, any webapp that accessed that postgres jar would not start.  If RDF does not use postgres, I do not know if it would encounter the error.
It sounds like @pbecker and I should be able to validate the issue.
@pbecker are you familiar with Sword?

Mark Wood [10:14 AM]
If dspace-rdf uses a Context then it is using the DBMS.

Pascal Becker [10:14 AM]
Not much. I would appreciate if someone else could take care for sword.

Tim Donohue [10:14 AM]
Ok, I was just trying to understand if PR #2104 *fixed* the issue.  It sounds to me like it should be fixed, but needs further testing.

Pascal Becker [10:15 AM]
@mwood I’m not sure that it uses a context at all. It is basically a proxy in front of the triple store. The data is converted into RDF as part of dspace-api.

Tim Donohue [10:15 AM]
And yes, it makes sense that Java 7 would cause all webapps to fail -- as, prior to PR #2104 we were accidentally using a Java 8 Connector to Postgres -- and Java 7 didn't work with it at all.
So, @terrywbrady, if you are willing to act as 5.10 Release Coordinator, that'd be wonderful. It sounds like the latest `dspace-5_x` branch may already have all the fixes we require (PR#2104 and PR#2197)
So, it might just be a matter of testing that branch & cutting a release

Terry Brady [10:17 AM]
Considering that we had 2 different POM dependency issues, I am curious to to a minimal check of every webapp esp the less frequently used ones.  Any tips on testing Sword?
and Swordv2?

Tim Donohue [10:17 AM]
We have sample commandline commands in both the SWORD and SWORDv2 docs that can be used for simple tests...lemme find them

Terry Brady [10:18 AM]
That would be great to have.  Thanks.

Tim Donohue [10:18 AM]
SWORDv1 submit: https://wiki.duraspace.org/display/DSDOC6x/SWORDv1+Server#SWORDv1Server-DeposittoSWORDServer

Terry Brady [10:18 AM]
Keep this on the agenda.  I will hope to find time over the next couple weeks.

Tim Donohue [10:18 AM]
SWORDv2 submit:https://wiki.duraspace.org/display/DSDOC6x/SWORDv2+Server#SWORDv2Server-DeposittoSWORDv2Server
I linked to the 6.x docs there, but the commands are identical for 5.x

Mark Wood [10:19 AM]
LocalURIRedirectionServlet does indeed create a Context, which it needs to resolve a Handle.

Pablo Prieto [10:19 AM]
Hi all. Excuse me for the late arrival.

Tim Donohue [10:20 AM]
There's a sample SWORD package (example.zip) that can be used for submission tests at: https://github.com/DSpace/DSpace/tree/dspace-5_x/dspace-sword/example

Terry Brady [10:20 AM]
Hi @Pablo Prieto
Thanks.  As a result of this we should also be able to generate some documentation on running the RDF service with Docker.

Tim Donohue [10:22 AM]
That'd be great!  Thanks again, @terrywbrady for being willing to coordinate this.
One final thing...I think we should create a small 5.10 Release Status page to track this work...something like https://wiki.duraspace.org/display/DSPACE/DSpace+Release+5.9+Status

Terry Brady [10:23 AM]
I will create one.

Tim Donohue [10:23 AM]
And I'll keep this on the agenda in coming weeks so we can check in on progress
Any last comments/questions here?
(One obvious comment -- if anyone else listening in finds additional small 5.x bugs, report them immediately please...and we can consider for 5.10)

Pascal Becker [10:24 AM]
@mwood thank you for taking a look into dspace-rdf.
Do we have the same problem with DSpace 4?

Tim Donohue [10:26 AM]
4.x did not receive the same backports at 5.x.  So, I'm doubtful it was affected -- but it's worth checking if any of these changes made it into 4.x
Specifically, there were only two security fixes (both to JSPUI) that were released in 4.9: https://wiki.duraspace.org/display/DSPACE/DSpace+Release+4.9+Status
No other bugs were backported

Pascal Becker [10:28 AM]
:+1:

Tim Donohue [10:28 AM]
So, that should mean 4.x is fine.  Neither of these security fixes seems to be the cause of these issues
Ok, sounds like that topic is wrapped up then.  We'll move along

Terry Brady [10:30 AM]
https://wiki.duraspace.org/display/DSPACE/DSpace+Release+5.10+Status

Tim Donohue [10:30 AM]
I left this on the agenda at #4.... I know we're still looking for more ideas for future Dev Show & Tell meetings: https://wiki.duraspace.org/display/DSPACE/DSpace+Developer+Show+and+Tell+Meetings
I'm not sure we have a topic for September.  So, the question is...what would folks like to talk about?  Should we have a September meeting?  Is there some other Dev discussion this could be used for?

Pascal Becker [10:33 AM]
Is it already the right time to dig deeper into the DSpace 7 architecture?
I know that this is still moving, but I think it would be a chance to help new people onboard.
I’m thinking on some more techincal aspects.
Example given: how will we encapsualte our authority layer into spring security, ….

Tim Donohue [10:35 AM]
I'm not against the idea, but some of this is definitely a moving target.  The onboarding workshops from OR2018 are still the most "up to date"

Pascal Becker [10:35 AM]
And another idea: Would it be worth to have a Dev Show and Tell to discuss if we could and should use Spring even more than we already do?

Terry Brady [10:36 AM]
@pbecker, that sounds like an architectural direction question... I recommend that for a DSpace 7 meeting.

Pascal Becker [10:36 AM]
@tdonohue you can judge much better that me if it would be helpful and it could help with the next sprint.
@terrywbrady I’m not so much into spring. I don’t know how others are. So the idea was to use this as a starting point. I was just a more spontanious idea that came to my mind. Maybe you’re right and other topics are better for the Dev show and tell.

Terry Brady [10:37 AM]
I love the idea of digging deeper into things like Spring or the authorization layer. If you could propose a topic and a potential speaker, I would be happy to invite folks and coordinate the invite.

Tim Donohue [10:38 AM]
I think general developer Q&A / brainstorms are reasonable. I think the problem is in finding a "speaker" (if we want this to be "show & tell")... Or we consider just turning this one into more of a brainstorming meeting?

Mark Wood [10:38 AM]
There's an old Jira issue about recognizing that Spring is our one and only actual ServiceManager, cleaning out some complex un-needed code.

Tim Donohue [10:39 AM]
Running another DSpace 7 "workshop"/ training session might be difficult right now, as it takes away from our DSpace 7 dev resources (which we sorely need to keep "on schedule").
I would love to have such a DSpace 7 training session in the future, I just worry this is not the best time...and the training would be nearly identical to what was done at OR2018

Pascal Becker [10:40 AM]
@tdonohue who is working on the spring-security for dspace 7? Couldn’t that be a speaker for this specific topic?

Tim Donohue [10:40 AM]
Tom Desair (who recently got a new job) was the lead on spring-security for DSpace 7

Pascal Becker [10:40 AM]
@tdonohue it was just an idea. If you thing it is not the write time now, than postpone it.
@tdonohue @tom_desair wants to stay active (of course with reduced time on DSpace). Maybe it is worth asking him? Could be helpful to transfer the knowledge.

Tim Donohue [10:41 AM]
The spring-security stuff could be talked about more...but admittedly, we need more folks to dive in here & chip in... Tom got the ball rolling, but it's not entirely "done" yet.

Terry Brady [10:41 AM]
If one of us wants to lead the discussion from the perspective "here are my questions about Spring" that could also be a good place to start a conversation.

Mark Wood [10:42 AM]
Hardy Pottinger was interested in consolidating authNZ in something like Spring Security.  I don't know how much time he has for that right now.

Tim Donohue [10:43 AM]
If someone wants to ask @tom_desair, I'm OK with it. Not sure whether he'll have free time for it yet...but it's worth asking.
It sounds like we don't have a firm idea here. I know I'm not going to be able to answer all the questions on Spring that folks have, but I'm willing to help with a Developer Q&A session of sorts (if folks come with questions)

Terry Brady [10:46 AM]
I will update the Show and Tell Wiki Page with that as a possible future topic.

Tim Donohue [10:47 AM]
I think the reality on DSpace 7 specific topics here is that we are trying to present on a moving target (and I worry anything we present on could be outdated before end of year)
So, discussion / Q&A on DSpace 7 is perfectly fine to me.  But, formal training/presentations are maybe not easy right now

Mark Wood [10:48 AM]
We may hope that the target will slow down toward the end of the year, so maybe later would be an appropriate time for presenting on it as the code settles down.

Terry Brady [10:48 AM]
We can put the meetings on hold until we have a compelling topic.

Tim Donohue [10:48 AM]
But, now that I type that...maybe we could hold a session on "Ask us about DSpace 7!" ? Come get your questions answered on the future of DSpace (edited)

Pascal Becker [10:49 AM]
When I started with DSpace, I had problems to understand the structure of the DSpace API. Would it make sense to offer some introduction to the DSpace API? I was surprised last time how many people that I didn’t knew before joined the meeting about Docker. What does other think here could that be another topic? Some kind of introduction for new developers….

Terry Brady [10:51 AM]
I like the idea of a general Q&A for new developers/contributors.
I could post a question to the email lists and slack channels to compile questions.

Tim Donohue [10:52 AM]
I'd sign up for any sort of Q&A.  I'm happy to answer any questions that developers have. I wouldn't come with any formal "presentation" or "show & tell", but I'll gladly hang out for 60-90mins answering questions (or pointing people to resources to learn more)

Mark Wood [10:52 AM]
Topic for next year, after DSpace 7 releases:  when I look at the UI, how to figure out where a feature comes from and where to build a new feature to have it appear *there*.
That is the thing that consumes 90% of the time I spend on XMLUI.

Tim Donohue [10:53 AM]
@mwood: Yes, I think that's the goal for early next year & at OR2019.  To actually have training on customizing the new DSpace UI.  We don't have that yet, but will

Terry Brady [10:54 AM]
I am capturing ideas here: https://wiki.duraspace.org/display/DSPACE/DSpace+Developer+Show+and+Tell+Meetings#DSpaceDeveloperShowandTellMeetings-WishListofTopics

Tim Donohue [10:55 AM]
Ok, so lots of brainstorms here. It doesn't sound to me like a formal decision.  But, I'm willing to be involved with a Q&A for developers or a Q&A on DSpace 7 (or both) at some point in the future.

Pascal Becker [10:55 AM]
I think we could also bring up something like: the states of an item, how do I bring a new Item into DSpace, what is it with the DAOs, the Services and the Impls.
I have to run now, but we should continue on this.
bye everyone!

Pablo Prieto [10:55 AM]
Bye

Terry Brady [10:55 AM]
Have a good week @pbecker.  I will be in touch about RDF.
Bye @Pablo Prieto (edited)

Tim Donohue [10:56 AM]
We are nearing the end of the meeting time...I don't know that we have many more topics though
I noted under topic #5 that I'm still looking at crediting folks for code reviews, etc (via GitHub GraphQL)

Pablo Prieto [10:56 AM]
I have a small topic. I don't know if this is the place.

Tim Donohue [10:56 AM]
You can look at my notes there for more details though
@Pablo Prieto: sure, go ahead with your small topic

Pablo Prieto [10:57 AM]
I don't know If this is implemented. But Is there a way to do the community-subcommunity filliation other than the cli?

Alexander Sulfrian [10:58 AM]
I don't think that's currently possible.

Tim Donohue [10:58 AM]
You mean moving/restructuring Community/Subcommunity hierarchy?  No, that's not in the UIs in DSpace 6

Pablo Prieto [11:00 AM]
This came up because as some more security-oriented entities use DSpace, their IT deparments restrict access to the server shell, so Dspace admins are left to either request support tickets or just create another community.
I don't know your considerations on this, but I think we should figure out a way to do that through the UI

Tim Donohue [11:00 AM]
It's not yet in the DSpace 7 REST API either...but, it likely could be added at that layer (which would allow folks to create a UI screen for it in DSpace 7)

Terry Brady [11:01 AM]
@Pablo Prieto we have built a companion set of tools for our DSpace (written in PHP) that provide web page access to various CLI functions.

Pablo Prieto [11:01 AM]
But that would mean adding php capabilites to the server?
IT guys will go crazy on that.

Terry Brady [11:01 AM]
I would not recommend implementing in PHP... I am just saying that we had the same need.

Pablo Prieto [11:01 AM]
Maybe we could add this to the existing REST-API

Tim Donohue [11:02 AM]
To me, it seems like this should be possible from the UI.  But, we've just never had anyone add it in.  It seems like it shouldn't be too bad to add to the DSpace 7 REST API .. since the underlying Java API already exists

Pablo Prieto [11:02 AM]
I you can be patient enough, and this seems usable for other,s, I could take a look into it.
So that would be for DS7+

Tim Donohue [11:04 AM]
Yes, I'd propose new features like this be aimed more for DSpace 7.  We tend to encourage new features only for upcoming (unreleased) versions... so, 6.x won't get any more new features -- just bug fixes, security fixes & minor improvements

Terry Brady [11:04 AM]
If you are curious, here is an overview of what we have implemented.  https://github.com/Georgetown-University-Libraries/batch-tools

All of this functionality would be nice to migrate to REST.  Even better if it could migrate to a curation task that could be invoked from either REST or CLI.

Tim Donohue [11:05 AM]
(And for the 7.x release, it's very important to not add more major features to 6.x, as that makes 7.x a moving target -- anything we add to 6.x must also be added to 7.x)
I'm realizing we are over time here (5 past the hour)
So, I'm going to go ahead and wrap up today's meeting.  Discussion is welcome to continue, but we won't call any more agenda items.

Mark Wood [11:06 AM]
However, if the feature is needed soon, 6.x might make a good prototyping platform.  I don't think that porting to 7 would be terribly difficult.

Pablo Prieto [11:06 AM]
OK, I'll look into that. I also have some other ideas. I'll wrap them in a wiki page and report back.

Tim Donohue [11:06 AM]
Yes, it's perfectly fine to develop against 6.x locally & use it locally. We just won't merge it into the public 6.x branch

Terry Brady [11:07 AM]
The REST interfaces are so different between 6 and 7 so a solution would not port to the new API.

Mark Wood [11:07 AM]
The concepts will be similar, though.

Tim Donohue [11:08 AM]
I've unfortunately got to run...so, the meeting is "complete" for today.  But, as noted, feel free to continue discussion (and I'll copy it into the notes).  Thanks all!

Mark Wood [11:09 AM]
And there is a lot to work out, concerning how the feature will be presented to the user.  That will have to be translated to the way things are done in the Angular UI, but again you don't lose *everything*.
It's a question of whether having the feature without waiting for 7 is worth the effort of porting later.

Pablo Prieto [11:10 AM]
Both rest APIs call the underlying dspace-api, so the approach might be similar.
Whats the ETA for DSpace 7?

Terry Brady [11:12 AM]
As long as the migration risk is understood, it is never a bad idea to experiment.  I hope you all have a good week.

Mark Wood [11:13 AM]
I think that the current estimate is "early 2019."

Pablo Prieto [11:13 AM]
Thank you  @terrywbrady. The same for you!
Oh, thats not too far.

Tim Donohue [11:14 AM]
Yes, goals for DSpace 7 are beta by end of 2018, and final release by early 2019.  Those are still estimates, but Steering Group has made it clear that they want us to get 7.x out as soon as possible (so we are trying to hit those goals)

Pablo Prieto [11:14 AM]
Ok
Thanks!

Terry Brady [11:16 AM]
What is the LNI service?  I am creating a grid for the webapps in DSpace 5.9 and I had forgotten that exists.

Tim Donohue [11:17 AM]
https://wiki.duraspace.org/display/DSPACE/LightweightNetworkInterface (Deprecated in 5.x, removed in 6.x)

Mark Wood [11:17 AM]
It's a remote submission service, like SWORD except only DSpace has it.  It has very few users, if any are still left.

Tim Donohue [11:17 AM]
It was never known to be used by *anyone*.  We contacted the community about deprecation/removal and heard nothing

Mark Wood [11:18 AM]
Wasn't there one site in the UK that was using it?  I think they planned to move off it, though.

Tim Donohue [11:20 AM]
Edinburgh, possibly...they used it at one time a long time ago.  It was built way before SWORD, and I'm not convinced it was ever really used other than by Edinburgh & Harvard (who built it).
In any case, if it "builds", I wouldn't worry about LNI until someone complains
(And I've never seen a complaint about LNI since I've been at DuraSpace...which is nearing 10 years)

Terry Brady [11:21 AM]
For 5.10, I am going to start by filling in the following grid: https://wiki.duraspace.org/display/DSPACE/DSpace+Release+5.10+Status#DSpaceRelease5.10Status-WebappStatus

Mark Wood [11:22 AM]
That grid looks like a good starting point.

Tim Donohue [11:24 AM]
@terrywbrady: seems like a good starting point.  Just a minor clarification regarding DS-3938... Pretty sure 5.9 works fine (except REST API) if you are using Java 8.   But, if you use Java 7, then issues will arise. So, which version of Java you are using with 5.9 may be a factor here

Terry Brady [11:24 AM]
Thanks.  That will be useful to note.

Tim Donohue [11:25 AM]
Since 5.x documents that it works with Java 7, we should test it with Java 7 to be sure.  But, Java 7 is EOL (for some time), so ideally, most folks should be upgraded to Java 8 these days.