Andrea Bollini [10:32 AM]
hi, I can start with some update on the rest-api work [10:33]
@terrywbrady has made a PR to add new resources to the rest-api webapp see https://github.com/DSpace/DSpace/pull/1661
[10:35]
I have take a first look to it and it seems great, so I'm in favor to approve it and improve some aspects later
[10:36]
before to merge I want to give it a try. I will do this in the next couple of hours Terry Brady [10:36 AM]
Good to hear. I have a few questions about how to handle DSpace objects reference by another DSpace object Andrea Bollini [10:36 AM]
yes! this pr open a few questions that we want to discuss Terry Brady [10:36 AM]
We could discuss that here or in the PR comments Andrea Bollini [10:37 AM]
I think that it will be useful to discuss a bit now, and copy our conclusion on the PR as well Terry Brady [10:37 AM]
I see that an object can have (1)attributes (2)embedded objects (3)links to external objects Andrea Bollini [10:38 AM]
yes, the rest model is a pojo Terry Brady [10:38 AM]
What is the proper way to designate a field/property into one of these locations? Andrea Bollini [10:38 AM]
it is named with the Rest postfix (ItemRest, BitstreamRest, etc.) [10:39]
other than the rest model classes we have the hal resources [10:39]
they are named with a Resource Postfix [10:40]
the resource class wrap the rest class adding the HAL functionalities (links, embedding) Terry Brady [10:40 AM]
That helps. How do I designate that something is linked vs embedded? Andrea Bollini [10:41 AM]
every thinks that is embedded is also linked, the opposite is not true [10:42]
my idea is to use reflection to automatically discover from the model (rest) class the linked entities and embed the information if
1) available
2) requested by the client [10:44]
additional links can be added extending the DSpaceResource class (i.e. ItemResource) to "manually" add the links in the constructor (after call the super) Terry Brady [10:44 AM]
Thanks. It will be good to see some more examples Andrea Bollini [10:45 AM]
both spring data rest than katharsis use reflection and annotations to discover relations between rest resources Terry Brady [10:45 AM]
I added owningCollection to RestItem but it did not quite behave as I expected Luiz Claudio Santos [10:45 AM]
Andrea Bollini: Do the Rest classes (BitStreamRest, ItemRest) proxies to the DSpace classes? Andrea Bollini [10:45 AM]
@luizsan they are a different representation, in some way a DTO layer [10:46]
we have the opportunity to build a rest model more close to the domain model Terry Brady [10:46 AM]
I think we will want to create a document explaining how to code up each of these scenarios (and to explain the data instantiation implications of each option). Andrea Bollini [10:47 AM]
unfortunately we don't have an answer right now. As we need to build our infrastructure to do some of this job. But, as said we can follow what as been done in other successful projects [10:49]
for instance you already have get the automatic conversion of an attribute of the itemrest class (the collection) in a link with embedding Luiz Claudio Santos [10:49 AM]
@bollini do you already any project in mind? Andrea Bollini [10:50 AM]
@luizsan yes, we have agree to mimic as much as possible the architecture of spring data rest [10:50]
http://projects.spring.io/spring-data-rest/ [10:51]
please note that spring data rest is NOT what we are using, as we stay on
spring mvc + spring hateoas Luiz Claudio Santos [10:51 AM]
@bollini ok, I got it, thanks Tim Donohue [10:51 AM]
Perhaps this would be worth still capturing as "best practices" or "brainstorms" on a Wiki page (or something)? It might help others (still coming on board or in the future) to get up to speed on these ideas
Terry Brady [10:52 AM]
I will start a page
Tim Donohue [10:52 AM]
i.e. more docs (even if they are rough/brainstorms) is a good thing
Terry Brady [10:52 AM]
to capture the types of questions I need answered.
[10:53]
Much of this stack is new to me, so I just want to learn how to make the data flow out of the system regardless of what the underlying technology is
Andrea Bollini [10:54 AM]
take in mind that something can be improved later. For instance we should have a central way to manage lazy-loading Luiz Claudio Santos [10:54 AM]
@terrywbrady Could you share the page in this slack channel later? Terry Brady [10:54 AM]
@luizsan , I will once I get a template going
Tim Donohue [10:55 AM]
recommends adding the REST API notes/brainstorms as a subpage of https://wiki.duraspace.org/display/DSPACE/DSpace+7+UI+Working+Group (and also sharing on this channel, of course)
Andrea Bollini [10:55 AM]
in the rest class, you have one resource linked to another. Our converter and/or the resource wrapper will be (at some point) able to check if the linked resource is lazy-loaded Terry Brady [10:55 AM]
Whatever we do, there is also some hibernate magic happening in the background and we will want to make sure that behavior does waht we need Andrea Bollini [10:56 AM]
agree Terry Brady [10:56 AM]
On a side note, I discovered the following bug https://jira.duraspace.org/browse/DS-3515 [10:56]
It makes testing slow since the pagination parameters are not actually used in the HAL browser Andrea Bollini [10:57 AM]
I will reply to that later [10:57]
you need to build a proper pagination in the response, I will provide you an example Terry Brady [10:57 AM]
thanks Andrea Bollini [10:58 AM]
unfortunately I have issue to access jira today... still trying to figure why Tim Donohue [10:58 AM]
Sounds like a lot of opportunities for documenting hints/tips here on getting started, best practices, etc :wink: Good discussion Andrea Bollini [10:58 AM]
the other thing that I want to highlight about this week progress is [10:58]
https://github.com/DSpace-Labs/Rest7Contract/issues/3 [10:59]
it is about the metadata representation as discussed in the past meeting Luiz Claudio Santos [10:59 AM]
one question, the DSpace's master branch is the one that you guys are using to develop new thing in the REST-API? Andrea Bollini [11:00 AM]
right now we are using a feature branch (rest7) that is build on the master Tim Donohue [11:00 AM]
https://github.com/DSpace/DSpace/tree/rest7 Andrea Bollini [11:00 AM]
this is because our feature branch still fails test due to issue in the dependencies management Art Lowel [11:00 AM]
@bollini `nullQualifier` means no element? Andrea Bollini [11:01 AM]
mean no qualifier [11:01]
dc.title for instance Terry Brady [11:01 AM]
@bollini , I like that representation if I want to retrieve a single field Art Lowel [11:01 AM]
right :facepalm: (edited) Tim Donohue [11:01 AM]
(Once that `rest7` branch stabilizes, it will be merged into "master" and from that point forward development will be on "master". However, as noted by @bollini, there are still known failing tests, etc) Luiz Claudio Santos [11:01 AM]
thanks Art Lowel [11:01 AM]
yeah, I like it too Terry Brady [11:01 AM]
I am less if that is an optimal structure for processing all of the metadata for an item [11:03]
One common need that I have with the current rest api is that I want to retrieve 200 items and I want specific metadata fields from each of those items. In that use case, the metadata=dc.title,dc.contributor.* is attractive [11:04]
I should be able to accomplish that with either metadata structure... Andrea Bollini [11:04 AM]
yes, we can always support such kind of parameter to filter our response [11:05]
but your use case is more oriented to the discovery layer than the "persistence" [11:06]
we don't need to have the same structure for both as we can optimize each structure for their specific use cases Terry Brady [11:06 AM]
That makes sense Tim Donohue [11:07 AM]
notes we are past the 1/2 hour point. Discussion can still continue though in either #rest-api or #angular-ui as needed Terry Brady [11:07 AM]
I reserved the following wiki page for the documentation we discussed above: https://wiki.duraspace.org/display/DSPACE/DSpace+7+REST%3A+Coding+DSpace+Objects [11:07]
I will add more to it and then alert this channel Andrea Bollini [11:09 AM]
@terrywbrady here you can see how a large list of bitstreamformat is limited to a specific page https://github.com/DSpace/DSpace/blob/rest7/dspace-spring-rest/src/main/java/org/dspace/app/rest/repository/BitstreamFormatRestRepository.java#L63
Tim Donohue [11:09 AM]
Quick note...thanks everyone for joining the discussions today. The next DSpace 7 Mtg will be next Thurs (Mar 9) at the same time (16UTC) in Google Hangouts. Reminders will go to the dspace-devel list next week (and to Slack)
Andrea Bollini [11:11 AM]
this is more or less the "same workaround" that we apply in the current rest implementation: retrieve all the object from the persistence layer and filter our the single page that we need. It is a temporary workaround, the proper way will be to introduce pagination in the persistence layer Terry Brady [11:11 AM]
I will be out of the office until 3/13, so I will try to leave my contributions in a good spot at the end of today. Andrea Bollini [11:12 AM]
ok thanks. Another thing about your PR. We want probably discuss about community and collection [11:13]
I will like to try to hide this difference in the REST layer but I will prefer to move ahead with your current implementation and revise that later Terry Brady [11:14 AM]
That makes sense to me. I imagine it will be easier to approve some imperfect stuff along the way to get things moving Andrea Bollini [11:14 AM]
I will open an issue in the REST contract about that, but we need to move on until we take the decision [11:15]
yes, absolutely. Now we need to prefer move fast over perfectionism [11:19]
I will be around for still 1hour, feel free to ask more questions if you want Terry Brady [11:54 AM]
@bollini , I have started some edits to https://wiki.duraspace.org/display/DSPACE/DSpace+7+REST%3A+Coding+DSpace+Objects Andrea Bollini [12:05 PM]
looks good. I have just added some additional details in the HATEOAS section Terry Brady [12:06 PM]
thanks. I am still editing and I will work those notes into the doc. [12:06]
I will also link to the doc from https://github.com/DSpace/DSpace/blob/rest7/dspace-spring-rest/README.md [12:07]
Will I need to create a PR for that, or can I edit the .md file through github? Andrea Bollini [12:12 PM]
feel free to edit [12:12]
at the current stage we can just improve it :wink: Terry Brady [12:13 PM]
Done. Link added Terry Brady [12:35 PM]
@bollini , I am still trying to understand something. What do I need to do in the code to trigger these 2 conditions? https://github.com/DSpace/DSpace/blob/rest7/dspace-spring-rest/src/main/java/org/dspace/app/rest/model/hateoas/DSpaceResource.java#L52-L61 Andrea Bollini [12:36 PM]
nothing, any attribute that implement the RestModel interface will be automatically recognized up as a linked resource Terry Brady [12:38 PM]
Thanks. So this code identifies the embedded RestModels and makes links. [12:39]
If I want to create a link without embedding an object, I need to do that elsewhere. (I know you answered this earlier, but I am trying to capture this in easy to follow instructions) Andrea Bollini [12:42 PM]
@terrywbrady https://github.com/Georgetown-University-Libraries/DSpace/pull/9 this is all you need to fix your PR [12:42]
I need to go now, I will check the chat later Terry Brady [12:43 PM]
thanks and thanks for the PR! Terry Brady [1:45 PM]
The org.dspace.content DAO methods for item and bitstream do not support pagination. I am adding those methods to allow pagination of these resources as well. Tom Desair (Atmire) [1:52 PM]
For items, aren’t we going to use the pagination provided by Discovery? That one already takes into account the resource policies defined on an item. Terry Brady [1:52 PM]
Probably, but we have no solr interaction right now [1:57]
I can package this up in a separate PR in case there is controversy. There appear to be slight variations in the method signatures for the findAll() methods for the DSpaceObject DAO classes. I am finding it difficult to test without the ability to paginate on item or bitstream. Every request is returning the entire repository. Tom Desair (Atmire) [2:13 PM]
The difficult part here is taking into account the resource policies. We could create an “admin section” of the REST API which would allow direct pagination via the database (DAOs) and disregards the resource policies. But that those endpoints should go below a separate “admin” endpoint I think, e.g. `/admin/items`, `/admin/bitstreams`... (edited) [2:18]
For the “regular” `/items` endpoint, I would use SOLR, that shouldn’t be to difficult and would also allow filter parameters (similar to this https://github.com/milieuinfo/dspace54-atmire/blob/ontwikkel/dspace/modules/rest/src/main/java/org/dspace/rest/SearchResource.java#L94) (edited) [2:19]
But I don’t have an idea yet on how to efficiently paginate the `/bitstreams` endpoint taking resource policies into account and without doing the authorisation check in-memory. (edited) Terry Brady [3:21 PM]
I created https://github.com/DSpace/DSpace/pull/1666 which offers pagination for items and bitstreams. [3:22]
Note that https://wiki.duraspace.org/display/DSPACE/DSpace+7+REST%3A+Coding+DSpace+Objects has been updated as well. |