Page tree
Skip to end of metadata
Go to start of metadata

Date

Angular meeting

Attendees

Notes

Slack Logs (times are all CST)

Tim Donohue [10:01 AM]
All, it's DSpace 7 Meeting time. The #angular-ui meeting is first (for 30mins). Unfortunately, I'm going to have to hand these meetings directly to @art-lowel (followed by @bollini), as I've got a sick child at home today. I'll be around, but mostly lurking (following when I can)

Art Lowel [10:01 AM]
Ok

[10:01]
Then I guess I’ll start with some status updates

[10:02]
@wwelling recommended putting the store in UniversalCache to share the state between server and client, and ensure that the same requests to the rest api don't need to happen again on the client in this issue

[10:02]
https://github.com/DSpace/dspace-angular/issues/69

[10:02]
succeeded in doing so, but it wasn't straightforward:

[10:02]
you should be able initialize the store with a snapshot of a previous store, and that works in dev mode, but I couldn't get it to work with the AoT build.

[10:02]
To fix that issue I added a "rehydrate" action to the root-reducer, that will replace the entire store with the object in the action's payload.

[10:02]
https://github.com/artlowel/dspace-angular/blob/7745938027e1e58b72d38a5dddb850d9b059aa5b/src/app/app.reducers.ts#L23

[10:03]
When the MainModule is initialized in the browser, that action is dispatched, with the cached store as its payload

[10:03]
https://github.com/artlowel/dspace-angular/blob/7745938027e1e58b72d38a5dddb850d9b059aa5b/src/browser.module.ts#L93

[10:03]
(links are provided only if you care about the tech details)

[10:03]
That means the app can now do something significant the prototype couldn't for the first time :slightly_smiling_face:

[10:03]
Which is actually getting a performance benefit from the fact that the first page is rendered on the server, for people with JS enabled.


[10:03]
There are some loose ends to tie up though:

[10:04]
e.g. When something gets cached, a timestamp is stored, to determine how long it can be cached for

[10:04]
The server doesn't necessarily use the same timezone as the client. So those timestamps will need to be updated when the store is rehydrated.

[10:04]
That should be possible by adding an effect that listens for the rehydrate action and updates the timestamps.

[10:04]
The REST Services PR has been updated:

[10:04]
https://github.com/DSpace/dspace-angular/pull/62

[10:04]
@wwelling also created a ticket suggesting we take steps to ensure reducers don't accidentally change the previous state.
https://github.com/DSpace/dspace-angular/issues/68

[10:05]
The most popular library to do so is immutable.js . It gives you a collection of special objects, lists and maps that can be safely cloned, added to, removed from etc with without the risk of accidentally mutating a previous version.

[10:05]
The advantage is that you don't have to think too hard about immutable operations. e.g. instead of an array you use an Immutable.List, and it will pretty much have the same operations as an array.

[10:05]
A downside is that you have to remember work with those specific objects, if you forget, you're still at risk of inadvertently mutating the state. Their api's are also slightly different than the plain javascript versions.

[10:05]
But the biggest downside in my opinion is that these objects are slower than their plain javascript counterparts
and they are used in production.

[10:05]
But there is no need to check for immutability in a production build, as redux allows you to write a test for immutability for each action, and if they pass you can be sure it won't happen in production, without sacrificing performance. So that's what I've been doing up to now.

[10:05]
I recently came across middleware for ngrx that can do those tests automatically, and only in development mode.

[10:06]
https://github.com/codewareio/ngrx-store-freeze

[10:06]
It will throw an exception when the state is mutated.

[10:06]
It doesn't guarantee that no action will mutate the state, only the actions you've actually used during development.

[10:06]
I'd suggest we use both: unit tests, and this middleware as a fail safe.

[10:07]
does everybody agree with that?


[10:08]
Is anybody, besides Tim reading this? :slightly_smiling_face:


Nelson Torres [10:08 AM]
I am

James Creel [10:08 AM]
So far so good, but it's a lot to consider.

William Welling [10:08 AM]
I am

Art Lowel [10:08 AM]
good to know :slightly_smiling_face:

[10:09]
ok, last update: @hpottinger verified that the yarn PR works on linux.

[10:09]
He did find an unrelated issue that has since been fixed.

[10:09]
So now we just need a volunteer to test it on windows.

[10:09]
https://github.com/DSpace/dspace-angular/pull/66

[10:11]
Does anybody else have status updates? or something they want feedback on?

Giuseppe Digilio [10:12 AM]
I can test yarn on windows


Art Lowel [10:12 AM]
thanks!

Hardy Pottinger [10:12 AM]
sorry, just now catching up on discussion

Tom Hutchinson [10:12 AM]
joined #angular-ui

Art Lowel [10:14 AM]
@wwelling did you get a chance to look at the current spinner component PR and give your feedback?

William Welling [10:14 AM]
Sorry, no.

Art Lowel [10:15 AM]
Is there anything else related to the UI that people want to discuss?

Tim Donohue [10:16 AM]
notes this all sounds like great progress. I'm following along (while juggling a sick 2 year old)

Tim Donohue [10:18 AM]
I also will note that for folks who missed it, the DSpace 7 Webinar on Tues has slides/video posted at: http://duraspace.org/hot-topics (top of page)

Art Lowel [10:19 AM]
Then I guess I’ll see you in 10 minutes in the rest meeting

Tim Donohue [10:20 AM]
On a sidenote here, @bollini says he's seeing bad performance (hanging / slow load times) from JIRA. If everyone has a chance could you check how it's working where you are in the world? https://jira.duraspace.org/secure/Dashboard.jspa (On my end, it is speedy)

Hardy Pottinger [10:22 AM]
is it on S3?

Tim Donohue [10:24 AM]
Not sure if it's on Amazon at all ...it's hosted with a third party. Just trying to scope the issue here (see if this behavior is specific to Italy or also seen elsewhere, as the REST API team tickets are all in JIRA)

Hardy Pottinger [10:24 AM]
seems speedy here in mid-Missouri, USA

Tim Donohue [10:26 AM]
I'll move this over to #general to see if others can test

Tom Hutchinson [10:26 AM]
Seems fine here in PA

Nelson Torres [10:26 AM]
In Portugal, JIRA seems ok

Tom Desair (Atmire) [10:28 AM]
In Belgium it also seems ok


Waffle APP [10:28 AM]
*artlowel* moved DSpace/dspace-angular#68 from *Backlog* to *Done*
#68 Ensure content in Redux store is immutable
Column
----------------
Done

Open in Waffle

[10:28]
*Art Lowel* moved DSpace/dspace-angular#70 from *Backlog* to *Ready (to be worked on)*
#70 Add ngrx-store-freeze
Column
----------------
Ready (to be worked on)

Open in Waffle

Tom Hutchinson [10:29 AM]
Just to be clear, the UI for DSpace 7 will be this angular-ui. Is that correct? Will xmlui be removed then?

Art Lowel [10:29 AM]
That is correct

[10:30]
both xmlui and jspui have already been removed on dspace’s current master branch

Tim Donohue [10:30 AM]
@th5 : yes, what @art-lowel said. The XMLUI and JSPUI are already removed. Their last release was in DSpace 6.

Tom Hutchinson [10:30 AM]
Thank you

[10:31]
We're about to start a redesign on our dspace. I'm wondering if it's too early to start it in angular.

Tim Donohue [10:31 AM]
It's now the second 1/2 of our DSpace 7 Meeting...which means it's time to switch over to #rest-api for REST API updates with @bollini (and others)

Art Lowel [10:31 AM]
If you’re starting now, I’d say its too early

[10:31]
if you can wait 6 months, maybe

Tom Hutchinson [10:31 AM]
Thanks - very helpful

Tim Donohue [10:32 AM]
(Questions are welcome to continue here...but the meeting is moving to #rest-api channel)

Terry Brady [10:32 AM]
@th5 , I suspect 1-2 years.

Tim Donohue [10:33 AM]
@th5: current (rough) estimates on DSpace 7 would be that it'd be released in 2018 (likely at least a year from now). But that's highly dependent on work in coming months...should have better estimates by OR2017 (in June)

Tom Hutchinson [10:34 AM]
Great. Our catalog (which was just redesigned) is based on bootstrap2. Planning on doing it for DS in xmlui.

Terry Brady [10:35 AM]
Are you using Mirage2? It is quite nice to work with.

Hardy Pottinger [10:35 AM]
@th5: an upgrade to 6x should be "pretty easy" and Bootstrap theming is also pretty straightforward (just use Maven 3.3.9+)

Tom Hutchinson [10:36 AM]
We are but it's basically the out of the box theme. Not tied to any particular frontend for DS. Want to port over our catalog's bootstrap theme.

Hardy Pottinger [10:37 AM]
@antoine has a write-up about how to get a standard bootstrap theme to work with Mirage2

Tom Hutchinson [10:38 AM]
Thank you!

Hardy Pottinger [10:38 AM]
in case anyone else missed it... the meeting has moved over to #rest-api


REST meeting

Attendees

Notes:

Slack Logs (times are all CST)

Andrea Bollini [10:32 AM]
hi, I can start with some update on the rest-api work

[10:33]
@terrywbrady has made a PR to add new resources to the rest-api webapp see https://github.com/DSpace/DSpace/pull/1661


[10:35]
I have take a first look to it and it seems great, so I'm in favor to approve it and improve some aspects later

[10:36]
before to merge I want to give it a try. I will do this in the next couple of hours

Terry Brady [10:36 AM]
Good to hear. I have a few questions about how to handle DSpace objects reference by another DSpace object

Andrea Bollini [10:36 AM]
yes! this pr open a few questions that we want to discuss

Terry Brady [10:36 AM]
We could discuss that here or in the PR comments

Andrea Bollini [10:37 AM]
I think that it will be useful to discuss a bit now, and copy our conclusion on the PR as well

Terry Brady [10:37 AM]
I see that an object can have (1)attributes (2)embedded objects (3)links to external objects

Andrea Bollini [10:38 AM]
yes, the rest model is a pojo

Terry Brady [10:38 AM]
What is the proper way to designate a field/property into one of these locations?

Andrea Bollini [10:38 AM]
it is named with the Rest postfix (ItemRest, BitstreamRest, etc.)

[10:39]
other than the rest model classes we have the hal resources

[10:39]
they are named with a Resource Postfix

[10:40]
the resource class wrap the rest class adding the HAL functionalities (links, embedding)

Terry Brady [10:40 AM]
That helps. How do I designate that something is linked vs embedded?

Andrea Bollini [10:41 AM]
every thinks that is embedded is also linked, the opposite is not true

[10:42]
my idea is to use reflection to automatically discover from the model (rest) class the linked entities and embed the information if
1) available
2) requested by the client

[10:44]
additional links can be added extending the DSpaceResource class (i.e. ItemResource) to "manually" add the links in the constructor (after call the super)

Terry Brady [10:44 AM]
Thanks. It will be good to see some more examples

Andrea Bollini [10:45 AM]
both spring data rest than katharsis use reflection and annotations to discover relations between rest resources

Terry Brady [10:45 AM]
I added owningCollection to RestItem but it did not quite behave as I expected

Luiz Claudio Santos [10:45 AM]
Andrea Bollini: Do the Rest classes (BitStreamRest, ItemRest) proxies to the DSpace classes?

Andrea Bollini [10:45 AM]
@luizsan they are a different representation, in some way a DTO layer

[10:46]
we have the opportunity to build a rest model more close to the domain model

Terry Brady [10:46 AM]
I think we will want to create a document explaining how to code up each of these scenarios (and to explain the data instantiation implications of each option).

Andrea Bollini [10:47 AM]
unfortunately we don't have an answer right now. As we need to build our infrastructure to do some of this job. But, as said we can follow what as been done in other successful projects

[10:49]
for instance you already have get the automatic conversion of an attribute of the itemrest class (the collection) in a link with embedding

Luiz Claudio Santos [10:49 AM]
@bollini do you already any project in mind?

Andrea Bollini [10:50 AM]
@luizsan yes, we have agree to mimic as much as possible the architecture of spring data rest

[10:50]
http://projects.spring.io/spring-data-rest/

[10:51]
please note that spring data rest is NOT what we are using, as we stay on
spring mvc + spring hateoas

Luiz Claudio Santos [10:51 AM]
@bollini ok, I got it, thanks

Tim Donohue [10:51 AM]
Perhaps this would be worth still capturing as "best practices" or "brainstorms" on a Wiki page (or something)? It might help others (still coming on board or in the future) to get up to speed on these ideas


Terry Brady [10:52 AM]
I will start a page


Tim Donohue [10:52 AM]
i.e. more docs (even if they are rough/brainstorms) is a good thing

Terry Brady [10:52 AM]
to capture the types of questions I need answered.


[10:53]
Much of this stack is new to me, so I just want to learn how to make the data flow out of the system regardless of what the underlying technology is

Andrea Bollini [10:54 AM]
take in mind that something can be improved later. For instance we should have a central way to manage lazy-loading

Luiz Claudio Santos [10:54 AM]
@terrywbrady Could you share the page in this slack channel later?

Terry Brady [10:54 AM]
@luizsan , I will once I get a template going


Tim Donohue [10:55 AM]
recommends adding the REST API notes/brainstorms as a subpage of https://wiki.duraspace.org/display/DSPACE/DSpace+7+UI+Working+Group (and also sharing on this channel, of course)

Andrea Bollini [10:55 AM]
in the rest class, you have one resource linked to another. Our converter and/or the resource wrapper will be (at some point) able to check if the linked resource is lazy-loaded

Terry Brady [10:55 AM]
Whatever we do, there is also some hibernate magic happening in the background and we will want to make sure that behavior does waht we need

Andrea Bollini [10:56 AM]
agree

Terry Brady [10:56 AM]
On a side note, I discovered the following bug https://jira.duraspace.org/browse/DS-3515

[10:56]
It makes testing slow since the pagination parameters are not actually used in the HAL browser

Andrea Bollini [10:57 AM]
I will reply to that later

[10:57]
you need to build a proper pagination in the response, I will provide you an example

Terry Brady [10:57 AM]
thanks

Andrea Bollini [10:58 AM]
unfortunately I have issue to access jira today... still trying to figure why

Tim Donohue [10:58 AM]
Sounds like a lot of opportunities for documenting hints/tips here on getting started, best practices, etc :wink: Good discussion

Andrea Bollini [10:58 AM]
the other thing that I want to highlight about this week progress is

[10:58]
https://github.com/DSpace-Labs/Rest7Contract/issues/3

[10:59]
it is about the metadata representation as discussed in the past meeting

Luiz Claudio Santos [10:59 AM]
one question, the DSpace's master branch is the one that you guys are using to develop new thing in the REST-API?

Andrea Bollini [11:00 AM]
right now we are using a feature branch (rest7) that is build on the master

Tim Donohue [11:00 AM]
https://github.com/DSpace/DSpace/tree/rest7

Andrea Bollini [11:00 AM]
this is because our feature branch still fails test due to issue in the dependencies management

Art Lowel [11:00 AM]
@bollini `nullQualifier` means no element?

Andrea Bollini [11:01 AM]
mean no qualifier

[11:01]
dc.title for instance

Terry Brady [11:01 AM]
@bollini , I like that representation if I want to retrieve a single field

Art Lowel [11:01 AM]
right :facepalm: (edited)

Tim Donohue [11:01 AM]
(Once that `rest7` branch stabilizes, it will be merged into "master" and from that point forward development will be on "master". However, as noted by @bollini, there are still known failing tests, etc)

Luiz Claudio Santos [11:01 AM]
thanks

Art Lowel [11:01 AM]
yeah, I like it too

Terry Brady [11:01 AM]
I am less if that is an optimal structure for processing all of the metadata for an item

[11:03]
One common need that I have with the current rest api is that I want to retrieve 200 items and I want specific metadata fields from each of those items. In that use case, the metadata=dc.title,dc.contributor.* is attractive

[11:04]
I should be able to accomplish that with either metadata structure...

Andrea Bollini [11:04 AM]
yes, we can always support such kind of parameter to filter our response

[11:05]
but your use case is more oriented to the discovery layer than the "persistence"

[11:06]
we don't need to have the same structure for both as we can optimize each structure for their specific use cases

Terry Brady [11:06 AM]
That makes sense

Tim Donohue [11:07 AM]
notes we are past the 1/2 hour point. Discussion can still continue though in either #rest-api or #angular-ui as needed

Terry Brady [11:07 AM]
I reserved the following wiki page for the documentation we discussed above: https://wiki.duraspace.org/display/DSPACE/DSpace+7+REST%3A+Coding+DSpace+Objects

[11:07]
I will add more to it and then alert this channel

Andrea Bollini [11:09 AM]
@terrywbrady here you can see how a large list of bitstreamformat is limited to a specific page https://github.com/DSpace/DSpace/blob/rest7/dspace-spring-rest/src/main/java/org/dspace/app/rest/repository/BitstreamFormatRestRepository.java#L63


Tim Donohue [11:09 AM]
Quick note...thanks everyone for joining the discussions today. The next DSpace 7 Mtg will be next Thurs (Mar 9) at the same time (16UTC) in Google Hangouts. Reminders will go to the dspace-devel list next week (and to Slack)

Andrea Bollini [11:11 AM]
this is more or less the "same workaround" that we apply in the current rest implementation: retrieve all the object from the persistence layer and filter our the single page that we need. It is a temporary workaround, the proper way will be to introduce pagination in the persistence layer

Terry Brady [11:11 AM]
I will be out of the office until 3/13, so I will try to leave my contributions in a good spot at the end of today.

Andrea Bollini [11:12 AM]
ok thanks. Another thing about your PR. We want probably discuss about community and collection

[11:13]
I will like to try to hide this difference in the REST layer but I will prefer to move ahead with your current implementation and revise that later

Terry Brady [11:14 AM]
That makes sense to me. I imagine it will be easier to approve some imperfect stuff along the way to get things moving

Andrea Bollini [11:14 AM]
I will open an issue in the REST contract about that, but we need to move on until we take the decision

[11:15]
yes, absolutely. Now we need to prefer move fast over perfectionism

[11:19]
I will be around for still 1hour, feel free to ask more questions if you want

Terry Brady [11:54 AM]
@bollini , I have started some edits to https://wiki.duraspace.org/display/DSPACE/DSpace+7+REST%3A+Coding+DSpace+Objects

Andrea Bollini [12:05 PM]
looks good. I have just added some additional details in the HATEOAS section

Terry Brady [12:06 PM]
thanks. I am still editing and I will work those notes into the doc.

[12:06]
I will also link to the doc from https://github.com/DSpace/DSpace/blob/rest7/dspace-spring-rest/README.md

[12:07]
Will I need to create a PR for that, or can I edit the .md file through github?

Andrea Bollini [12:12 PM]
feel free to edit

[12:12]
at the current stage we can just improve it :wink:

Terry Brady [12:13 PM]
Done. Link added

Terry Brady [12:35 PM]
@bollini , I am still trying to understand something. What do I need to do in the code to trigger these 2 conditions? https://github.com/DSpace/DSpace/blob/rest7/dspace-spring-rest/src/main/java/org/dspace/app/rest/model/hateoas/DSpaceResource.java#L52-L61

Andrea Bollini [12:36 PM]
nothing, any attribute that implement the RestModel interface will be automatically recognized up as a linked resource

Terry Brady [12:38 PM]
Thanks. So this code identifies the embedded RestModels and makes links.

[12:39]
If I want to create a link without embedding an object, I need to do that elsewhere. (I know you answered this earlier, but I am trying to capture this in easy to follow instructions)

Andrea Bollini [12:42 PM]
@terrywbrady https://github.com/Georgetown-University-Libraries/DSpace/pull/9 this is all you need to fix your PR

[12:42]
I need to go now, I will check the chat later

Terry Brady [12:43 PM]
thanks and thanks for the PR!

Terry Brady [1:45 PM]
The org.dspace.content DAO methods for item and bitstream do not support pagination. I am adding those methods to allow pagination of these resources as well.

Tom Desair (Atmire) [1:52 PM]
For items, aren’t we going to use the pagination provided by Discovery? That one already takes into account the resource policies defined on an item.

Terry Brady [1:52 PM]
Probably, but we have no solr interaction right now

[1:57]
I can package this up in a separate PR in case there is controversy. There appear to be slight variations in the method signatures for the findAll() methods for the DSpaceObject DAO classes. I am finding it difficult to test without the ability to paginate on item or bitstream. Every request is returning the entire repository.

Tom Desair (Atmire) [2:13 PM]
The difficult part here is taking into account the resource policies. We could create an “admin section” of the REST API which would allow direct pagination via the database (DAOs) and disregards the resource policies. But that those endpoints should go below a separate “admin” endpoint I think, e.g. `/admin/items`, `/admin/bitstreams`... (edited)

[2:18]
For the “regular” `/items` endpoint, I would use SOLR, that shouldn’t be to difficult and would also allow filter parameters (similar to this https://github.com/milieuinfo/dspace54-atmire/blob/ontwikkel/dspace/modules/rest/src/main/java/org/dspace/rest/SearchResource.java#L94) (edited)

[2:19]
But I don’t have an idea yet on how to efficiently paginate the `/bitstreams` endpoint taking resource policies into account and without doing the authorisation check in-memory. (edited)

Terry Brady [3:21 PM]
I created https://github.com/DSpace/DSpace/pull/1666 which offers pagination for items and bitstreams.

[3:22]
Note that https://wiki.duraspace.org/display/DSPACE/DSpace+7+REST%3A+Coding+DSpace+Objects has been updated as well.

  • No labels