Mark Bussey - Digital Curation Experts
- Steven Ng - Temple University
- Alicia Cozine - Digital Curation Experts
- Many Others
- Discuss Vagrant and what it offers for streamlining development or devops processes.
- Discuss other orchestration tools and their potential use for automating the workload of devops.
- Conversation is the order of the day.
- What does mark know? what do we know?
- a lot of people use vagrant (~15/23) for virtualization of some kind.
- DCE works with many different institutions so the abstraction of virtualization processes is important.
- The stack is very complicated. Virtualization is good for testing / scripting edits, changes so it doesn't need to be done in production.
- DCE is primarily a Mac shop, so is the majority of the community (by vote of laptop brands at least), so non-apple users contributions to the community is important for other non-apple users' adoption of hydra.
- Institutions and organizations at hydra use all sorts of virtualization environments, Amazon Web Services, Google Compute Engine, OpenStack, WebSphere, etc.
- Mark and Alicia have built a lot of VMs on AWS and are working toward a simpler process for building VMs that can deploy to AWS and may other end targets.
- InfoGraphic - Sections / Layers: Platform, Bootstrap, Hydra Stack, Application - the four areas of deployment.
- Concerning VMs for Hydra Camp: need to support all sorts of environments. Problems crop up all over trying to get environments up and running for teaching initiatives.
- Mark showcases his development:
- Iteration 1: github.com/mark-dce/camper
- Iteration 2: Mark learned how to use kickstarter files to build their own boxes from scratch (bootstrap layer).
- Talk of ssh keys for generic users.
- Included installation steps - virtual box guest additions, etc
- Configuration manage all the things!
- The process of using "orchestration management tools" (in this case, vagrant, ansible, and AWS) took DCE's workload from being measured in days (3-4) to being measured in minutes (10-20?).
- Are the bootstrapped boxes published (somewhere) after the work is complete?
- Mostly published on thumb drives due to the purpose of Hydra Camp and related issue of network congestion.
- But they can be published somewhere.
- DCE uses different versions of files for different targets: local boxes, AWS, VSphere, etc.
- Kieran's (my) spiel about packer.io - roughly that Packer can take the role of managing those different versions of files for different targets. Works with all major orchestration tools.
- Connecting to a VM as the vagrant user - problematic when attempting to commit development work.
- A behavioral solution: use "vagrant" only as a "build" user (should be rare) while manually or as a part of the orchestration tool process adding an ssh key (or keys) for a user (users) who will be committing code from that environment.
- A possible technological solution: use https rather than ssh.
- Having these orchestration tools seems incredibly useful to the present members of the community.
- Brief conversation about docker - put docker images on "the registry".
- like ansible galaxy
- like puppet modules
- like hashicorp images
- Is this (the pursuit of researching and implementing orchestration tools) an initiative within the community?
- Where should information about his initiative live?
- Possibility of an interest group? Alicia, Kieran (myself), Mark, (someone else behind me I couldn't see).
- Hydra-in-a-Box community would be very interested in the results of this conversation (id est, Joseph Atzberger)
- What about huge files and using vagrant?
- No immediate solution.
- Next Steps
- Interest group
- Documentation for the meeting
- Be gentle about pull requests.