Gateways2020

The tutorial was presented at Gateways 2020 as a full day (~8 hrs) in total split up across 2 days. The materials in the Gateways2020 directory are organized in accordance with the schedule of events at the conference, as detailed in the list below. Supplemental materials, such as Dockerfiles and Exercise instructions, have been included as well.

Day 1

Part 1 (1 hr 30 min)

1. Introduction to Containers by Jeremy Fischer, Indiana University

   • Discussion of how containers can help Gateways
   • What is Docker?
   • Docker Hub
   • What is Singularity?
   • SingularityHub
   • Differences / relationship to each other

2. Environment Introduction by Eric Coulter, Indiana University

   • Distributed login credentials for tutorial resources (not available outside of conference attendance)
   • Overview of container development environment

3. Simple (Docker) Container Creation by Sanjana Sudarshan, Indiana University

   • Build via Dockerfile
   • Exploring Dockerfile structure/conventions
   • Container examples

4. Exercise - Docker Build by Stephen Bird, Indiana University

   • Build simple Docker container

Part 2 (1 hr 30 min)

5. Singularity Containers by Eric Coulter, Indiana University

   • Definition Files
   • Creating Singularity containers (from scratch)
   • Singularity user environment

6. Docker To Singularity Conversion by Peter Vaillancourt, Cornell University

   • Best Practices for conversion
   • Example conversion

7. Exercise - Singularity by Eric Coulter, Indiana University

   • Convert to Singularity
   • Build via definition file
   • Upload to registry
   • Run via SLURM

8. Gateways Tie-in by Suresh Marru, Indiana University

   • Overview of components in a Gateway lifecycle
   • Big picture in a gateways context
   • Discuss Part 4 Bucket choice

Day 2

Part 3 (1 hr 30 min)

1. Open Q&A by Jeremy Fischer, Indiana University

   • Catchup and unresolved questions

2. Exercise - MATLAB Container Build by Stephen Bird, Indiana University

   • Dockerfile walkthrough
   • Container build and run
   • Conversion

3. Best Practices & Advanced Topics by Peter Vaillancourt, Cornell University

   • Lifecycle of Containers
   • Development vs. Production
   • Reducing container sizes
   • Data management
   • Reproducibility
   • Security
   • Container Orchestration

Part 4 (1 hr 30 min)

4. Containers for Gateways with Exercises by Suresh Marru, Indiana University

   • Intro to Gateways
   • Sign in to a gateway and run a pre-packaged containerized application with pre-provided inputs
   • Execute applications with user provided input files and plot outputs
   • Add a Containerized application to the Gateway
   • Goals to Cover:
     • What is a Gateway for?
     • What features do Gateways add when using containers?
     • How can Gateways now enable the community with these containers?
     • Run/provide your app via a gateway!

5. Bucket of Topics (Open to Audience Input) by Eric Coulter and Suresh Marru, Indiana University

   • Gateway Hosting
   • Container Orchestration and management
   • CI/CD Overview (Code -> Gateways pipelines)
   • Container Registries (Building or usage of public registries)
   • Decisions about container vs. host for software/dependency locality