ECS Stack Assumptions

• Will access main account resources during transition
• Will access main account internal load balancers
• Will access main account DNS
• Main account resources will not access ECS resources
• Services running in ECS should be rebuild/redeployed weekly
• Some mechanism should be in place to scale up

Pre-requisites

• Direct all application output to STDOUT
  • Direct java service output to console when running in docker #2337
  • Determine the level of filtering that is needed before we go live
• Provide a reasonable log processing config for each service
• Customize ECS roles per service
  • Merritt Admin (many permissions)
  • Merritt Service (S3, SSM, perms needed for networking)
  • Open Source Service (hopefully minimal)
• Determine how email will be handled from the stack

Service Chart

• ✅ Migrate admin tool

  • Needs access to main account RDS
  • Needs access to main account DNS (zk hosts) and internal load balancers
  • Would like access to ZFS (not feasible to mount to ECS)
  • Terry will start a ticket... assume new user pool, vpc

• Migrate UI

  • include load balancer and WAF
    • Note that IAS had some challenges configuring WAF rules in CloudFormation
  • Make certain we have a solid approach for rails master.key handling
  • Tuning: what is the optimal container cpu/memory and the optimal number of copies
  • UI must have an EBS volume with sufficient storage space for a file upload

• Migrate LDAP

  • Validate cert deployment
  • Confirmed: Eric can use Session Manager and port forwarding to access the ldap admin tool
    • most ldap functions have migrated to admin tool
    • user creation will be implemented in the admin tool
    • the ldap command line tools will also be available from the container
  • Ensure data backup to S3
  • Ensure that Eric has a reliable LDAP Admin client

• Migrate Audit

• Migrate Replic

• Configure EFS for the Stack

• Migrate Access Service

  • Configure EFS
  • Perform assembly on EFS in spite of slowness

• Migrate Inventory (Part 1)

  • Leave local id service behind in the main account until ingest has migrated
  • Inventory should use Access (rather than Storage) to retrieve manifests

• Cloud Library - Implement download to S3

  • need performant working folder solution (probably S3 over EFS) Migrate "ingest folder" file system to S3 #2300 Part 1

• Migrate Storage (EFS or S3)

  • Drive storage workload via ZK rather than API calls Consider driving storage requests via ZK rather than by API call from ingest #2339 (In progress)
  • ~otherwise, determine a suitable solution for long-running storage api calls Storage Service Hostname Resolution in an ECS Context #2233 ~
  • Implement manual service scaling capability

• Migrate Ingest

  • Migrate collection profiles to CDLUC3 repo and deploy to S3
  • Store downloaded producer files in S3 Migrate "ingest folder" file system to S3 #2300 Part 2
  • ⚠ Ensure that all depositors can allow us to download without depending on IP ranges Risk
    • Migrate away from the dependency for campus depositors to know Ingest Server IP addresses #1838
    • keep a download machine running in the main account
    • encourage depositors to perform a push to S3
  • Implement manual service scaling capability

• Disable Main Account Local Id Service

• Migrate ZK

  • ensure proper backup and restore
  • ensure systematic shutdown and restart

• Migrate RDS

• Migrate Misc crons

  • Replace remaining cron jobs with AWS Code Build or explore AWS Batch
    • consistency/billing
    • billing visualizations
    • Migrated Nuxeo to an ECS Task
    • Re-create billing visualization as an ECS Task

• Service Refinement

  • Implement Responsive Service Scaling Tool
  • Implement Weekly cycling of running containers
  • Implement Graceful Shutdown Mode for Services
    • Stop taking new requests
    • Complete active processing

Migration Issues

• SSM variables will need to be replicated
  • this will create a temporary challenge for credential rotation
  • this is an opportunity to scrutinize SSM variable use
• Misc ZK
  • ensure no name collisions for backups from different stacks