what

• Improve imports and deep-merging of variables of any complex types
• Use the latest variant2 features (recursive user-defined functions, list of file paths in config) for stack configuration module

why

• Simplify code
• Deep-merge any number of import files with any complex structures (maps/objects)
• Deep-merge all variables in this order: global-scoped, component-type-scoped, and component-scoped
• Deep-merge all backend attributes in this order: global-scoped, component-type-scoped, and component-scoped

references

• mumoshu/variant2#43

test

 ✗ . example ⨠ atmos stack config eks -s ue2-dev --config-type=vars
"cluster_kubernetes_version": "1.18"
"environment": "ue2"
"namespace": "eg"
"region": "us-east-2"
"stage": "dev"
"test-map":
  "a": "a4"
  "atr":
    "atr1": "1-1"
    "atr2": "2-4"
    "atr3": 3
    "atr4": 4
    "atr5": 5
    "list":
    - "1c"
  "atr-2":
    "atr1": 1
    "atr2": 2
  "b": "b2"
  "c": "c1"
  "d": "d4"
  "e": "e4"
  "f": "f4"
  "g": "g1"
  "list":
  - 4
  - 5
  - 6
  "list2":
  - 1
  - 2
  - 3

what

• Add interactive mode to Terraform commands
• Add 10 levels of imports

why

• Support interactive mode for TF commands that require user input (e.g. when moving TF state from local to remote state)
• The current 5 levels of imports were arbitrary and not enough in some cases

what

• Use hierarchical imports
• Use new stacks config format
• Add generic Terraform backend config
• Simplify

why

• Hierarchical imports - support import up to five level deep (so we can, for example, import environment globals into a stack, which in turn can import globals for the entire solution)

• Use new stacks config format - all stack config files use the same schema. No special config files, they are all the same and can be arbitrary named. All Terraform and helmfile vars, commands, and backends are defined in separate sections in any of the config files or imported config files (no pollution of Terraform variables with provided but not declared variables, which will be an error in TF 0.14). All sections in the config files and imported config files are merged together into configs consumed by the relevant modules in the CLI

• Add generic Terraform backend config - the ability to specify many different Terraform backends (e.g. s3, remote, vault) and select a particular one per component, stage, environment, or the entire solution

what

• chmod kubeconfig to 600 to avoid warning messages

why

• Avoid warnings like

  WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /dev/shm/uw2-demo-kubecfg
  WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /dev/shm/uw2-demo-kubecfg

what

• Rename kubeconfig-profile-pattern to helm-aws-profile-pattern
• Use helm-aws-profile-pattern to select an AWS profile for helmfile commands

why

• helm-aws-profile-pattern is used not only for kubeconfig, but also for helm and helmfile
• By default, assume the role specified by the helm-aws-profile-pattern when calling the helmfile CLI (still having the option to specify a different AWS profile)

what

• Add region to kubeconfig commands
• Add helmfile-command job

why

• These features were left behind during the transition to stacks

what

• Update CLI for Terraform state backend

why

• Move the backend config logic from the variant files into YAML configuration
• Separate the backend configs from Terraform configs - do not pollute Terraform vars with not-related variables
• Make terraform-backend.variant completely generic - it will work with any backends (e.g. s3, remote) and is completely configuration driven

what

• Update to stacks (instead of environment/stage)
• Update to components (instead of projects)
• Add import to all configs
• Remove hardcoded settings
• Add Terraform S3 backend generation
• Simplify

why

• Use stacks instead of environment/stage to define configurations - this will allow arbitrary naming the stacks and not tying them to env/stage, as well as separation of stacks for the same environment/stage into different files (e.g. terraform config in one file, helmfiles config in another), or separation of stacks by AWS resources
• Add import to YAML stack configs - allow automatically importing global-globals and stack-globals into each configuration
• Remove hardcoded settings - make the CLI generic (no hardcoded namespaces, regions, IAM roles, etc.)
• Add (and simplify) the CLI commands to generate Terraform S3 backend configs for each Terraform component

what

• Update to support reference architectures

why

• Rename opsctl to atmos
• Rename projects/ to components/
• Add vendor CLI for synching reference architecture components
• Add terraform-backend CLI to generate Terraform S3 backends for Terraform components

what

• Allow terraform clean to take --stage and --environment options, even though they will be ignored

why

• Some scripts and tools provide those options for all Terraform commands that operate on a project, so it is convenient to allow them to provide the options to terraform clean even if they will be ignored, rather than raise an error about unknown options