Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@ejholmes
Copy link
Contributor

@ejholmes ejholmes commented Nov 1, 2018

When stacker attempts to update a stack that's already in UPDATE_IN_PROGRESS, the existing behavior is to throw an exception:

[2018-11-01T22:19:50] acme-inc: failed (Stack: "acme-inc" cannot be updated nor re-created from state UPDATE_IN_PROGRESS: Update already in-progress)

In a CI/CD environment where lots of concurrent changes are happening, this can be problematic.

This PR adds a new per-stack in_progress option, which allows you to configure the behavior of stacker when the stack is UPDATE_IN_PROGRESS or CREATE_IN_PROGRESS. The default is the current behavior (error out), but can be set to wait, which will cause stacker to wait for the stack to be updateable before proceeding.

@ejholmes ejholmes requested a review from phobologic November 1, 2018 23:01
Copy link
Member

@phobologic phobologic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome. Not sure I love the in_progress config name, but I can't think of a better one and don't want to delay this for some bike shedding :)

@phobologic
Copy link
Member

Some questions:

  • anything we should worry about with rollback states or anything like that?
  • do we only want it to be per-stack (and not globally)?

@russellballestrini
Copy link
Member

russellballestrini commented Nov 1, 2018

concurrency_handler: error default

concurrency_handler: wait

@ejholmes
Copy link
Contributor Author

ejholmes commented Nov 2, 2018

anything we should worry about with rollback states or anything like that?

Probably. I just kept it simple for this version, but in reality it should probably not return True immediately after waiting, and instead continue down with the new stack status so that rollbacks get handled.

do we only want it to be per-stack (and not globally)?

I could go either way. I kinda lean towards per-stack just cause that's more flexible, but we could do both (per-stack overrides global default).


Re: the naming, I'm also not a huge fan. It's purpose is to configure the desired behavior when a stack is in an "in progress" state, or otherwise not updatable (CREATE_IN_PROGRESS, UPDATE_IN_PROGRESS, DELETE_IN_PROGRESS, UPDATE_COMPLETE_CLEANUP_IN_PROGRESS). All of these have IN_PROGRESS in the name, which is why I went with in_progress.

Just brainstorming:

in_progress: wait
in_progress_behavior: wait
operation_in_progress: wait
not_ready: wait

concurrency_handler makes me think threading, and not necessarily accurate (it may just be that someone did something manually, and not necessarily a result of concurrent stacker builds).

@ejholmes
Copy link
Contributor Author

ejholmes commented Nov 2, 2018

I've changed the implementation slightly so that it's all implemented within _launch_stack, instead of in the aws provider. This should be better because:

  1. SIGTERM/SIGINT works properly
  2. It let's us update the status of the stack to indicate that stacker is waiting

asciicast

@russellballestrini
Copy link
Member

I vote for in_progress_behavior:

@ejholmes
Copy link
Contributor Author

ejholmes commented Nov 2, 2018

Cool, I like in_progress_behavior too. Updated to that.

@phobologic
Copy link
Member

Yeah, in_progress_behavior is probably the most descriptive/easy to understand. It's long, and that's annoying, but I think it's the best.

Copy link
Member

@phobologic phobologic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, this got crazy simple - it seems like we likely will want some tests though for this new status, even though it's just an extension of the Pending status.



PENDING = PendingStatus()
WAITING = PendingStatus(reason="waiting")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be worth something like waiting on previous update for the reason, just to be more explicit.

@russellballestrini
Copy link
Member

@ejholmes could we get this PR rebased and merged in? We had people in the chatroom asking for it.

@ejholmes ejholmes merged commit fadf97d into master Dec 5, 2018
@ejholmes ejholmes deleted the in-progress branch December 5, 2018 03:38
phrohdoh pushed a commit to phrohdoh/stacker that referenced this pull request Dec 18, 2018
Add `in_progress` option to stack config.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants