Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@shtripat
Copy link
Contributor

Community Contribution License

All community contributions in this pull request are licensed to the project maintainers
under the terms of the Apache 2 license.
By creating this pull request I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 license.

Description

If while start of MinIO server, some of IAM systems (e.g. OpenID/ldap) are not reachable, start a thread to check their availability runtime and re-initialize the configurations properly. This makes sure if openid/ldap comes online later, console would allow using the those creds for login.

Motivation and Context

Fixes: #20118

How to test this PR?

Follow instructions at #20118

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Optimization (provides speedup with no functional changes)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Unit tests added/updated
  • Internal documentation updated
  • Create a documentation update request here

Copy link
Contributor

@klauspost klauspost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL.

@harshavardhana harshavardhana force-pushed the reconnect-openid branch 2 times, most recently from 5ce5e85 to 12f748c Compare September 7, 2024 10:46
@harshavardhana
Copy link
Member

I simplified the overall behavior @klauspost instead of having many goroutines, it is done in one self contained place PTAL.

@harshavardhana
Copy link
Member

I simplified the overall behavior @klauspost instead of having many goroutines, it is done in one self contained place PTAL.

PTAL @shtripat

Copy link
Contributor

@taran-p taran-p left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IAMsys stays uninitialized if one of the subsystems is down, so actions by non-root users fail and IAM mc actions hang

@harshavardhana
Copy link
Member

IAMsys stays uninitialized if one of the subsystems is down, so actions by non-root users fail and IAM mc actions hang

that is to be expected, you need to bring those endpoints online @taran-p

@harshavardhana
Copy link
Member

the idea is that those endpoints do come online lazily, and we can operate without them being online when the server is coming up.

@shtripat
Copy link
Contributor Author

shtripat commented Sep 9, 2024

I tried verifying the latest changes. With this change, when openid is down and we restart MinIO server, it keeps waiting for opendid to come online and keeps logging below error till openid comes online.

API: SYSTEM.iam
Time: 04:12:30 UTC 09/09/2024
DeploymentID: 35d9d4e7-0850-4994-ab92-423d2b5e15ba
Error: Unable to initialize OpenID: Get "http://localhost:5557/dex/.well-known/openid-configuration": dial tcp 127.0.0.1:5557: connect: connection refused (*fmt.wrapError)
       6: internal/logger/logger.go:268:logger.LogIf()
       5: cmd/logging.go:29:cmd.iamLogIf()
       4: cmd/iam.go:277:cmd.(*IAMSys).Init()
       3: cmd/server-main.go:980:cmd.serverMain.func14.1()
       2: cmd/server-main.go:561:cmd.bootstrapTrace()
       1: cmd/server-main.go:979:cmd.serverMain.func14()
INFO: Waiting for OpenID to be initialized.. (retrying in 1.280339766s)

API: SYSTEM.authZ
Time: 04:12:31 UTC 09/09/2024
DeploymentID: 35d9d4e7-0850-4994-ab92-423d2b5e15ba
Error: The policies "[readwrite]" mapped to role ARN arn:minio:iam:::role/domXb70kze7Ugc1SaxaeFchhLP4 are not defined - this role may not work as expected. (*errors.errorString)
       7: internal/logger/logger.go:268:logger.LogIf()
       6: cmd/logging.go:54:cmd.authZLogIf()
       5: cmd/iam.go:519:cmd.(*IAMSys).validateAndAddRolePolicyMappings()
       4: cmd/iam.go:366:cmd.(*IAMSys).Init()
       3: cmd/server-main.go:980:cmd.serverMain.func14.1()
       2: cmd/server-main.go:561:cmd.bootstrapTrace()
       1: cmd/server-main.go:979:cmd.serverMain.func14()

During this time console doesnt get initialized and we cannot open it. Once openid comes online everything starts working as expected.

If we dont restart MinIO while openid goes offline temporarily, the console falls back to userid/paswd based login and if we bring openid back online, it starts using the same for login.
Hope this is acceptable behavior and as

  • while start of MinIO if configured openid down, it doesnt start well
  • if openid eventually gets offline while MinIO is running the login sxreen falls back to userid/passwd based login and once openid is online again it starts using the same.

Approving the changes based on the same.

@harshavardhana
Copy link
Member

  • while start of MinIO if configured openid down, it doesnt start well
  • if openid eventually gets offline while MinIO is running the login sxreen falls back to userid/passwd based login and once openid is online again it starts using the same.

I mean it is precisely doing what we wanted to do, it isn't about not working well. SSO down is catastrophic, we cannot allow access to resources.

We could initialize the console earlier than IAM, which is okay. Then, it will work the same way as when SSO goes offline while the server is running.

@harshavardhana
Copy link
Member

We could initialize the console earlier than IAM, which is okay. Then, it will work the same way as when SSO goes offline while the server is running.

@shtripat can you move the code around for starting Console UI in server-main.go

@shtripat
Copy link
Contributor Author

We could initialize the console earlier than IAM, which is okay. Then, it will work the same way as when SSO goes offline while the server is running.

@shtripat can you move the code around for starting Console UI in server-main.go

Juts by moving the console start before IAM init, would start the UI server but still would root creds work for login till we start openid as its already configured?
Also even after successful start and init of openid, we would need to re-init the console to make sure it starts using openid creds for login.

@shtripat
Copy link
Contributor Author

With new commit now behavior is as below

  • in start when openid down, it uses root creds
  • when we bring openid up in middle, after few secs it starts using openid
  • if we bring down openid in middle, it falls back to root creds
  • if we bring back openid again, after few secs it again starts using openid creds

shtripat and others added 7 commits September 10, 2024 21:09
Fixes: minio#20118

Signed-off-by: Shubhendu Ram Tripathi <[email protected]>
Signed-off-by: Shubhendu Ram Tripathi <[email protected]>
Signed-off-by: Shubhendu Ram Tripathi <[email protected]>
Signed-off-by: Shubhendu Ram Tripathi <[email protected]>
Signed-off-by: Shubhendu Ram Tripathi <[email protected]>
@shtripat
Copy link
Contributor Author

With new commit now behavior is as below

  • in start when openid down, it uses root creds
  • when we bring openid up in middle, after few secs it starts using openid
  • if we bring down openid in middle, it falls back to root creds
  • if we bring back openid again, after few secs it again starts using openid creds

Reverted this change as not too safe way to handle

Copy link
Member

@harshavardhana harshavardhana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay with this change but will confirm once with @abperiasamy

@harshavardhana
Copy link
Member

@shtripat please move this PR to EOS only

@mircea-pavel-anton
Copy link

Hi, @harshavardhana !

I see this PR got closed, though I was very interested in this functionality being present, since currently if MinIO starts up before my IDP, then I simply lose all access to the console for my users unless I restart the k8s pod which in turn creates some downtime.

Are there any updates on this? What is the current status?

@harshavardhana
Copy link
Member

Hi, @harshavardhana !

I see this PR got closed, though I was very interested in this functionality being present, since currently if MinIO starts up before my IDP, then I simply lose all access to the console for my users unless I restart the k8s pod which in turn creates some downtime.

Are there any updates on this? What is the current status?

PR will be made available in our EOS releases.

@mircea-pavel-anton
Copy link

Hi, @harshavardhana !

I see this PR got closed, though I was very interested in this functionality being present, since currently if MinIO starts up before my IDP, then I simply lose all access to the console for my users unless I restart the k8s pod which in turn creates some downtime.

Are there any updates on this? What is the current status?

PR will be made available in our EOS releases.

So this will be an enterprise only feature?

@harshavardhana
Copy link
Member

So this will be an enterprise only feature?

Correct.

@mircea-pavel-anton
Copy link

So this will be an enterprise only feature?

Correct.

Well that's just a shame... The work was already done over here. I find it weird that SSO is NOT an enterprise only feature but rather it working properly is...

@harshavardhana
Copy link
Member

harshavardhana commented Sep 22, 2024

Well that's just a shame... The work was already done over here. I find it weird that SSO is NOT an enterprise only feature but rather it working properly is...

@mircea-pavel-anton you are free to work on it, the code is available keep it in your fork.

@harshavardhana harshavardhana reopened this Apr 2, 2025
@harshavardhana harshavardhana merged commit 07f31e5 into minio:master Apr 2, 2025
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Reconnect OpenID configuration at startup, if fails

6 participants