feat: [ISS-2950] IAM auth for AWS redis #3029

willwill96 · 2025-04-12T00:11:29Z

Description

Adds configuration which allows Oauth2 Proxy to connect to AWS redis services using IAM auth.

Motivation and Context

This should be useful for organizations who use AWS IAM auth for redis connections.

Resolves #2950

How Has This Been Tested?

I primarily tested this in my organization's infrastructure. I may be able to get permission to screen share the before/after of these changes, if that's useful. Otherwise, I can try to get a setup working in my AWS cluster.

Checklist:

My change requires a change to the documentation or CHANGELOG.
I have updated the documentation/CHANGELOG accordingly.
I have created a feature (non-master) branch for my PR.
I have written tests for my code changes.

pkg/sessions/redis/aws-iam/auth.go

willwill96 · 2025-04-12T01:01:46Z

pkg/apis/options/options.go

+	flagSet.Bool("redis-aws-use-iam-auth", false, "Use AWS IAM authentication for Redis. Must set --redis-aws-service-name, --redis-aws-cluster-name, & --redis-aws-username to use this feature")
+	flagSet.String("redis-aws-service-name", "", "AWS service name for Redis IAM authentication -  `elasticache` or  `memorydb`")
+	flagSet.String("redis-aws-cluster-name", "", "AWS cluster name for Redis IAM authentication")
+	flagSet.String("redis-aws-username", "", "AWS username for Redis IAM authentication")


Code climate is complaining that this function is too long now. Should the redis flags be extracted to their own flagSet?

Do we need these as flags/could they be added only to the alpha/structured configuration?

There's no need for them to be flags specifically. There is no session/redis config for the alpha config from what I can tell, but I can take a stab at adding that.

@JoelSpeed this was probably not the best call as session settings are only being introduced to the alpha config with one of my PRs

structured config #4: support for session and probes #2632

Issue now would be that we would introduce the AWS Redis support at the toplevel of the alpha config just to refactor / break the config API in a couple months again

willwill96 · 2025-06-10T19:00:24Z

Is there any possibility of this being reviewed/merged?

rfahsel3 · 2025-07-08T15:58:09Z

@JoelSpeed , I see you have a lot of contributions to this project. Is there any chance you could review this or explain what steps we would need to take to get a review? Without this, we will be forced to fork this repo for our organization. I know @willwill96 is amenable to make any changes that would make this MR any easier for you guys to review. Thank you for reading.

rfahsel3 · 2025-07-18T19:20:59Z

@tuunit is there any expectation on when something like this could be reviewed? Is there anything we can do to make it easier or more straightforward on the maintainers? It seems like we are following the process, but we are open to whatever changes we might need to make. We are really really hoping to not have to fork it.

tuunit · 2025-07-22T20:18:09Z

Hi @willwill96 / @rfahsel3, we are only one or two maintainers checking the PRs in our free time. Therefore it often takes quite some time before get around to these things and priorities bugfixes and generic changes over provider specific.

In the case of your change, we don't have an AWS account to validate the changes in your PR. I'll try to make some time for it :)

rfahsel3 · 2025-07-31T12:51:03Z

Thank you very much @tuunit . Your message makes a lot of sense.

JoelSpeed

Is there any way we can add testing locally to prove this is working? Is there a service that we could use to mock the AWS auth?

JoelSpeed · 2025-08-03T12:20:19Z

pkg/apis/options/options.go

+	flagSet.Bool("redis-aws-use-iam-auth", false, "Use AWS IAM authentication for Redis. Must set --redis-aws-service-name, --redis-aws-cluster-name, & --redis-aws-username to use this feature")
+	flagSet.String("redis-aws-service-name", "", "AWS service name for Redis IAM authentication -  `elasticache` or  `memorydb`")
+	flagSet.String("redis-aws-cluster-name", "", "AWS cluster name for Redis IAM authentication")
+	flagSet.String("redis-aws-username", "", "AWS username for Redis IAM authentication")


Do we need these as flags/could they be added only to the alpha/structured configuration?

JoelSpeed · 2025-08-03T12:21:08Z

pkg/sessions/redis/aws-iam/auth.go

+	hexEncodedSHA256EmptyString = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+)
+
+type IAMTokenGenerator struct {


Nit missing comment on exported struct

Does it actually need to be exported? Could be behind an interface?

JoelSpeed · 2025-08-03T12:21:18Z

pkg/sessions/redis/aws-iam/auth.go

+	signer              *v4.Signer
+}
+
+func New(serviceName, clusterName, userName string) (*IAMTokenGenerator, error) {


Missing comment on exported function

JoelSpeed · 2025-08-03T12:22:00Z

pkg/sessions/redis/aws-iam/auth.go

+		return "", fmt.Errorf("AWS IAM request signing failed - %v", err)
+	}
+
+	signedURL = strings.Replace(signedURL, "http://", "", 1)


Can you add a comment explaining why we do this?

I couldn't find any docs explaining why this is recommended, but it is in many of the aws iam examples. For example here: https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/auth-iam.html#auth-iam-Connecting

This value is used as a bearer token so my best guess is that the http:// breaks the validation of that header in some way.

I will add a comment with a link to the aws docs

JoelSpeed · 2025-08-03T12:22:46Z

pkg/sessions/redis/redis_store.go

+		}
+		return opts.AWSUsername, token
+	}
+	// AWS services has a max connection lifetime of 12 hours. This is set to 11 hours to give some buffer time


I assume there's no action needed and the connection refreshes itself automatically after this period?

Yes, once the connection expires, the redis client will reconnect on the next request, using a regenerated auth token.

willwill96 · 2025-08-05T15:06:18Z

Is there any way we can add testing locally to prove this is working? Is there a service that we could use to mock the AWS auth?

I haven't been able to find a way to test this locally. I investigated localstack, which does support a mock service for Elasticache here, but:

The feature does not actually validate the iam auth token - it ignores the authorization and accepts all connections.
The feature is behind a pay wall for the pro accounts.

I'll take a look at adding some unit tests to at least validate what we can.

We are also going to try to put together a screen recording of the important parts working so that we can share here.

willwill96 · 2025-08-08T17:19:24Z

pkg/sessions/redis/aws-iam/auth_test.go

+	"github.com/stretchr/testify/require"
+)
+
+func TestAWSIAMTokenGenerator(t *testing.T) {


I saw another OSS project add a similar unit test when they added IAM auth for postgres:
https://github.com/authzed/spicedb/pull/1858/files

willwill96 · 2025-08-09T01:10:57Z

Hi @JoelSpeed,

Thank you for taking a look at our changes. I think that I have addressed your comments to the best of my ability, but happy to make additional changes if necessary.

Additionally, I have sent an email to you & @tuunit containing a demo of the IAM auth changes, which I hope will inspire a bit more confidence in these changes.

If you have any additional feedback or questions on these changes, I'm happy to help where I can.

Best regards.

Signed-off-by: Jan Larwig <[email protected]>

willwill96 · 2025-08-25T17:13:13Z

@tuunit Thank you for pushing some updates to this MR. I've resolved the linting issues that were causing a build issue before and updated the changelog entry to reflect the latest minor version. Please let me know if there's anything else I can to do help.

rfahsel3 · 2025-08-28T21:22:51Z

Hi @tuunit ! I saw you made some contributions to this MR. Is this ready to merge?

rfahsel3 · 2025-09-09T17:31:45Z

@tuunit and @JoelSpeed , I'm sorry to be annoying, but we are really trying to figure out if we are going to fork this or not. We have a pretty hard deadline at the end of the month. I feel like we were making good progress by sending the video and working through the comments. Do we foresee this being merged in the next week, or should we plan on forking the repo? We just need to know for our planning.

willwill96 requested a review from a team as a code owner April 12, 2025 00:11

github-actions bot added dependencies Pull requests that update a dependency file go labels Apr 12, 2025

willwill96 force-pushed the redis-iam branch from 5481873 to c681958 Compare April 12, 2025 00:13

willwill96 commented Apr 12, 2025

View reviewed changes

pkg/sessions/redis/aws-iam/auth.go Show resolved Hide resolved

willwill96 force-pushed the redis-iam branch from c681958 to c287be0 Compare April 12, 2025 00:56

willwill96 commented Apr 12, 2025

View reviewed changes

willwill96 mentioned this pull request Apr 12, 2025

[Feature]: AWS IRSA Redis Authentication #2950

Open

willwill96 force-pushed the redis-iam branch from 8319d5e to 4961b8e Compare April 16, 2025 05:13

willwill96 closed this Jun 10, 2025

willwill96 reopened this Jun 10, 2025

JoelSpeed reviewed Aug 3, 2025

View reviewed changes

willwill96 force-pushed the redis-iam branch from cb30ef8 to f95f06d Compare August 8, 2025 17:05

github-actions bot added the tests label Aug 8, 2025

willwill96 commented Aug 8, 2025

View reviewed changes

willwill96 added 8 commits August 18, 2025 21:38

feat: [ISS-2950] Add Ability to authenticate with AWS redis services

561fd23

ensure credentials are regenerated on every connection

3ac52f3

add max lifetime to connections

4e939ca

fix lint issue

e81d50a

do not use aws iam if not opted in

7ce6e86

remove aws fields as flags

b5f409d

add additional comments

61df473

Switch to alpha options

ede2770

willwill96 added 5 commits August 18, 2025 21:38

Add unit tests

474d869

rename alpha option to be more explicit

cbd3b8c

fix tests

cbfba29

capitalization

951f627

remove test file

b68aff9

tuunit force-pushed the redis-iam branch from 6f2f915 to 1eb99a5 Compare August 18, 2025 19:45

github-actions bot added the docs label Aug 18, 2025

add changelog entry

58e2e60

Signed-off-by: Jan Larwig <[email protected]>

tuunit force-pushed the redis-iam branch from 1eb99a5 to 58e2e60 Compare August 19, 2025 05:52

willwill96 added 4 commits August 19, 2025 11:21

linting

a273f60

Merge remote-tracking branch 'oauth2-proxy/master' into redis-iam

a0d76f6

fix alpha options

ea6d629

fixes

f7f52a5

Uh oh!

feat: [ISS-2950] IAM auth for AWS redis #3029

Are you sure you want to change the base?

feat: [ISS-2950] IAM auth for AWS redis #3029

Uh oh!

Conversation

willwill96 commented Apr 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

How Has This Been Tested?

Checklist:

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tuunit Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

willwill96 commented Jun 10, 2025

Uh oh!

rfahsel3 commented Jul 8, 2025

Uh oh!

rfahsel3 commented Jul 18, 2025

Uh oh!

tuunit commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rfahsel3 commented Jul 31, 2025

Uh oh!

JoelSpeed left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

willwill96 commented Aug 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

willwill96 commented Aug 9, 2025

Uh oh!

willwill96 commented Aug 25, 2025

Uh oh!

rfahsel3 commented Aug 28, 2025

Uh oh!

rfahsel3 commented Sep 9, 2025

Uh oh!

Uh oh!

willwill96 commented Apr 12, 2025 •

edited

Loading

tuunit Aug 18, 2025 •

edited

Loading

tuunit commented Jul 22, 2025 •

edited

Loading