Provisioned service tests #5

sadlerap · 2022-06-09T14:50:11Z

Implement more tests surrounding the provisioned service specification. This should implement coverage for a few scenarios:

Binding a workload with a provisioned service
Overriding type in the ServiceBinding
Overriding provider in the ServiceBinding
Overriding $SERVICE_BINDING_ROOT in the workload
Have $SERVICE_BINDING_ROOT fall back to a default when not specified

scothis · 2022-06-10T15:45:54Z

features/bindAppToProvisionedService.feature

+              name: $scenario_id
+          spec:
+              foo: bar
+          status:


question does this work? You typically cannot create a resource with status

So far, I haven't run into issues. ProvisionedBackend is just a CustomResource here, there's no backing reconciler anywhere.

scothis · 2022-06-10T15:47:28Z

features/bindAppToProvisionedService.feature

+              apiVersion: apps/v1
+              kind: Deployment
+          """
+    Then Service Binding becomes ready


question the ServiceBinding won't become ready unless the Deployment is found and projected. Where is the deployment defined for this test case?

The deployment in this case is implicitly made in line 33, when we set up the generic application. It automatically gets the name $scenario_id (which gets replaced before submission), and it leaves $SERVICE_BINDING_ROOT unset unless you ask for it to be set.

features/bindAppToProvisionedService.feature

scothis · 2022-06-10T15:52:27Z

features/bindAppToProvisionedService.feature

+              kind: Deployment
+          """
+    Then Service Binding becomes ready
+    And The application env var "SERVICE_BINDING_ROOT" has value "/bindings"


question in which container is this asserting being made? It may not matter currently, but will once we need to control which container is bound and/or project into initcontainers.

This gets asserted in the deployment of the generic test app's container. I don't have anything for handling initcontainers specially.

On reconsideration, since I think I may have misunderstood your question last time:

We use whatever kubectl expose does by default (relevant exposure code can be found here). We might have to revisit this when we do initContainer testing; our current approach approach may or may not work.

features/bindAppToProvisionedService.feature

features/steps/generic_testapp.py

features/bindAppToProvisionedService.feature

features/environment.py

baijum · 2022-07-29T15:21:39Z

A general observation: The indentation for YAML is not consistent. I can see a mix of 2 spaces and 4 spaces.

Probably we can run the YAML through a linter before running the tests.

baijum · 2022-07-29T15:46:08Z

We need a test to ensure the runtime validates well-known secret entries.

baijum · 2022-07-29T15:52:05Z

We need a test to ensure the ClusterRole has three verbs (get, list, and watch). Add a tag @rbac for this scenario.

Implements a few common scenarios surrounding provisioned services and their projected values. Signed-off-by: Andy Sadler <[email protected]>

Overriding the provider field in the service binding should be a separate test from overriding the binding root in the workload. This splits them into two separate test cases. Signed-off-by: Andy Sadler <[email protected]>

The generic-test-app image requires an image label to work correctly. For now, tag it with 'main' as a label to get it working. Signed-off-by: Andy Sadler <[email protected]>

We now assert in this test that a SERVICE_BINDING_ROOT environment variable was set and that the bindings we projected under it have the values we expect them to be. Signed-off-by: Andy Sadler <[email protected]>

sadlerap · 2022-07-29T17:26:09Z

A general observation: The indentation for YAML is not consistent. I can see a mix of 2 spaces and 4 spaces.

Probably we can run the YAML through a linter before running the tests.

Fixed.

We need a test to ensure the runtime validates well-known secret entries.

Two thoughts:

I'm not entirely sure how to test some of the required field types. For example, how can we validate a host?
I'm don't think we should require implementations to look through secrets and attempt to validate them; this seems like a bad idea from a security perspective.

sadlerap · 2022-07-29T18:55:57Z

We need a test to ensure the ClusterRole has three verbs (get, list, and watch). Add a tag @rbac for this scenario.

I'll punt this to a separate PR.

scothis · 2022-07-29T20:38:32Z

We need a test to ensure the ClusterRole has three verbs (get, list, and watch). Add a tag @rbac for this scenario.

There's nothing for us to directly enforce relating to RBAC. The "must"s in the RBAC section are qualified by a proceeding should. While not a best practice, it's perfectly within spec for an implementation to grant itself cluster-admin and not need any additional permissions. It's also within spec for a cluster to not use RBAC.

What we can test is to create a CRD for a provisioned service and a workload, with the respective ClusterRole permissions and then validate that the implementation is able to complete the binding. This should not involve assertions about the specific permissions of any ClusterRole.

baijum

LGTM

sadlerap force-pushed the provisioned-service-tests branch from 116ce2c to b9f951c Compare June 9, 2022 17:26

sadlerap marked this pull request as ready for review June 9, 2022 17:26

sadlerap requested review from baijum and scothis June 9, 2022 17:27

scothis reviewed Jun 10, 2022

View reviewed changes

sadlerap force-pushed the provisioned-service-tests branch 2 times, most recently from 25530f7 to 665044f Compare June 24, 2022 16:20

scothis reviewed Jul 6, 2022

View reviewed changes

features/steps/generic_testapp.py Outdated Show resolved Hide resolved

scothis reviewed Jul 6, 2022

View reviewed changes

features/steps/generic_testapp.py Outdated Show resolved Hide resolved

features/bindAppToProvisionedService.feature Show resolved Hide resolved

features/bindAppToProvisionedService.feature Show resolved Hide resolved

sadlerap requested a review from scothis July 20, 2022 20:55

scothis approved these changes Jul 21, 2022

View reviewed changes

sadlerap force-pushed the provisioned-service-tests branch 3 times, most recently from 89f3590 to 049129e Compare July 27, 2022 18:58

sadlerap added 4 commits July 29, 2022 11:52

test: implement tests for provisioned services

95bf97a

Implements a few common scenarios surrounding provisioned services and their projected values. Signed-off-by: Andy Sadler <[email protected]>

Split provider override test from binding root override

b48b99c

Overriding the provider field in the service binding should be a separate test from overriding the binding root in the workload. This splits them into two separate test cases. Signed-off-by: Andy Sadler <[email protected]>

Fix generic-test-app from not deploying correctly

5a68d2a

The generic-test-app image requires an image label to work correctly. For now, tag it with 'main' as a label to get it working. Signed-off-by: Andy Sadler <[email protected]>

Check projected bindings on defaulted binding root

a264645

We now assert in this test that a SERVICE_BINDING_ROOT environment variable was set and that the bindings we projected under it have the values we expect them to be. Signed-off-by: Andy Sadler <[email protected]>

sadlerap force-pushed the provisioned-service-tests branch from 049129e to a264645 Compare July 29, 2022 16:55

sadlerap mentioned this pull request Jul 29, 2022

Direct secret projection #7

Merged

baijum approved these changes Jul 30, 2022

View reviewed changes

baijum merged commit f053f9e into servicebinding:main Jul 30, 2022

sadlerap deleted the provisioned-service-tests branch August 1, 2022 15:31

Provisioned service tests #5

Provisioned service tests #5

Uh oh!

Conversation

sadlerap commented Jun 9, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

scothis Jun 10, 2022

Choose a reason for hiding this comment

Uh oh!

sadlerap Jun 10, 2022

Choose a reason for hiding this comment

Uh oh!

scothis Jun 10, 2022

Choose a reason for hiding this comment

Uh oh!

sadlerap Jun 10, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

scothis Jun 10, 2022

Choose a reason for hiding this comment

Uh oh!

sadlerap Jun 10, 2022

Choose a reason for hiding this comment

Uh oh!

sadlerap Jul 20, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

baijum commented Jul 29, 2022

Uh oh!

baijum commented Jul 29, 2022

Uh oh!

baijum commented Jul 29, 2022

Uh oh!

sadlerap commented Jul 29, 2022

Uh oh!

sadlerap commented Jul 29, 2022

Uh oh!

scothis commented Jul 29, 2022

Uh oh!

baijum left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sadlerap commented Jun 9, 2022 •

edited

Loading