Aggregate attributes from remote adapters #99

sanjeevchopra · 2017-03-13T20:43:51Z

No description provided.

sanjeevchopra · 2017-03-13T21:16:34Z

...s/src/test/java/com/ge/predix/integration/test/PolicyEvaluationWithAttributeConnectorIT.java

+        attributeConnector.setIsActive(true);
+        attributeConnector.setMaxCachedIntervalMinutes(60);
+        attributeConnector.setAdapters(new HashSet<>(Collections.singletonList(
+                new AttributeAdapterConnection("https://acs-asset-adapter.run.aws-usw02-dev.ice.predix.io/v1/attribute",


please parameterize the adapter settings. settings will change for each env

sanjeevchopra · 2017-03-13T21:20:34Z

...s/src/test/java/com/ge/predix/integration/test/PolicyEvaluationWithAttributeConnectorIT.java

+                new HttpEntity<>(attributeConnector, httpHeaders), AttributeConnector.class);
+    }
+
+    private void setupPredixAcs() throws IOException {


can these 3 methods can be shared between different integ tests - maybe in a new version of createTestZone method, which tests can start using over time

I haven't refactored this yet but we can as part of this story or maybe another one?

sanjeevchopra · 2017-03-13T21:21:08Z

...s/src/test/java/com/ge/predix/integration/test/PolicyEvaluationWithAttributeConnectorIT.java

+        }
+
+        this.zone = this.zoneHelper.createTestZone(this.acsAdminRestTemplate, this.acsZone1Name, this.registerWithZac);
+        this.configureAttributeConnector();


where is the data in the adapter populated ? please add it as part of this test.

Done (Asset data is now configured/deconfigured in this test class)

sanjeevchopra · 2017-03-13T21:43:47Z

service/src/main/java/com/ge/predix/acs/attribute/readers/ExternalAttributeReader.java

+
+    private OAuth2RestTemplate getAdapterOauth2RestTemplate(
+            final AttributeAdapterConnectionEntity attributeAdapterConnectionEntity) {
+        ClientCredentialsResourceDetails clientCredentials = new ClientCredentialsResourceDetails();


rest template should be cached in this object, instead of creating on every request. This will prevent a "get token" on every request

pls review anything else which can be done once or at construction time

sanjeevchopra · 2017-03-13T21:45:01Z

service/src/main/java/com/ge/predix/acs/attribute/readers/ExternalAttributeReader.java

+            ResponseEntity<AttributesResponse> attributesResponse;
+            try {
+                attributesResponse = this.getAttributesFromAdapter(adapterUrl, adapterEntity, adapterRestTemplate);
+            } catch (final RestClientException e) {


catch 'Throwable' here to ensure evaluation service can handle the AttributeRetrievalException

Lets make sure there is a test which asserts that policy eval service handles any exception and returns a INDETERMINATE

Done. Also there already seems to be a test that asserts that Effect.INDETERMINATE is produced when a RuntimeException exception is thrown: PolicyEvaluationServiceTest#testPolicyEvaluationExceptionHandling

sanjeevchopra · 2017-03-13T21:49:16Z

service/src/main/java/com/ge/predix/acs/attribute/readers/ExternalAttributeReader.java


-                // TODO: Reset the TTL for the policy evaluation cache key concerning this resource to:
-                // total customer TTL - this.attributeCache.getConfiguredTtlInSeconds()
+                //long policyEvalDecisionCacheTtlInSeconds =


Policy eval simply constructs a key and call the decision cache. When to use the cached decision will have to be in the decision cache layer. The decision cache does not have a TTL configured on the cache itself - see AbstractPolicyEvaluationCache.isCachedRequestInvalid - after looking up both the decision and the lastModifiedTime for the resource(& subject) involved, it decides to use the decision or not.
We need to modify this code path to:

check if resource or subject connector is enabled

if yes, the decision cache should be invalidated if its been cached for >= (connector.CacheInterval - attributeCacehInterval - 1 min for safety).

This is one way to make this functional - we should reconsider the relationship between decision and attribute cache when we implement a real cache for attributes. Maybe one abstraction should provide the decision Or attributes (if the decision is stale).

As discussed, implemented this; a related test hasn't been added yet because it would be time-based and thus error prone.

sanjeevchopra · 2017-03-13T21:56:28Z

service/src/test/java/com/ge/predix/acs/zone/management/ZoneRepositoryTest.java

        Assert.assertNull(this.adapterRepository.findOne(adapterId));
    }

-    @Test


why remove ? isnt this the expected behavior right now ?

This has been re-introduced.

sanjeevchopra · 2017-03-13T22:02:57Z

...ava/com/ge/predix/acs/service/policy/evaluation/PolicyEvaluationWithAttributeReaderTest.java

+                .thenThrow(AttributeRetrievalException.class);
+
+        PolicyEvaluationResult evalResult = this.evaluatePolicy();
+        Assert.assertEquals(evalResult.getEffect(), Effect.INDETERMINATE);


this test would be better if you do a successful PERMIT first, then only mock the rule above to produce a INDETERMIANTE.

Also, in this case eval service should set a appropriate message in PoliceEvaluationResult.message indicating some problem with the adapter, along with reference info for the adapter. Just enough info, so the end-user can pass on the reference to service owner who is using ACS.

There already is a test for a successful Effect.PERMIT: PolicyEvaluationWithAttributeReaderTest#testPolicyEvaluation so not sure how adding another one here would add value.

Also the adapter endpoint, UAA token URL and UAA client ID are added as part of PolicyEvaluationResult.message.

sanjeevchopra · 2017-03-13T22:06:25Z

service/src/main/java/com/ge/predix/acs/attribute/readers/AttributeReaderFactory.java

+            return this.privilegeServiceSubjectAttributeReader;
+        }
+
+        return new ExternalSubjectAttributeReader(zone,


This factory needs to make sure that a subject or resource externalreader is only created once for each zone.

also need to check isActive property on the connector, once its added to the entity

Done (uses maps keyed on the zone ID). The isActive property is now also checked.

sanjeevchopra · 2017-03-14T22:18:09Z

...s/src/test/java/com/ge/predix/integration/test/PolicyEvaluationWithAttributeConnectorIT.java

+public class PolicyEvaluationWithAttributeConnectorIT extends AbstractTestNGSpringContextTests {
+
+    @Value("${ZONE1_NAME:testzone1}")
+    private String acsZone1Name;


do you need 3 acs zones for this test ? Also, pls use a unique zone name to allow concurrent execution with other integ tests. (now that we dont use sub-domain routing, any zone name can be provisioned on the fly)

Point taken.. removed all but 1 zone. If we go with a separate integration test suite for connector-related policy evaluation integration tests, we can create a unique zone then but since this currently runs as part of the existing integration test suite I'm reusing a zone.

sanjeevchopra · 2017-03-14T22:28:14Z

discussed this with anubhav:
acs-integration test should push the asset adapter and run the connector integ test concurrently with the other integ tests. pls add to a separate test suite, if needed.

anubhavi25 · 2017-03-17T00:10:28Z

@sanjeevchopra: we should probably create a separate story to push the asset adapter as part of mvn verify since I'm not sure how difficult it is to push 2 apps via 2 executions of the cf-maven-plugin in the pre-integration phase. For now, it's already pushed and available on CF3 integration.

anubhavi25 · 2017-03-17T05:48:44Z

...s/src/test/java/com/ge/predix/integration/test/PolicyEvaluationWithAttributeConnectorIT.java

    private OAuth2RestTemplate assetRestTemplate;
    private boolean registerWithZac;
-    private URI resourceAttributeConnectorUrl;
+    private URI resourceAttributeConnectorUrl = URI.create(this.zoneHelper.getAcsBaseURL() + "/v1/resource/connector");


This will throw a NullPointerException when the test is run (or not run tests at all) since this.zoneHelper isn't initialized yet. We should move this initialization to the @BeforeClass method below.

Also the connector URI is /v1/connector/resource :) . It might just be better to use AcsApiUriTemplates.V1 + AcsApiUriTemplates.RESOURCE_CONNECTOR_URL to construct the URI path like before so that we only have to make a change in one place.

anubhavi25 · 2017-03-17T05:52:52Z

model/src/main/java/com/ge/predix/acs/rest/AttributeConnector.java

 public class AttributeConnector {
    private boolean isActive;

+    @JsonProperty(defaultValue="480", required=false)


I don't think this will actually work.. Jackson's core databind currently only uses defaultValue for documenting the expected default value of the property instead of actually enforcing it. If you want to keep this, we need to also change the actual assignment to use a default value like: private int maxCachedIntervalMinutes = 480;

anubhavi25 · 2017-03-17T06:04:55Z

...s/src/test/java/com/ge/predix/integration/test/PolicyEvaluationWithAttributeConnectorIT.java


    @Test(dataProvider = "adapterStatusesAndResultingEffects")
-    public void testPolicyEvaluationWithAdapters(final boolean adapterActive, final Effect effect) throws Exception {
-        HttpHeaders httpHeaders = this.getHeadersWithAcsZoneId();


Since the zone ID doesn't change throughout the test, it's more efficient to instantiate the zone headers once and use them instead of constructing multiple instances by calling zoneHeader() multiple times

since this is a test, I gave precedence readability over efficiency.

Can't we get both with: HttpHeaders zoneHeaders = this.zoneHeader(); ?

sanjeevchopra · 2017-03-17T19:24:06Z

service/src/main/java/com/ge/predix/acs/attribute/readers/ExternalAttributeReader.java

+        String uaaClientId = attributeAdapterConnectionEntity.getUaaClientId();
+        String uaaClientSecret = attributeAdapterConnectionEntity.getUaaClientSecret();
+
+        Integer key = new HashCodeBuilder().append(uaaTokenUrl).append(uaaClientId).append(uaaClientSecret)


better to implement hashcode in AdapterConnection class and use the object itself as the key, so the key can be leveraged anywhere. plus its better encapsulation

…tes from connectors configured in current zone. Signed-off-by: Sanjeev Chopra <[email protected]> Signed-off-by: Frank Gasparovic <[email protected]>

sanjeevchopra · 2017-03-21T05:05:42Z

Pull request build is clean - integration-tests # 318-321
https://predix1.jenkins.build.ge.com/job/Predix-Security/job/ACS/job/ACS-PullRequest/job/acs-integration-pullrequest/320/

sanjeevchopra commented Mar 13, 2017

View reviewed changes

irinaepshteyn force-pushed the US92090 branch 2 times, most recently from 55f8438 to 1522538 Compare March 14, 2017 18:07

sanjeevchopra commented Mar 14, 2017

View reviewed changes

irinaepshteyn force-pushed the US92090 branch from c1cee2e to d631b0e Compare March 15, 2017 17:41

FrankGasparovic force-pushed the US87156 branch 2 times, most recently from ac38a6f to e12ef94 Compare March 15, 2017 20:24

sanjeevchopra changed the base branch from US92090 to develop March 16, 2017 21:07

FrankGasparovic force-pushed the US87156 branch from b80b645 to 0fad616 Compare March 16, 2017 22:08

anubhavi25 suggested changes Mar 17, 2017

View reviewed changes

anubhavi25 reviewed Mar 17, 2017

View reviewed changes

sanjeevchopra changed the title ~~For review only: Aggregate attributes from remote adapters~~ Aggregate attributes from remote adapters Mar 18, 2017

sanjeevchopra commented Mar 18, 2017

View reviewed changes

sanjeevchopra self-assigned this Mar 18, 2017

sanjeevchopra force-pushed the US87156 branch 2 times, most recently from c183c35 to 5c86c4a Compare March 19, 2017 19:31

Implement ExternalReaders for subject and resources to source attribu…

1696a22

…tes from connectors configured in current zone. Signed-off-by: Sanjeev Chopra <[email protected]> Signed-off-by: Frank Gasparovic <[email protected]>

sanjeevchopra force-pushed the US87156 branch from b24e39b to 1696a22 Compare March 21, 2017 05:05

sanjeevchopra merged commit a01ad42 into develop Mar 21, 2017

anubhavi25 deleted the US87156 branch March 21, 2017 05:53

Aggregate attributes from remote adapters #99

Aggregate attributes from remote adapters #99

Uh oh!

Conversation

sanjeevchopra commented Mar 13, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanjeevchopra Mar 13, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanjeevchopra Mar 13, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anubhavi25 Mar 16, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanjeevchopra Mar 13, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anubhavi25 Mar 17, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanjeevchopra commented Mar 14, 2017

Uh oh!

anubhavi25 commented Mar 17, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anubhavi25 Mar 17, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anubhavi25 Mar 17, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanjeevchopra commented Mar 21, 2017

Uh oh!

Reviewers

sanjeevchopra Mar 13, 2017 •

edited

Loading

sanjeevchopra Mar 13, 2017 •

edited

Loading

anubhavi25 Mar 16, 2017 •

edited

Loading

sanjeevchopra Mar 13, 2017 •

edited

Loading

anubhavi25 Mar 17, 2017 •

edited

Loading

anubhavi25 commented Mar 17, 2017 •

edited

Loading

anubhavi25 Mar 17, 2017 •

edited

Loading

anubhavi25 Mar 17, 2017 •

edited

Loading