Authenticate audit events #571

dividedmind · 2018-06-16T00:27:52Z

This adds 'authenticate' audit events to the audit stream.

Note the volume of these messages is substantial so we should consider whether they should be enabled by default, or somehow filtered. (These messages weren't present in v4.)

Comments welcome especially on the message schema (see docs/audit.md).

jonahx

I've reviewed everything except the authenticate_controller code that you're changing, and only glanced at the tests. I'll give them a closer look when I do the final review but they look good so far anyway.

jonahx · 2018-06-18T15:09:58Z

app/models/audit.rb

+      logger_method, syslog_severity = details
+      define_method meth do |msg, msgid, facility: nil, **data|
+        logger.send logger_method, LogMessage.new(msg, msgid, data, syslog_severity, facility)
+      end


Just a suggestion here: Not that this is super cryptic, but I'd still prefer a more straightforward approach of defining the 3 methods explicitly, and then DRYing up repetition by having them delegate their implementation to a private method that does most of the work. Lmk if that isn't clear, but I think you'll know what I mean. Two advantages would be:

Less likely to violate the principle of least surprise: If I see one of these methods being used in client code, I'd expect there to be a def <method_name> somewhere.

For IDEs like rubymine (I don't use it atm, but I sometimes do...) I believe this will break that autolookup / jump to definition type features.

You're probably right, however I'll leave this as is for now.

jonahx · 2018-06-18T15:55:38Z

app/models/audit.rb

    ACTION = conjur_sdid 'action'
  end

  class LogMessage < String


Add a note clarifying the reasons you inheritied from String here, since it's an choice and might confuse:

Yeah, that's a funny thing. I wanted to make it compatible with Ruby's built-in log formatter and it actually checks if a message is a String and uses #inspect if it isn't. I'm not very happy with that solution and I might actually end up ditching the compatibility on some refactor, but that's how it is for now.
So in short it's a workaround for a bug in Logger::Formatter or whatever it's called. Which is also duplicated by ActiveSupport's formatter.

I'll add it, thanks.

jonahx · 2018-06-18T16:07:44Z

app/models/audit.rb

    CONJUR_PEN = 43868
    def self.conjur_sdid label
-      [label, CONJUR_PEN].join('@').freeze
+      [label, CONJUR_PEN].join('@').intern


I'd like to see some more comments about what this SDID is used for.

I'm not sure what you mean. The SD-IDs are documented in audit.md.

After your experience with my code, I now believe that any time something in code needs an explanation, having it documented elsewhere is not enough. Either repeat the explanation directly in the code comments, or include a link to the explanation in the code comments.

Fair :) Could you say more about what do you feel needs explanation?

jonahx · 2018-06-18T16:14:45Z

app/models/audit.rb

+    attr_reader :msgid, :structured_data, :severity, :facility
  end

  # Middleware to store request ID in a thread variable


Confusing to me why this is here as part of audit. If there is a good reason, I'd add a comment explaining it.

There is no specific reason. It'll be moved out at some point.

jonahx · 2018-06-18T16:15:32Z

app/models/audit.rb

      msgid = msg.msgid if msg.respond_to? :msgid
      sd = format_sd sd

+      facility = msg.try(:facility) || 4


I'd move the whole RFC5424Formatter class to its own file, as it stands independently.

Yeah, I'll probably do that eventually.

jonahx · 2018-06-18T16:20:39Z

app/models/audit/event/authn.rb

+    end
+  end
+
+  SUCCESS_TEMPLATE = "%s successfully authenticated with authenticator %s service %s".freeze


I like to put constants at top per ruby style guide

If they'd been meant for external use, I would. But since these are in a way only local, I like to keep them near the code that uses them.

jonahx · 2018-06-18T17:12:08Z

app/models/audit/event/authn.rb

@@ -0,0 +1,53 @@
+require 'ostruct'
+
+class Audit::Event::Authn < OpenStruct


Currently the interface for this class is only implicit: you have to read through every method to figure out what attribute assignments must be made before it can be used. It must be made explicit one way or the other. My preference would be:

Write a normal intializer which accepts the member variables. This has the advantage of not only being explicit, but also making the client code interface clear and consistent. There's really no reason anyone should be using attr setters with this object. Also, you can enforce immutability.

If you don't want to do 1. right now, please at least document the intended interface in comments.

Good point, I'll make it clearer.

jonahx · 2018-06-18T17:16:12Z

app/models/audit/event/authn.rb

+  end
+
+  def service_id
+    service.try :id


I'd prefer not to rely on Rails methods in our domain code. I'd like to see a require active_support if you want to keep it (just to make that dep clear) or else service.id rescue nil

I understand where you're coming from, but I like the expressiveness of some of ActiveSupport utilities and this code is unlikely to be moved out.

jonahx · 2018-06-18T17:22:46Z

app/models/audit/event/authn.rb

+require 'ostruct'
+
+class Audit::Event::Authn < OpenStruct
+  def emit_success


You don't need to fix this, but there's something confusing to me about the naming. we have an "event" emitting a "success" (a "success event"). so an event is emitting an event, which reads oddly.

There's also a kind of strange inversion in the design, in that this object delegates to the thing that actually writes the log. Ie, we have an Event with an Audit (the actually log writer) inside of it, and the event calls its own emit_success which then calls Audit, passing the events own structured data to Audit.info.

I think a simpler, clearer design would be to just have pass events to Audit's writer methods, and allow client code to use Audit. But that's a larger (though not very large) refactor -- just food for thought.

Spent some time thinking about it actually. Maybe it'll get refactored at some point.

jonahx · 2018-06-18T17:26:06Z

app/models/resource.rb

    end
+
+    # Specialization to allow lookup by composite ids,
+    # eg. Resource[account, kind, id]


Just a note for later (not specific to your code), but just to make you aware: we have similar logic to this in multiple places in our code base (and others I believe) including my rotator code. At some point we should DRY it all up.

Agreed. That's why I added this method in lieu of copying the same functionality again, even though I didn't go the one step further and found all the places where it could be used.

I don't understand you what you mean, this does copy the same functionality again. It can't be reused beause here it exists in a very specific context (inside an audit event object), whereas in the codebase in general it's used in many different contexts. I'm not saying this because I want you to change anything atm, just want to clarify my initial point.

What prevents this method from being used in all the other different contexts?

kgilpin · 2018-06-19T10:17:59Z

Some of your answers to the questions in the PR really seem like they would be very helpful as comments in the code. For example, why you inherit from String.

…

On Tue, Jun 19, 2018 at 5:50 AM Rafał Rzepecki ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In app/models/resource.rb <#571 (comment)>: > @@ -52,6 +52,12 @@ def find_if_visible role, *a res = find *a res if res.try :visible_to?, role end + + # Specialization to allow lookup by composite ids, + # eg. Resource[account, kind, id] Agreed. That's why I added this method in lieu of copying the same functionality again, even though I didn't go the one step further and found all the places where it could be used. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#571 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFRexaFrSfjEtZyG5VsWikfiS5Uw8nEks5t-MlQgaJpZM4UqSSP> .

dividedmind · 2018-06-19T10:36:05Z

Listen @jonahx , I spent all night trying to get to a working solution that would make us both happy, but I spent way too much time on it already and am over any reasonable timebox. Let's leave this as is and please file a tech debt issue if you want.

Not that the time has been wasted, as I added a lot of test code and noticed and fixed a couple bugs on the way, which is why this PR grown a little bit; you might want to give it another look-over. I tried to address some of your concerns (however some are outside the scope of this change).

The commits are clean so you can go through them one-by-one, and also if you're going to merge do either rebase merge or straight merge.

For the record here are WIPs of two approached I tried:

jvanderhoof · 2018-06-19T15:24:50Z

@dividedmind: I like the approach you and Jonah have taken on this one. I'd ask that you please file issues for work that we reasonably plan to address in the future. As the implementor and visionary of this work, you're in the best place to understand the work that is being pushed out to a future date.

jonahx

I am effectively approving this unreviewed.

It breaks the authentication design per our previous conversation, and my previous WIP with a an unfinished suggestion for how to integrate into the existing design:
f1387b6.

In addition, many other places have what amount to undone TODOs, and I didn't have time to do a thorough review of the test code.

I'm just noting all this for the record. We can address it all later.

jonahx · 2018-06-20T15:34:20Z

app/models/audit.rb

    CONJUR_PEN = 43868
    def self.conjur_sdid label
-      [label, CONJUR_PEN].join('@').freeze
+      [label, CONJUR_PEN].join('@').intern


After your experience with my code, I now believe that any time something in code needs an explanation, having it documented elsewhere is not enough. Either repeat the explanation directly in the code comments, or include a link to the explanation in the code comments.

jonahx · 2018-06-20T15:41:00Z

app/models/resource.rb

    end
+
+    # Specialization to allow lookup by composite ids,
+    # eg. Resource[account, kind, id]


I don't understand you what you mean, this does copy the same functionality again. It can't be reused beause here it exists in a very specific context (inside an audit event object), whereas in the codebase in general it's used in many different contexts. I'm not saying this because I want you to change anything atm, just want to clarify my initial point.

dividedmind · 2018-06-20T19:26:00Z

Since somebody pointed out to me that it might not be entirely clear from this thread and the rebased history, I'll note that (except for the central disagreement) I addressed most of the comments that pertain to the changed code and responded to the rest. I've also addressed some comments on parts of code that didn't change, and those of these which I haven't addressed I've responded to and acknowledged, and they're good points but addressing them would require refactoring beyond the scope of this PR and we're trying not to 'big batch', however the refactoring is likely to happen at some point in the future when the code in question is touched again. (In fact, I am currently changing that code doing other work, so this is most probably a closer rather than farther point in the future.)

If I did miss something and the above comment is not accurate, please do tell me :)

dividedmind · 2018-06-20T19:37:51Z

security-scan.sh

Thank you! I was just about to add this :)

No worries. I think we need to start building our own utility containers for stuff like this....

I don't think it's worth the effort for one-tool containers like that. I do think, however, that we ought to start pinning known good versions. Even better, actually review the images to make sure there are no backdoors or surprises of some sort, and then pin by specific SHA that we reviewed.

dividedmind · 2018-06-20T19:44:27Z

I need to take a look at why are those new cukes failing in Jenkins. Stay tuned.

dividedmind · 2018-06-20T19:45:35Z

(I'll go ahead and merge after I fix that, if there aren't any objections.)

Note, currently this only works when running cukes locally.

CNJR-5196 update REXML to mitigate CVE-2024-49761

ghost assigned dividedmind Jun 16, 2018

ghost added in progress labels Jun 16, 2018

dividedmind removed the in progress label Jun 18, 2018

jonahx reviewed Jun 18, 2018

View reviewed changes

dividedmind force-pushed the authenticate-events branch from 8827ec2 to 20c5f96 Compare June 19, 2018 09:58

jonahx approved these changes Jun 20, 2018

View reviewed changes

dividedmind force-pushed the authenticate-events branch 2 times, most recently from 2cb604b to c9ada67 Compare June 20, 2018 17:23

ghost assigned jvanderhoof Jun 20, 2018

dividedmind commented Jun 20, 2018

View reviewed changes

dividedmind unassigned jvanderhoof Jun 20, 2018

dividedmind force-pushed the authenticate-events branch from 9af4cbb to 8f75532 Compare June 21, 2018 09:18

dividedmind added 11 commits June 21, 2018 12:17

Automatically run only focused specs

a34b016

Allow or and and postfix usage

a3a6e33

Change simplecov config to not balk on missing env

ad90ca6

Add Role.by_login finder

199c0fe

Add the ability to lookup Resource by composite id

baef779

Authn audit events

4dd2db3

Allow specifying facility when sending audit message

0c8b2a3

Add some clarifying comments to audit code

909cb7c

Add audit steps and support code for cuke testing

19c4fb7

Note, currently this only works when running cukes locally.

Don't assume the app is run from the project root

b121067

Emit authentication events

074156a

dividedmind force-pushed the authenticate-events branch from 8f75532 to 074156a Compare June 21, 2018 10:28

dividedmind merged commit 7113362 into master Jun 21, 2018

ghost removed the review label Jun 21, 2018

dividedmind deleted the authenticate-events branch June 21, 2018 10:54

conjur-jenkins pushed a commit that referenced this pull request Nov 22, 2024

Merge pull request #571 from Conjur-Enterprise/CVE-2024-49761

29cdca8

CNJR-5196 update REXML to mitigate CVE-2024-49761

		@@ -0,0 +1,53 @@
		require 'ostruct'

		class Audit::Event::Authn < OpenStruct

Authenticate audit events #571

Authenticate audit events #571

Uh oh!

Conversation

dividedmind commented Jun 16, 2018

Uh oh!

jonahx left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kgilpin commented Jun 19, 2018 via email

Uh oh!

dividedmind commented Jun 19, 2018

Uh oh!

jvanderhoof commented Jun 19, 2018

Uh oh!

jonahx left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dividedmind commented Jun 20, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jonahx left a comment •

edited

Loading

dividedmind commented Jun 20, 2018 •

edited

Loading

dividedmind commented Jun 20, 2018 •

edited

Loading