Update docs for osquery_manager for 7.14 #1281

james-elastic · 2021-06-30T15:48:50Z

Enhancement

What does this PR do?

Update docs for osquery_manager for 7.14

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

Author's Checklist

[ ]

How to test this PR locally

Related issues

Screenshots

Update PR number

elasticmachine · 2021-06-30T15:58:56Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-07-06T20:22:10.394+0000
Duration: 12 min 56 sec
Commit: 70264f6

Test stats 🧪

Test	Results
Failed	0
Passed	1
Skipped	0
Total	1

Trends 🧪

melissaburpo

I found a couple points we should check with the team, noted in the review comments, and one small typo

melissaburpo · 2021-06-30T15:57:14Z

packages/osquery_manager/docs/README.md

+4. (optional) Select a saved query to run.
 5. Click **Submit**.
 6. Monitor the status and results of your request under the **Check results** section. Depending on the number of agents queried, this request may take some time. The status area is updated as query results are returned.
 7. To view the query results and data, click **Results**.


Based on the chat we just had with @patrykkopycinski, we might need to update steps 6 and 7. It sounds like results are shown right away now.

Patryk - can you confirm, would this revision make sense following the update you've made?

(after step 5)

Review the results. Depending on the number of agents queried, this request may take some time. Status info is updated as available.

(optional) Click the Status tab to view more info about the request, for example, if there are any failures.

Switched it!

melissaburpo · 2021-06-30T15:59:51Z

packages/osquery_manager/docs/README.md

+  - Platform: Sets the operating system required to run the query. 
+

 ### Query statuses


@lykkin / @patrykkopycinski - do we need to revise the statuses we list? We currently list Successful, Failed, and Not yet responded. I just remembered there may be a new Expired status (or something similar), for query requests that timed out?

yeah Expired is now a status, good catch

melissaburpo · 2021-06-30T16:01:34Z

packages/osquery_manager/docs/README.md

 - The `host.*` and `agent.*` fields are mapped to ECS.
 - The `action_data.query` has the query that was sent.
+- All query results are [snapshot logs](https://osquery.readthedocs.io/en/stable/deployment/logging/#snapshot-logs) that represent a point in time set of results, with no differentials. [Differential logs](https://osquery.readthedocs.io/en/stable/deployment/logging/#differential-logs) are not supported.
+Osquery data is stored in the `logs-osquery_manager.result-default` datastream, and the result row data is under the `osquery` property in the document.


I think we missed a bullet point here, just need to add a -. In the rendered view, row 91 is included with the bullet on row 90.

Osquery data is stored...

…nto osquery-714

melissaburpo

looks good, thanks @james-elastic!

nastasha-solomon · 2021-06-30T21:03:24Z

packages/osquery_manager/docs/README.md

 To run a live query:

+
 1. From Kibana, go to Management > Osquery.


Recommend telling users to open the main menu (the left sidebar) to get to Osquery. For example: From Kibana, open the main menu and click Management > Osquery.

nastasha-solomon · 2021-06-30T21:05:47Z

packages/osquery_manager/docs/README.md

-2. Click the **New live query** button.
-3. Select the agents or groups you want to query. You can select one or more.
-4. Enter a SQL query. The query field provides intellisense suggestions based on the Osquery schema.
+2. Click the New live query button.


Might help orient the user if you also include the page name. For example: On the Live queries history page, click the New live query button. Another option is to include a line like "The Live queries history page displays" at the end of the first step.

nastasha-solomon · 2021-06-30T21:30:06Z

packages/osquery_manager/docs/README.md

-3. Select the agents or groups you want to query. You can select one or more.
-4. Enter a SQL query. The query field provides intellisense suggestions based on the Osquery schema.
+2. Click the New live query button.
+3. Select the agents or groups you want to query. You can select one or more. Note that agents with a green dot indicate that the agent is online, based on a check that runs every 5 minutes.


Might be able to combine the first two sentences in this step to something like: Select one or more agents or groups to query.

Also recommend explaining how offline agents are represented and whether they can be queried. For example, do they have a red dot next to them or are they not marked at all? And are results returned after users submit their query if they selected an agent that's offline?

nastasha-solomon · 2021-06-30T21:35:12Z

packages/osquery_manager/docs/README.md

-4. Enter a SQL query. The query field provides intellisense suggestions based on the Osquery schema.
+2. Click the New live query button.
+3. Select the agents or groups you want to query. You can select one or more. Note that agents with a green dot indicate that the agent is online, based on a check that runs every 5 minutes.
+4. (optional) Select a saved query to run.


Recommend capitalizing the "o" in "optional" since it's at the start of this sentence. Also, are you saying here that it's optional to select a saved query before submitting the new live query, or are you saying that users can create a brand new query (instead of selecting a saved one) and use that when they submit the new live query?

We're saying the The user can optional to select a saved query before submitting the new live query

Yes, ok - I did not do a good job of fully explaining what confused me about this step, so retry that here :)

If I'm understanding the workflow correctly, users have two options at this point: they can either select a saved query to run or enter their own into the textbox under the Build from a saved query (optional) field. Regardless of the option they choose, they do have to enter and submit a query in order to successfully complete the workflow. The way this step is currently written, it could be interpreted as "users have the option to enter a query or to pass on it" and that doesn't make sense because the whole point of this task is to create a new live query.

So, my suggestion here would be to remove the "optional" text at the start of the step and to expand the step's content to include the other option users can choose--which is entering a query in the box beneath the Build from a saved query (optional) field.

Hopefully that makes more sense, but of course, feel free ping me if it doesn't. :)

nastasha-solomon · 2021-06-30T21:48:50Z

packages/osquery_manager/docs/README.md

 5. Click **Submit**.
-6. Monitor the status and results of your request under the **Check results** section. Depending on the number of agents queried, this request may take some time. The status area is updated as query results are returned.
-7. To view the query results and data, click **Results**.
+6. Review the results. Depending on the number of agents queried, this request may take some time. Status info is updated as available.


Is the status area the Check results area on the New live query page? If it is, you might be able to combine the last sentence with the first. For example:
View the status and results of your request under the Check results section. Queries with multiple agents or groups might take longer to return results.

It appears on the same page under the live query form; example (gif) -

Gotcha - and I see that too from my end :) Upon second thought, it might not be necessary to direct the user towards the Check results section. I'm fairly certain that it's safe to assume users won't be navigating away from this page/form or to different tabs as they're waiting for results to return.

nastasha-solomon · 2021-06-30T21:55:44Z

packages/osquery_manager/docs/README.md

-6. Monitor the status and results of your request under the **Check results** section. Depending on the number of agents queried, this request may take some time. The status area is updated as query results are returned.
-7. To view the query results and data, click **Results**.
+6. Review the results. Depending on the number of agents queried, this request may take some time. Status info is updated as available.
+7. (optional) Click the **Status** tab to view more info about the request, for example, if there are any failures.


Recommend capitalizing the "o" in "optional".

nastasha-solomon · 2021-07-06T17:23:50Z

packages/osquery_manager/docs/README.md

+7. (optional) Click the **Status** tab to view more info about the request, for example, if there are any failures.
+
+
+> Note: If an agent is offline, the request Status remains in **pending** as we retry the request.  By default, a query request times out after 5 minutes. Note that this time out applies to the time it takes to deliver the action request to an agent to run a query. If the action completes after the timeout period, the results are still returned.


A couple of notes here:

Believe the "s" in status should be lowercase.

In the first sentence, is "we" the osquery app? If it is, I'd recommend updating it to reflect that--currently, the sentence could be interpreted as "If an agent is offline, the request status remains in pending as Elastic retries the request".

👍 Switched Elastic -> Kibana

nastasha-solomon · 2021-07-06T17:31:55Z

packages/osquery_manager/docs/README.md

 After selecting a scheduled query group to edit or adding a new scheduled query group:

- *To add queries individually*: Click **Add query**. In the fly-out, enter an ID for the query, the query, and the query interval (seconds).
+- *To add queries individually*: Click **Add query**. In the fly-out, enter an ID for the query, the query, and the query interval (seconds). You can also optionally set the minimum osquery version required to run the query and set the Platform required to run the query.


We don't typically hyphenate "flyout" in the Security docs or capitalize the term "platform" so just two minor changes there. Also, is the Platform field an optional setting? Currently, it looks like the Minimum Osquery version field is the only optional field when creating a new query. If the Platform field is mandatory, I'd recommend adding it to the second sentence (e.g., In the fly-out, enter an ID for the query, type in a query, specify the query interval (seconds), and select one or more platform to run the query.").

both should be optional @patrykkopycinski can correct me if i'm incorrect

nastasha-solomon · 2021-07-06T18:00:36Z

packages/osquery_manager/docs/README.md

 To save your changes, click **Save query**. Once saved, the changes are pushed out to the agents in the policy. 

+### Saved queries
+The Saved queries page lists all queries that have been saved. Queries can be saved when running a live query or from the Saved queries page, and they can be run from the Live queries page or added to a scheduled query group.


There might be some repeated information in the first and second sentences. Could you remove the duplicate details and consolidate the sentences to something like:
The Saved queries page lists all queries that have been saved. From the Live queries page, you can create and save a new live query. From the Scheduled query groups page, you can add saved queries to a scheduled query group.

nastasha-solomon · 2021-07-06T18:02:15Z

packages/osquery_manager/docs/README.md

 - Everything prefaced with `osquery.` is part of the query response. Note that these fields are not mapped to ECS.
 - The `host.*` and `agent.*` fields are mapped to ECS.
 - The `action_data.query` has the query that was sent.
+- All query results are [snapshot logs](https://osquery.readthedocs.io/en/stable/deployment/logging/#snapshot-logs) that represent a point in time set of results, with no differentials. [Differential logs](https://osquery.readthedocs.io/en/stable/deployment/logging/#differential-logs) are not supported.


I think a conjunction (maybe "and"?) might be missing between "time" and "set" in the first sentence, but am not sure.

Added with a between time and set

nastasha-solomon · 2021-07-06T18:03:38Z

packages/osquery_manager/docs/README.md

+- `/data/elastic-agent-054e22/logs/default/osquerybeat-json.log`
+
+To get more details in the logs, you can change the agent logging level to debug:
+1. From Kibana, go to Fleet > Agents, then select the agent you want to debug. 


Minor change here - bold Fleet > Agents.

nastasha-solomon · 2021-07-06T18:06:08Z

packages/osquery_manager/docs/README.md


 ### Exported Fields
+This section describes the fields that can be returned in osquery results. Note the following about osquery fields:
+- Some fields list multiple descriptions, because the one that applies depends on which table was queried. For example, a result stored in the `osquery.autoupdate` field may represent a response from the `firefox_addons` table or the `windows_security_center` table. 


Don't think you need the comma before "because" in the first sentence.

nastasha-solomon

I made a few suggestions about wording, style, and punctuation, but overall everything looks great!

patrykkopycinski

LGTM

melissaburpo · 2021-07-07T15:48:52Z

@james-elastic - I just re-reviewed with the latest changes, and it LGTM

james-elastic and others added 2 commits June 30, 2021 11:48

Update docs for osquery_manager for 7.14

0649206

Update changelog.yml

13c1847

Update PR number

melissaburpo reviewed Jun 30, 2021

View reviewed changes

melissaburpo requested review from nastasha-solomon and jmikell821 June 30, 2021 16:07

james-elastic added 3 commits June 30, 2021 13:42

Update working

8ef797b

Merge branch 'osquery-714' of github.com:james-elastic/integrations i…

6ce74e6

…nto osquery-714

Add expired

f227dfd

melissaburpo approved these changes Jun 30, 2021

View reviewed changes

nastasha-solomon reviewed Jun 30, 2021

View reviewed changes

nastasha-solomon reviewed Jul 6, 2021

View reviewed changes

Update with suggestions

70264f6

patrykkopycinski approved these changes Jul 7, 2021

View reviewed changes

james-elastic merged commit 3586ae1 into elastic:master Jul 7, 2021

		- Platform: Sets the operating system required to run the query.


		### Query statuses

		To run a live query:


		1. From Kibana, go to Management > Osquery.

		7. (optional) Click the Status tab to view more info about the request, for example, if there are any failures.


		> Note: If an agent is offline, the request Status remains in pending as we retry the request. By default, a query request times out after 5 minutes. Note that this time out applies to the time it takes to deliver the action request to an agent to run a query. If the action completes after the timeout period, the results are still returned.

Update docs for osquery_manager for 7.14 #1281

Update docs for osquery_manager for 7.14 #1281

Uh oh!

Conversation

james-elastic commented Jun 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

elasticmachine commented Jun 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

Uh oh!

melissaburpo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

melissaburpo Jun 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

melissaburpo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nastasha-solomon Jun 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nastasha-solomon Jun 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nastasha-solomon Jul 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

james-elastic commented Jun 30, 2021 •

edited

Loading

elasticmachine commented Jun 30, 2021 •

edited

Loading

melissaburpo Jun 30, 2021 •

edited

Loading

nastasha-solomon Jun 30, 2021 •

edited

Loading

nastasha-solomon Jun 30, 2021 •

edited

Loading

nastasha-solomon Jul 6, 2021 •

edited

Loading