Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

james-elastic
Copy link
Contributor

@james-elastic james-elastic commented Jun 30, 2021

Enhancement

What does this PR do?

Update docs for osquery_manager for 7.14

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@elasticmachine
Copy link

elasticmachine commented Jun 30, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-07-06T20:22:10.394+0000

  • Duration: 12 min 56 sec

  • Commit: 70264f6

Test stats 🧪

Test Results
Failed 0
Passed 1
Skipped 0
Total 1

Trends 🧪

Image of Build Times

Image of Tests

Copy link
Contributor

@melissaburpo melissaburpo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found a couple points we should check with the team, noted in the review comments, and one small typo

4. (optional) Select a saved query to run.
5. Click **Submit**.
6. Monitor the status and results of your request under the **Check results** section. Depending on the number of agents queried, this request may take some time. The status area is updated as query results are returned.
7. To view the query results and data, click **Results**.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the chat we just had with @patrykkopycinski, we might need to update steps 6 and 7. It sounds like results are shown right away now.

Patryk - can you confirm, would this revision make sense following the update you've made?

(after step 5)

  1. Review the results. Depending on the number of agents queried, this request may take some time. Status info is updated as available.
  2. (optional) Click the Status tab to view more info about the request, for example, if there are any failures.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched it!

- Platform: Sets the operating system required to run the query.


### Query statuses
Copy link
Contributor

@melissaburpo melissaburpo Jun 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lykkin / @patrykkopycinski - do we need to revise the statuses we list? We currently list Successful, Failed, and Not yet responded. I just remembered there may be a new Expired status (or something similar), for query requests that timed out?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah Expired is now a status, good catch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

- The `host.*` and `agent.*` fields are mapped to ECS.
- The `action_data.query` has the query that was sent.
- All query results are [snapshot logs](https://osquery.readthedocs.io/en/stable/deployment/logging/#snapshot-logs) that represent a point in time set of results, with no differentials. [Differential logs](https://osquery.readthedocs.io/en/stable/deployment/logging/#differential-logs) are not supported.
Osquery data is stored in the `logs-osquery_manager.result-default` datastream, and the result row data is under the `osquery` property in the document.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we missed a bullet point here, just need to add a -. In the rendered view, row 91 is included with the bullet on row 90.

  • Osquery data is stored...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Copy link
Contributor

@melissaburpo melissaburpo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, thanks @james-elastic!

To run a live query:


1. From Kibana, go to Management > Osquery.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend telling users to open the main menu (the left sidebar) to get to Osquery. For example: From Kibana, open the main menu and click Management > Osquery.

2. Click the **New live query** button.
3. Select the agents or groups you want to query. You can select one or more.
4. Enter a SQL query. The query field provides intellisense suggestions based on the Osquery schema.
2. Click the New live query button.
Copy link

@nastasha-solomon nastasha-solomon Jun 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might help orient the user if you also include the page name. For example: On the Live queries history page, click the New live query button. Another option is to include a line like "The Live queries history page displays" at the end of the first step.

3. Select the agents or groups you want to query. You can select one or more.
4. Enter a SQL query. The query field provides intellisense suggestions based on the Osquery schema.
2. Click the New live query button.
3. Select the agents or groups you want to query. You can select one or more. Note that agents with a green dot indicate that the agent is online, based on a check that runs every 5 minutes.
Copy link

@nastasha-solomon nastasha-solomon Jun 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be able to combine the first two sentences in this step to something like: Select one or more agents or groups to query.

Also recommend explaining how offline agents are represented and whether they can be queried. For example, do they have a red dot next to them or are they not marked at all? And are results returned after users submit their query if they selected an agent that's offline?

4. Enter a SQL query. The query field provides intellisense suggestions based on the Osquery schema.
2. Click the New live query button.
3. Select the agents or groups you want to query. You can select one or more. Note that agents with a green dot indicate that the agent is online, based on a check that runs every 5 minutes.
4. (optional) Select a saved query to run.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend capitalizing the "o" in "optional" since it's at the start of this sentence. Also, are you saying here that it's optional to select a saved query before submitting the new live query, or are you saying that users can create a brand new query (instead of selecting a saved one) and use that when they submit the new live query?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're saying the The user can optional to select a saved query before submitting the new live query

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, ok - I did not do a good job of fully explaining what confused me about this step, so retry that here :)

If I'm understanding the workflow correctly, users have two options at this point: they can either select a saved query to run or enter their own into the textbox under the Build from a saved query (optional) field. Regardless of the option they choose, they do have to enter and submit a query in order to successfully complete the workflow. The way this step is currently written, it could be interpreted as "users have the option to enter a query or to pass on it" and that doesn't make sense because the whole point of this task is to create a new live query.

So, my suggestion here would be to remove the "optional" text at the start of the step and to expand the step's content to include the other option users can choose--which is entering a query in the box beneath the Build from a saved query (optional) field.

Hopefully that makes more sense, but of course, feel free ping me if it doesn't. :)

5. Click **Submit**.
6. Monitor the status and results of your request under the **Check results** section. Depending on the number of agents queried, this request may take some time. The status area is updated as query results are returned.
7. To view the query results and data, click **Results**.
6. Review the results. Depending on the number of agents queried, this request may take some time. Status info is updated as available.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the status area the Check results area on the New live query page? If it is, you might be able to combine the last sentence with the first. For example:
View the status and results of your request under the Check results section. Queries with multiple agents or groups might take longer to return results.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears on the same page under the live query form; example (gif) -
content-order-reorder

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha - and I see that too from my end :) Upon second thought, it might not be necessary to direct the user towards the Check results section. I'm fairly certain that it's safe to assume users won't be navigating away from this page/form or to different tabs as they're waiting for results to return.

6. Monitor the status and results of your request under the **Check results** section. Depending on the number of agents queried, this request may take some time. The status area is updated as query results are returned.
7. To view the query results and data, click **Results**.
6. Review the results. Depending on the number of agents queried, this request may take some time. Status info is updated as available.
7. (optional) Click the **Status** tab to view more info about the request, for example, if there are any failures.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend capitalizing the "o" in "optional".

7. (optional) Click the **Status** tab to view more info about the request, for example, if there are any failures.


> Note: If an agent is offline, the request Status remains in **pending** as we retry the request. By default, a query request times out after 5 minutes. Note that this time out applies to the time it takes to deliver the action request to an agent to run a query. If the action completes after the timeout period, the results are still returned.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of notes here:

  • Believe the "s" in status should be lowercase.
  • In the first sentence, is "we" the osquery app? If it is, I'd recommend updating it to reflect that--currently, the sentence could be interpreted as "If an agent is offline, the request status remains in pending as Elastic retries the request".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Switched Elastic -> Kibana

After selecting a scheduled query group to edit or adding a new scheduled query group:

- *To add queries individually*: Click **Add query**. In the fly-out, enter an ID for the query, the query, and the query interval (seconds).
- *To add queries individually*: Click **Add query**. In the fly-out, enter an ID for the query, the query, and the query interval (seconds). You can also optionally set the minimum osquery version required to run the query and set the Platform required to run the query.
Copy link

@nastasha-solomon nastasha-solomon Jul 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't typically hyphenate "flyout" in the Security docs or capitalize the term "platform" so just two minor changes there. Also, is the Platform field an optional setting? Currently, it looks like the Minimum Osquery version field is the only optional field when creating a new query. If the Platform field is mandatory, I'd recommend adding it to the second sentence (e.g., In the fly-out, enter an ID for the query, type in a query, specify the query interval (seconds), and select one or more platform to run the query.").

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both should be optional @patrykkopycinski can correct me if i'm incorrect

To save your changes, click **Save query**. Once saved, the changes are pushed out to the agents in the policy.

### Saved queries
The Saved queries page lists all queries that have been saved. Queries can be saved when running a live query or from the Saved queries page, and they can be run from the Live queries page or added to a scheduled query group.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There might be some repeated information in the first and second sentences. Could you remove the duplicate details and consolidate the sentences to something like:
The Saved queries page lists all queries that have been saved. From the Live queries page, you can create and save a new live query. From the Scheduled query groups page, you can add saved queries to a scheduled query group.

- Everything prefaced with `osquery.` is part of the query response. Note that these fields are not mapped to ECS.
- The `host.*` and `agent.*` fields are mapped to ECS.
- The `action_data.query` has the query that was sent.
- All query results are [snapshot logs](https://osquery.readthedocs.io/en/stable/deployment/logging/#snapshot-logs) that represent a point in time set of results, with no differentials. [Differential logs](https://osquery.readthedocs.io/en/stable/deployment/logging/#differential-logs) are not supported.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a conjunction (maybe "and"?) might be missing between "time" and "set" in the first sentence, but am not sure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added with a between time and set

- `/data/elastic-agent-054e22/logs/default/osquerybeat-json.log`

To get more details in the logs, you can change the agent logging level to debug:
1. From Kibana, go to Fleet > Agents, then select the agent you want to debug.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor change here - bold Fleet > Agents.


### Exported Fields
This section describes the fields that can be returned in osquery results. Note the following about osquery fields:
- Some fields list multiple descriptions, because the one that applies depends on which table was queried. For example, a result stored in the `osquery.autoupdate` field may represent a response from the `firefox_addons` table or the `windows_security_center` table.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't think you need the comma before "because" in the first sentence.

Copy link

@nastasha-solomon nastasha-solomon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a few suggestions about wording, style, and punctuation, but overall everything looks great!

Copy link
Contributor

@patrykkopycinski patrykkopycinski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@melissaburpo
Copy link
Contributor

@james-elastic - I just re-reviewed with the latest changes, and it LGTM

@james-elastic james-elastic merged commit 3586ae1 into elastic:master Jul 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants