It's a little utility to print logs of pods in an EKS cluster (Amazon Web Services). The logs have to be synchronised from cloudwatch first, there is no interaction with kubernetes API.

I want to keep it as is, and don't want to make a generic utility to print logs of cloudwatch. The goal is to get the logs of pods that have been written in cloudwatch by fluentd.

Here are some documentation to setup fluentd :

• https://docs.aws.amazon.com/fr_fr/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs.html
• https://aws.amazon.com/fr/blogs/containers/fluent-bit-integration-in-cloudwatch-container-insights-for-eks/

The log events are retrieved from the loggroup named /aws/containerinsights/Name of your cluster/application.

The initial development has been done in quick and dirty mode. Maybe, this problem will be adressed in the future but it's a side project with very very low priority so don't expect a lot of features or improvements.

$ ekspodlogs
Tool to parse logs of applications in an EKS cluster from AWS Cloudwatch

First, you need to configure your AWS credentials with the AWS CLI.
Then, you will have to synchronise the local database with the logs of cloudwatch for a period.

Finally, you will be able to request the logs of a specific logstream for a period.

Usage:
  ekspodlogs [flags]
  ekspodlogs [command]

Available Commands:
  help        Help about any command
  list-groups list-groups lists the log groups
  purge       Purge the local database
  req         requests the local database
  sync        synchronise the local database with the logs of cloudwatch
  version     print version of gitlab-expiration-token

Flags:
  -h, --help   help for ekspodlogs

Use "ekspodlogs [command] --help" for more information about a command.

Option -p should be used to login to AWS API when you have an SSO configured. It is the name of the profile to use.

$ grep profile  ~/.aws/config 
[profile dev]
[profile prod]
$ aws sso login --profile dev
...
$ ekspodlogs list-groups -p dev
...

If there are multiples EKS clusters, you have to specify the name of the log group with -g option.

The -g option is optionnal, if you have only one loggroup named /aws/containerinsights/Name of your cluster/application, no need to specify it.

Start date and end date allow to select logs that happened in this range of time. Option -n allow to filter to the name of the pod which appears in the name of log stream.

List loggroups if needed :

$ aws sso login --profile dev
...
$ ekspodlogs list-groups -p dev
...

Synchronise the local database with the logs of cloudwatch :

$ ekspodlogs sync -p dev -n mypodname -b "2021-01-01 00:00:00" -e "2021-01-01 23:59:59"
...

Request the logs of a specific logstream for a period :

$ ekspodlogs req -p dev -n mypodname -b "2021-01-01 00:00:00" -e "2021-01-01 23:59:59"
...

The req command provides several formatting options:

Colorized Output (default):

$ ekspodlogs req -p dev -n mypodname -b "2021-01-01 00:00:00" -e "2021-01-01 23:59:59"
# Displays logs with color-coded log levels:
# - INFO/INFORMATION: Blue
# - WARN/WARNING: Yellow  
# - ERROR/FATAL/CRITICAL: Red

Show Container Names:

$ ekspodlogs req -c -p dev -n mypodname -b "2021-01-01 00:00:00" -e "2021-01-01 23:59:59"
# Adds container name column to output

Disable Colors:

$ ekspodlogs req --no-color -p dev -n mypodname -b "2021-01-01 00:00:00" -e "2021-01-01 23:59:59"
# Plain text output without colors (useful for scripts or piping)

Combined Options:

$ ekspodlogs req -c --no-color -p dev -n mypodname -b "2021-01-01 00:00:00" -e "2021-01-01 23:59:59"
# Container names without colors

The colorization automatically detects log levels in various formats:

• Bracketed: [INFO], [ERROR], [WARN]
• Colon format: INFO:, ERROR:, WARN:
• Structured: level=info, level=error
• JSON: "level":"info", "level":"error"

sudo apt-get update
sudo apt-get install -y libsqlite3-dev

sudo dnf install -y sqlite-devel

sudo pacman -Syu sqlite

Set env variable DEBUGLEVEL to one of this value :

• error
• warn
• info (default)
• debug

A log group contains the log streams. A log stream is a sequence of log events that share the same source. Each log stream has a unique name within its log group.