High-performance log viewer and processor that transforms logs in JSON and logfmt formats into a human-readable output. Built with efficiency in mind, it enables quick parsing and analysis of large log files with minimal overhead.

• Automatic Pager Integration: Automatically integrates with a pager for enhanced convenience, defaulting to less if available, but fully supporting any compatible pager.
• Log Streaming Mode: Enable log streaming with the -P flag, which disables the pager.
• Field-Based Filtering: Filter log records by key/value pairs using the -f option, with support for hierarchical keys.
• Level Filtering: Easily filter logs by level with the -l option.
• Timestamp Range Filtering: Filter logs by timestamp range using the --since and --until options with intuitive formats:
  • RFC-3339 timestamp format.
  • Current configured timestamp output format (via the -t option or environment variable).
  • User-friendly shortcuts like today, yesterday, friday, or relative offsets such as -3h or -14d.
• Field Visibility Control: Quickly hide or reveal specific fields using the -h option.
• Empty Field Hiding: Automatically hide empty fields with the -e flag.
• High-Speed Message Sorting: Achieve lightning-fast message sorting with automatic indexing via the -s flag.
  • Performs the initial scan at approximately 2 GiB/s, enabling rapid filtering by timestamp and level without re-scanning.
  • Efficiently handles hundreds of local files totaling hundreds of gigabytes.
  • Reindexes large, growing files at speeds up to roughly 10 GiB/s by skipping unmodified blocks.
• Live Follow Mode: Use the -F flag for live, timestamp-sorted message updates across multiple sources, with a preview of recent messages via the --tail option.
• Complex Query Support: Construct custom queries with logical conditions (AND/OR) and additional advanced filtering options.
• Non-JSON Prefix Handling: Process logs with non-JSON prefixes using the --allow-prefix flag.
• Timezone Flexibility: Displays timestamps in UTC by default while allowing effortless timezone switching with the -Z option or local timezone adjustments using the -L flag.
• Customizability and Themes: Fully customizable through configuration files and environment variables, with support for easy theme switching and custom themes.

• See performance section for more details.

• Install using homebrew on macOS

  brew install hl

• Download and extract using curl and tar on Linux (x86_64)

  curl -sSfL https://github.com/pamburus/hl/releases/latest/download/hl-linux-x86_64-musl.tar.gz | tar xz

• Install hl package from Arch Linux extra repository

  pacman -S hl

• Install from Scoop

  scoop bucket add pamburus https://github.com/pamburus/scoop-bucket.git
  scoop install hl

• Download latest release from download page

• Run using nix

  nix run github:pamburus/hl

  or binary package

  nix run github:pamburus/hl#bin

  or install with nix profile:

  nix profile add github:pamburus/hl

  or binary package

  nix profile add github:pamburus/hl#bin

• Install the package from source using nix-flakes

  Example how to update nix configuration

  {
    inputs = {
      nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05";
      hl.url = "github:pamburus/hl";
    };
    outputs = {nixpkgs, hl, ...}:
    let
      system = "x86_64-linux";
    in
    {
      # this is just an example!
      nixosConfigurations.yourHost = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [
          ({...}: {
            environment.systemPackages = [
              hl.packages.${system}
            ];
          })
        ];
      };
    };
  }

• Install the package with pre-built binaries using nix-flakes

  Example how to update nix configuration

  {
    inputs = {
      nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05";
      hl.url = "github:pamburus/hl";
    };
    outputs = {nixpkgs, hl, ...}:
    let
      system = "x86_64-linux";
    in
    {
      # this is just an example!
      nixosConfigurations.yourHost = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [
          ({...}: {
            environment.systemPackages = [
              hl.packages.${system}.bin
            ];
          })
        ];
      };
    };
  }

See other screenshots

• Concatenate all log files

  Command

  hl *.log

  Concatenates and displays all *.log files found in the current directory.

• Concatenate all log files including compressed log files

  Command

  hl $(ls -tr /var/log/example/*.{log,log.gz,log.zst,s})

  Concatenates and displays all *.log, *.log.gz, *.log.zst and *.s (will detect compression) files found in /var/log/example/.

• Use the default pager with the default parameters

  Command

  hl example.log

  Automatically opens less pager with the default parameters.

• Override options for default pager

  Command

  LESS=-SR hl example.log

  Opens less pager with disabled line wrapping.

• Use custom pager

  Command

  PAGER="most -w" hl example.log

  Opens most pager with -w option.

• Errors only

  Command

  hl -l e

  Displays only error log level messages.

• Errors and warnings

  Command

  hl -l w

  Displays only warning and error log level messages.

• Errors, warnings and informational

  Command

  hl -l i

  Displays all log messages except debug level messages.

• Command

  tail -f example.log | hl -P

  Tracks changes in the example.log file and displays them immediately. Flag -P disables automatic using of pager in this case.

• Command

  hl example.log --filter component=tsdb

  Displays only messages where the component field has the value tsdb.

• Command

  hl example.log -f component!=tsdb -f component!=uninteresting

  Displays only messages where the component field has a value other than tsdb or uninteresting.

• Command

  hl example.log -f provider~=string

  Displays only messages where the provider field contains the string sub-string.

• Command

  hl example.log -f 'provider!~=string'

  Displays only messages where the provider field does not contain the string sub-string.

• Command

  hl example.log -f 'request.method!=GET'

  Displays only messages where the request.method field is not equal to GET. In JSON messages, this also matches composite fields, e.g., {"request":{"method":"POST"}}.

• Command

  hl example.log -f 'span.[].name=sp0001'

  Displays only messages where the span field is an array that contains an object with a name field equal to sp0001.

• Command

  hl example.log -f 'span.[1].name=sp0001'

  Displays only messages where the span field is an array with at least two elements, and the element at index 1 (i.e., the second element) is an object with a name field equal to sp0001.

• Command

  hl my-service.log --query 'level > info or status-code >= 400 or duration > 0.5'

  Displays messages that either have a level higher than info (i.e. warning or error) or have a status code field with a numeric value >= 400 or a duration field with a numeric value >= 0.5.

• Command

  hl my-service.log -q '(request in (95c72499d9ec, 9697f7aa134f, bc3451d0ad60)) or (method != GET)'

  Displays all messages that have the 'request' field with one of these values, or the 'method' field with a value other than 'GET'.

• Complete set of supported operators

  • Logical operators
    • Logical conjunction - and, &&
    • Logical disjunction - or, ||
    • Logical negation - not, !
  • Comparison operators
    • Equal - eq, =
    • Not equal - ne, !=
    • Greater than - gt, >
    • Greater or equal - ge, >=
    • Less than - lt, <
    • Less or equal - le, <=
  • String matching operators
    • Sub-string check - (contain, ~=), (not contain, !~=)
    • Wildcard match - (like), (not like)
      • Wildcard characters are: * for zero or more characters and ? for a single character
    • Regular expression match - (match, ~~=), (not match, !~~=)
  • Operators with sets
    • Test if a value is one of the values in a set - in (v1, v2), not in (v1, v2)
    • Test if a value is one of the values in a set loaded from a file - in @filename, not in @filename, assuming that each element is a line in the file, which can be either a simple string or a JSON string
    • Test if a value is one of the values in a set loaded from stdin - in @-, not in @-

• Notes

  • Special field names that are reserved for filtering by predefined fields regardless of the actual source field names used to load the corresponding value: level, message, caller and logger.

  • To address a source field with one of these names instead of predefined fields, add a period before its name, i.e., .level will perform a match against the "level" source field.

  • To address a source field by its exact name, use a JSON-formatted string, i.e. -q '".level" = info'.

  • To specify special characters in field values, also use a JSON-formatted string, i.e.

    hl my-service.log -q 'message contain "Error:\nSomething unexpected happened"'

• Command

  hl example.log --since 'Jun 19 11:22:33' --until yesterday

  Displays only messages that occurred after Jun 19 11:22:33 UTC of the current year (or the previous year if the current date is less than Jun 19 11:22:33) and before yesterday midnight.

• Command

  hl example.log --since -3d

  Displays only messages from the past 72 hours.

• Command

  hl example.log --until '2021-06-01 18:00:00' --local

  Displays only messages that occurred before 6 PM local time on June 1, 2021, and shows timestamps in local time.

• Command

  hl example.log --hide provider

  Hides field provider.

• Command

  hl example.log --hide '*' --hide '!provider'

  Hides all fields except provider.

• Command

  hl example.log -h headers -h body -h '!headers.content-type'

  Hides fields headers and body but shows a single sub-field content-type inside field headers.

• Command

  hl -s *.log

  Displays log messages from all log files in the current directory sorted in chronological order.

• Command

  hl --sync-interval-ms 500 -F <(kubectl logs -l app=my-app-1 -f) <(kubectl logs -l app=my-app-2 -f)

  Runs without a pager in follow mode by merging messages from the outputs of these 2 commands and sorting them chronologically within a custom 500ms interval.

• Command

  hl -F --tail 100 app1.log app2.log app3.log

  Runs without a pager in follow mode, following the changes in three log files in the current directory and sorting them chronologically at a default interval of 100ms. Preloads 100 lines from the end of each file before filtering.

• Configuration files are automatically loaded if found in predefined platform-specific locations.

  OS	System-Wide Location	User Profile Location
  macOS	/etc/hl/config.{yaml,toml,json}	~/.config/hl/config.{yaml,toml,json}
  Linux	/etc/hl/config.{yaml,toml,json}	~/.config/hl/config.{yaml,toml,json}
  Windows	%PROGRAMDATA%\hl\config.{yaml,toml,json}	%USERPROFILE%\AppData\Roaming\hl\config.{yaml,toml,json}

• The path to the configuration file can be overridden using the HL_CONFIG environment variable or the --config command-line option.

  The order in which the configuration files are searched and loaded is as follows:

  1. The system-wide location.
  2. The user profile location.
  3. The location specified by the HL_CONFIG environment variable (unless the --config option is used).
  4. The locations specified by the --config option (can be specified multiple times).

  If a configuration file is found in multiple locations, the file in each subsequent location overrides only the parameters it contains.

  If HL_CONFIG or --config specifies - or an empty string, all default locations and any locations specified by previous --config options are discarded. The search for the configuration file locations starts over.

  To disable loading of configuration files and use the built-in defaults, --config - can be used.

• All parameters in the configuration file are optional and can be omitted. In this case, default values are used.

• config.yaml

• Many parameters that are defined in command line arguments and configuration files can also be specified by environment variables.

• Configuration file
• Environment variables
• Command-line arguments

• HL_TIME_FORMAT='%y-%m-%d %T.%3N' overrides the time format specified in the configuration file.
• HL_TIME_ZONE=Europe/Berlin overrides the time zone specified in the configuration file.
• HL_CONCURRENCY=4 overrides the concurrency limit specified in the configuration file.
• HL_PAGING=never specifies the default value for the paging option, but it can be overridden by command line arguments.

• themes

• Using theme value in the configuration file.
• Using environment variable, i.e. HL_THEME=classic, overrides the value specified in configuration file.
• Using command-line argument, i.e. --theme classic, overrides all other values.

To select themes with preview fzf tool can be used like this:

hl --list-themes | fzf --color='bg+:23,gutter:-1,pointer:210' --highlight-line --preview-window 'right,border-left,88%,<142(up,88%,border-bottom)' --preview="hl -t '%b %d %T' --input-info minimal -c --theme {} sample/*.log"

• Custom themes are automatically loaded when found in a predefined platform-specific location.

  OS	Location
  macOS	~/.config/hl/themes/*.{yaml,toml,json}
  Linux	~/.config/hl/themes/*.{yaml,toml,json}
  Windows	%USERPROFILE%\AppData\Roaming\hl\themes*.{yaml,toml,json}

• Format description

  • Section elements contains styles for predefined elements.

  • Section levels contains optional overrides for styles defined in elements sections per logging level, which are [trace, debug, info, warning, error].

  • Each element style contains optional background, foreground and modes parameters.

  • Example

    elements:
        <element>:
            foreground: <color>
            background: <color>
            modes: [<mode>, <mode>, ...]
    levels:
        <level>:
            <element>:
                foreground: <color>
                background: <color>
                modes: [<mode>, <mode>, ...]

  • Color format is one of

    • Keyword default specifies default color defined by the terminal.
    • ASCII basic color name, one of
      • black
      • red
      • green
      • yellow
      • blue
      • magenta
      • cyan
      • white
      • bright-black
      • bright-red
      • bright-green
      • bright-yellow
      • bright-blue
      • bright-magenta
      • bright-cyan
      • bright-white
    • 256-color palette code, from 0 to 255.
    • RGB color in hex web color format, i.e. #FFFF00 for bright yellow color.

  • Modes is a list of additional styles, each of them is one of

    • bold
    • faint
    • italic
    • underline
    • slow-blink
    • rapid-blink
    • reverse
    • conceal
    • crossed-out

• One Dark Neo
• Built-in "Light Background" color scheme

• One Dark Neo
  • Note: It is recommended to use draw_bold_text_with_bright_colors: true setting
• Light
  • Note: It is recommended to use draw_bold_text_with_bright_colors: false setting

JSON and logfmt log converter to human readable representation

Usage: hl [OPTIONS] [FILE]...

Arguments:
  [FILE]...  Files to process

Options:
      --config <FILE>                    Configuration file path [env: HL_CONFIG=]
  -s, --sort                             Sort messages chronologically
  -F, --follow                           Follow input streams and sort messages chronologically during time frame set by --sync-interval-ms option
      --tail <N>                         Number of last messages to preload from each file in --follow mode [default: 10]
      --sync-interval-ms <MILLISECONDS>  Synchronization interval for live streaming mode enabled by --follow option [default: 100]
      --paging <WHEN>                    Control pager usage (HL_PAGER or PAGER) [env: HL_PAGING=] [default: auto] [possible values: auto, always, never]
  -P                                     Handful alias for --paging=never, overrides --paging option
      --help                             Print help
  -V, --version                          Print version

Filtering Options:
  -l, --level <LEVEL>    Filter messages by level [env: HL_LEVEL=]
      --since <TIME>     Filter messages by timestamp >= <TIME> (--time-zone and --local options are honored)
      --until <TIME>     Filter messages by timestamp <= <TIME> (--time-zone and --local options are honored)
  -f, --filter <FILTER>  Filter messages by field values [k=v, k~=v, k~~=v, 'k!=v', 'k!~=v', 'k!~~=v'] where ~ does substring match and ~~ does regular expression match
  -q, --query <QUERY>    Filter using query, accepts expressions from --filter and supports '(', ')', 'and', 'or', 'not', 'in', 'contain', 'like', '<', '>', '<=', '>=', etc

Output Options:
      --color [<WHEN>]        Color output control [env: HL_COLOR=] [default: auto] [possible values: auto, always, never]
  -c                          Handful alias for --color=always, overrides --color option
      --theme <THEME>         Color theme [env: HL_THEME=] [default: uni]
  -r, --raw                   Output raw source messages instead of formatted messages, which can be useful for applying filters and saving results in their original format
      --no-raw                Disable raw source messages output, overrides --raw option
      --raw-fields            Output field values as is, without unescaping or prettifying
  -h, --hide <KEY>            Hide or reveal fields with the specified keys, prefix with ! to reveal, specify '!*' to reveal all
      --flatten <WHEN>        Whether to flatten objects [env: HL_FLATTEN=] [default: always] [possible values: never, always]
  -t, --time-format <FORMAT>  Time format, see https://man7.org/linux/man-pages/man1/date.1.html [env: HL_TIME_FORMAT=] [default: "%b %d %T.%3N"]
  -Z, --time-zone <TZ>        Time zone name, see column "TZ identifier" at https://en.wikipedia.org/wiki/List_of_tz_database_time_zones [env: HL_TIME_ZONE=] [default: UTC]
  -L, --local                 Use local time zone, overrides --time-zone option
      --no-local              Disable local time zone, overrides --local option
  -e, --hide-empty-fields     Hide empty fields, applies for null, string, object and array fields only [env: HL_HIDE_EMPTY_FIELDS=]
  -E, --show-empty-fields     Show empty fields, overrides --hide-empty-fields option [env: HL_SHOW_EMPTY_FIELDS=]
      --input-info <LAYOUTS>  Input number and filename layouts [default: auto] [possible values: auto, none, minimal, compact, full]
      --ascii [<WHEN>]        Controls whether to restrict punctuation to ASCII characters only [env: HL_ASCII=] [default: auto] [possible values: auto, never, always]
  -o, --output <FILE>         Output file

Input Options:
      --input-format <FORMAT>       Input format [env: HL_INPUT_FORMAT=] [default: auto] [possible values: auto, json, logfmt]
      --unix-timestamp-unit <UNIT>  Unix timestamp unit [env: HL_UNIX_TIMESTAMP_UNIT=] [default: auto] [possible values: auto, s, ms, us, ns]
      --allow-prefix                Allow non-JSON prefixes before JSON messages [env: HL_ALLOW_PREFIX=]
      --delimiter <DELIMITER>       Log message delimiter, [NUL, CR, LF, CRLF] or any custom string

Advanced Options:
      --interrupt-ignore-count <N>  Number of interrupts to ignore, i.e. Ctrl-C (SIGINT) [env: HL_INTERRUPT_IGNORE_COUNT=] [default: 3]
      --buffer-size <SIZE>          Buffer size [env: HL_BUFFER_SIZE=] [default: "256 KiB"]
      --max-message-size <SIZE>     Maximum message size [env: HL_MAX_MESSAGE_SIZE=] [default: "64 MiB"]
  -C, --concurrency <N>             Number of processing threads [env: HL_CONCURRENCY=]
      --shell-completions <SHELL>   Print shell auto-completion script and exit [possible values: bash, elvish, fish, powershell, zsh]
      --man-page                    Print man page and exit
      --list-themes[=<TAGS>]        Print available themes optionally filtered by tags [possible values: dark, light, 16color, 256color, truecolor]
      --dump-index                  Print debug index metadata (in --sort mode) and exit

• MacBook Pro (16-inch, 2021)

  • CPU: Apple M1 Max CPU

  • OS: macOS Sequoia 15.2

  • Data: ~ 2.3 GiB log file, 6 000 000 lines

    • hl v0.30.2 ~ 1.1 seconds

      $ time hl --config - example.log -c -o /dev/null
      hl --config - example.log -c -o /dev/null  9.80s user 0.61s system 915% cpu 1.138 total

    • hlogf v1.4.1 ~ 8.7 seconds

      $ time hlogf example.log --color always >/dev/null
      hlogf example.log --color always > /dev/null  6.85s user 1.94s system 100% cpu 8.738 total

    • humanlog v0.7.8 ~ 79 seconds

      $ time humanlog <example.log --color always >/dev/null
      humanlog> reading stdin...
      humanlog --color always < example.log > /dev/null  87.68s user 7.33s system 120% cpu 1:19.01 total

    • fblog v4.13.1 ~ 34 seconds

      $ time fblog example.log >/dev/null
      fblog example.log > /dev/null  31.32s user 2.22s system 99% cpu 33.553 total

    • fblog with -d flag v4.13.1 ~ 146 seconds

      $ time fblog -d example.log >/dev/null
      fblog -d example.log > /dev/null  131.88s user 14.55s system 99% cpu 2:26.45 total

  • See #132 for how to repeat measurements