A release that includes support for organizing compressed logs into datasets, a revamped UI, support for a new IR format designed for structured log events, and more advanced metadata storage within clp-json archives. This release also includes some other features and bug fixes.

The CLP release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• clp-json: Add support for organizing compressed logs into datasets, where each dataset can be searched separately. (#831, #839, #864, #868, #923, #1023, #1029, #1036, #1050, #1091)
• clp-json/clp-text: Replace the UI with a revamped version that has better performance while being more extensible and maintainable. (#645, #647, #791, #797, #804, #807, #824, #830, #848, #866, #875, #880, #891, #892, #899, #910, #912, #913, #915, #931, #933, #936, #937, #939, #941, #949, #950, #954, #956, #957, #958, #959, #962, #967, #972, #973, #974, #988, #990, #991, #993, #995, #997, #999, #1000, #1001, #1003, #1007, #1013, #1015, #1016, #1042, #1053, #1061, #1063, #1064, #1074, #1078, #1079, #1083)
• core: Add support for a new CLP intermediate representation (IR) stream format which is designed for compressing and searching structured (e.g., JSON) log events. (#411, #446, #502, #507, #511, #549, #653, #677, #695, #706, #726, #728, #731, #733, #769, #833, #856, #859, #860, #863, #873, #882, #888, #890, #909, #914)
• clp-s: Add support for storing metadata about ranges of log events (e.g., the name of the log file to which they were printed) in an archive. (#847, #889, #908, #929, #1005, #1026, #1043)

Bug fixes & improvements

• clp-s: Improve compression ratio by delta-encoding the log-order column. (#1021)
• clp-json/clp-text: Add support for configuring the compression level. (#774)
• clp-json: Add support for more AWS authentication methods (e.g., EC2 instance metadata, environment variables, AWS profiles, etc.). (#743, #788, #852)
• clp-json/clp-text: Don't remap output paths when mounting them into CLP's execution container (fixes #960). (#998)
• core: Try to exhaust Zstd's internal buffers when they might contain unconsumed data. (fixes #976). (#977)
• clp-json/clp-text: Use job duration for final compression speed summary. (#823)
• core: Update DictionaryReader::get_entry_matching_value to handle case-insensitive searches (fixes #648). (#690)
• webui: Improve tracking of compression jobs submitted in quick succession (fixes #667). (#679)

View the full changelog for more details.

Thanks to @aestriplex, @anlowee, @AVMatthews, @Bill-hbrhbr, @davemarco, @davidlion, @Eden-D-Zhang, @gibber9809, @haiqi96, @Henry8192, @hoophalab, @jackluo923, @junhaoliao, @kirkrodrigues, @LinZhihao-723, @quinntaylormitchell, @SharafMohamed, @sitaowang1998, and @wraymo for their contributions.