v7.1-beta.7

* Issue **gchq#2640** : Support adding formatted dates to index name in ElasticIndexingFilter.

* Issue **gchq#2675** : Improve Elasticsearch scroll query performance

* Issue **gchq#2677** : Respect column formatting in Excel cell styles.

* Issue **gchq#2703** : Improve search trace log to identify multi query search issues.

* Issue **gchq#2705** : Attempt to fix SIDSEGV issue caused by too many LMDB readers.

* Issue **gchq#2688** : Add `metaTypes` to Proxy's `proxyRequestConfig`. Change data receipt to validate types against this new config property.

* Issue **gchq#2688** : Change defaults for Strooms `metaTypes` config property. Change type of `metaTypes` from String to a Set of Strings. Change data receipt to validate against this configured set.

* Issue **gchq#2694** : Improve error handling when search result store has reached max capacity.

* Issue **gchq#2689** : Add logging.

* Issue **gchq#2679** : Add unique index on `meta_processor.pipeline_uuid`.

* Issue **gchq#2596** : Disable the Ace editor settings menu (`ctrl-,`) as users should not have access to it and it does not work with modal dialogs.

* Issue **gchq#2139** : Add DB migration scripts to the ZIP distribution.

* Issue **gchq#2685** : Fix issue where searchable stores are limited to 1 million rows.

* Issue **gchq#2696** : Change the byte buffer pool to not block by default and make that configurable.

* Issue **gchq#2666** : Fix handling of missing reference stream file.

* Change error table to wrap error message text.

* Issue **gchq#2644** : Fix error handling when volume is full during proxy aggregation.

* Issue **gchq#2650** : Fix handling of negative numbers and negation in expression parser.

* Issue **gchq#2664** : Fix issue with search where a complete coprocessor was causing other coprocessor payloads to be read incorrectly.

* Issue **gchq#2648** : Add error message to data viewer when ID is invalid.

* Issue **gchq#2657** : Change classes to use provided config to stop the system using stale config values.

* Issue **gchq#2603** : Suppress expected buffer underflow exception.

* Make directly injected NotInjectable config throw an exception to prevent it happening.

* Issue **gchq#2652** : Add delay to stream event map creation so extraction opens fewer streams.

* Issue **gchq#2639** : Fix error message when invalid property values are used.

* Issue **gchq#2578** : Fix how application config is (re)loaded so that it copes with null or sparse branches.

* Issue **gchq#2361** : Fix create_account command so it allows creation of accounts with no password.

* Issue **gchq#2627** : Add processor filter and processor task id to meta.

* Issue **gchq#2628** : Add processor filter and processor task id to info log.

* Issue **gchq#2647** : Fix `Unrecognised permission assigned` error in logs.

* Issue **gchq#2629** : Change the way duplicate output is handled by deleting previous output from complete tasks at the end of processing.

* Issue **gchq#2603** : Add debug to diagnose buffer underflow.

* Issue **gchq#2637** : Fix hanging searches.

* Issue **gchq#2631** : Add debug to diagnose UI issue.

* Issue **gchq#2633** : Fix shutdown.

* Issue **gchq#2632** : Suppress warnings.

* Issue **gchq#2603** : Improve error handling.

* Issue **gchq#2635** : Improve search completion code.

* Issue **gchq#2625** : Disable test temporarily to fix build.

* Issue **gchq#2630** : Fix NPE and improve logging.

* Issue **gchq#2625** : Disable test temporarily to fix build.

* Issue **gchq#2615** : Fix meta status change triggered by check superseded.

* Issue **gchq#2614** : Fix NPE.

* Issue **gchq#2621** : Improve search performance.

* Issue **gchq#2617** : Fix pipeline data equality for pipeline structure cache to be useful.

* Issue **gchq#2619** : Improve meta DAO performance.

* Issue **gchq#2611** : Fix java lang error.

* Issue **gchq#2608** : Fix segments for rolled streams created using the `StreamAppender`.

* Issue **gchq#2604** : Add code to diagnose unexpected event counts in search extraction.

* Issue **gchq#2605** : Create sub tasks for all search processes.

* Issue **gchq#2606** : Improve error handling during search.

* Issue **gchq#2472** : Fix the way the in memory config is updated by the file monitor. Improve the logging when config properties are changed on a node.

* Issue **gchq#2595** : Improve search performance and fix issues.

* Issue **gchq#2471** : Changes to ensure SQLite DB connection is only used by one process at a time.

* Issue **gchq#2471** : Changes to help diagnose proxy aggregation issues.

* Issue **gchq#2582** : Remove unnecessary legacy migration for processor filter.

* Issue **gchq#2577** : Fix issue with expression terms changing conditions incorrectly.

* Issue **gchq#2585** : Fix proxy aggregation task nesting.

* Issue **gchq#2574** : Improve processor filter creation to allow min and max create times.

* Issue **gchq#2579** : Fix processor filter creation from stream multi selection.

* Issue **gchq#2580** : Fix completion of processor filters by allowing the user to specify an end time.

* Issue **gchq#2576** : Fix scrolling of processor filter info pane.

* Uplift event-logging library to `5.0-beta.27_schema-v4.0-beta.3` to fix missing failure outcomes on logged events.

* Issue **gchq#2557** : Optimise meta queries to ensure join order is as expected.

* Issue **gchq#2538** : Change logging of quick filter searches to log the fully qualified filter input.

* Issue **gchq#2565** : Stop orphan file finder reporting dirs that contain child dirs as empty.

* Issue **gchq#2563** : Fix bad expression logic.

* Issue **gchq#2562** : Fix NPE in the UI related to uninitialised processor filter trackers.

* Stop logging audit events for most NodeResource call as they are not direct user actions.

* Issue **gchq#2553** : Prevent get and clear methods in LMDB data store from running at the same time. Add check for the LMDB env being closed to prevent JVM crash.

* Issue **gchq#2555** : Remove checkbox from dead tasks in the server tasks screen to stop users trying to delete them.

* Issue **gchq#2564** : Improve search performance.

* Issue **gchq#2582** : Fix DB migration for `processor_filter`.

* Issue **gchq#2542** : Improve autologged searches "raw" JSON.

* Issue **gchq#2534** : Implement ProcessorResource.fetch to fix event log.

* Issue **gchq#2533** : Fix corrupt event format for logout due to NPE.

* Issue **gchq#2530** : Fix issue of explorer tree not refreshing on copy, move and delete when auth is disabled.

* Add wait time debug logging around LMDB locks.

* Increase maxReaders default to 150 for reference data.

* Issue **gchq#2548** : Clear contents of `stroom.search.resultStore.lmdb.localDir` on boot.

* Issue **gchq#2549** : Remove `maxDbs` from `lmdb` config. Remove `readerBlockedByWriter` from `resultStore.lmdb` config.

* Issue **gchq#2544** : Add additional constraints to processor filter instead of using tracker state

* No changes. Previous build failed due to networking issues.

* Issue **gchq#2540** : No longer error when interrupting shard flush process

* Issue **gchq#2541** : Stop finished queries from deleting LMDB envs for all queries.

* Issue **gchq#2535** : Added validation to data type names.

* Issue **gchq#2530** : Fixed explorer refresh on copy and move

* Revert accidental whitespace change to V07_00_00_007__meta_retention_tracker.sql

* Issue **gchq#2519** : Added validation for processor filter expressions and more info about filters to the UI.

* Issue **gchq#2501** : Change reference data store to use consistent approach to last access time truncation.

* Issue **gchq#2424** : Change security filter to 404 any unexpected URIs.

* Issue **gchq#2493** : Fix missing part nav controls when data can't be decoded.

* Issue **gchq#2497** : Added summary to orphan file finder. Fixed issue with `OrphanFileFinder` incorrectly identifying some dirs as being empty.

* Issue **gchq#2500** : Add a primary key to the `meta_retention_tracker` table for MySQL Group Replication.

* Change meta retention tracking to track at the time period level so a killed job preserves the position of the periods already processed.

* Issue **gchq#2513** : Fixed stepping to unique values.

* Issue **gchq#2496** : Fixed issue where data browser was showing duplicate streams.

* Issue **gchq#2511** : Fixed stepping error handling.

* Issue **gchq#2512** : Fixed stepping error handling.

* Issue **gchq#2478** : Create a single place in config for the LMDB library path and extraction dir. Delete old LMDB library binaries on boot.

* Change `/api/refData/v1/purgeByAge/{purgeAge}`, `/api/refData/v1/purgeByStream/{refStreamId}` and `/api/refData/v1/clearBufferPool` to act on all nodes unless the `nodeName` query param is provided.

* Issue **gchq#2483** : Change reference data purge to delete entries in batchs to avoid large transactions that result in errors.

* Issue **gchq#2502** : Fix `Cannot read properties` error when switching streams after deleting one.

* Issue **gchq#2449** : Reduce the number of REST calls that property filtering in the UI makes.

* Issue **gchq#2523** : Uplift standard-pipelines to v0.4

* Issue **gchq#2527** : Fix stepping source hightlight for first record of fragment XML data.

* Issue **gchq#2526** : Fix incorrect input pane in stepper for cooked xml

* Issue **gchq#2531** : Fix incorrect value in Retention column in data browser and on Info tab when `Feed is` conditions are used.

* Issue **gchq#2532** : Change logged event for Manage Accounts quick filter use.

* Fix performance issue with ref data range lookups.

* Change default value for `stroom.pipeline.referenceData.readerBlockedByWriter` to true.

* Fix locking for `stroom.pipeline.referenceData.readerBlockedByWriter`.

* Issue **gchq#2494** : Changed logging to help diagnose problem.

* Issue **gchq#2429** : The server tasks screen now handles errors that occur when trying to contact unreachable nodes.

* Issue **gchq#2492** : Fixed issue getting filter priorities in processor task data store.

* Issue **gchq#2489** : Change to wrap long error messages in stepping display.

* Issue **gchq#2487** : Fixed issue flushing and deleting index shards.

* Issue **gchq#2442** : Removed enabled/disabled states for permission users and user groups as these are now controlled by account authentication.

* Issue **gchq#2431** : Fixed issue with item selection in account and token list pages.

* Issue **gchq#2484** : Fix failing ref lookups when one loader has range data.

* Issue **gchq#2485** : Autologger: UNLOGGED calls always being logged.

* Issue **gchq#2395** : Fixed timezone issue in API key display.

* Issue **gchq#2452** : New pipeline references now default to `Reference` data type.

* Issue **gchq#2414** : The processing tasks data source now exposes start, end and status times plus filter priorities.

* Issue **gchq#2441** : Improve logging and prevent autologger warning for `MetaResourcImpl`

* Issue **gchq#2440** : Prevent autologger warning for `ExplorerResourceImpl`

* Issue **gchq#2465** : `SaveAs` is now working for non admin users.

* Issue **gchq#2460** : Processing filters now reliably process feeds where wildcards are used to match feed names.

* Issue **gchq#2475** : Orphan file and meta finder now show the correct task progress.

* Issue **gchq#2474** : Fixed orphan file finder.

* Issue **gchq#2467** : Fix viewing of streams that can't be decoded. Errors now displayed in large banner rather than as text in the editor.

* Add a view as hex option to the Data Preview panes.

* Add option to show/hide the editor indent guides.

* Remove white space on editor context menu.

* Change data viewing 'progress' bar to have a minimum width of 3px to make it more visible.

* Issue **gchq#2469** : Made `SafeXmlFilter` available for use in the application.

* Issue **gchq#2403** : Fix guice bind errors in stroom and proxy when the `path` config branch is empty or set to null.

* Issue **gchq#2462** : Made changes to stop DB connections being used within the context of other open connections.

* Uplift LMDBJava to 0.8.2 to fix LMDBJava cursor comparator bug.

* Issue **gchq#2464** : Add `leakDetectionThreshold` to the hikari pool config.

* Issue **gchq#2463** : Integrate the hikari connection pool with drop wizard's health checks and metrics.

* Add debug logging to AuthenticationStateSessionUtil

* Issue **gchq#2448** : Change data receipt ERROR log messages to WARN. Also improve log message content for success and failure.

* Issue **gchq#2412** : You are now able to view locked streams or deleted data if it is still accessible.

* Issue **gchq#2413** : Removed duplication of data retention fields in info pane.

* Issue **gchq#2443** : Internal meta statistics are now added with processing user permissions.

* Issue **gchq#2455** : Improved error handling when streams not found.

* Issue **gchq#2399** : Now changing expression fields keeps the same condition if it is still applicable to the new field.

* Issue **gchq#2426** : Fixed user selection problem for document permissions.

* Issue **gchq#2436** : Fixed small UI issue whereby a user or group was not immediately removed from document permissions when the remove button was clicked.

* Issue **gchq#2416** : Added logging to identify cause of slow meta listing.

* Improve termination handling in reference data purge.

* Improve the data in the `/api/refData/v1/refStreamInfo` api method.

* Add API method to clear the byte buffer pool.

* Improve the output of the system info api call for the byte buffer pool.

* Issue **gchq#2439** : Change log level for an error in the byte buffer pool.

* Issue **gchq#2444** : Fix release of buffers to the pool that was causing ref data searches to hang.

* Issue **gchq#2433, gchq#2434** : Fixed shutdown task order.

* Issue **gchq#2430** : Removed file system clean task and replaced with orphan finding jobs for files and meta.

* Issue **gchq#2432** : Volume state is now updated without optimistic locking.

* Issue **gchq#2382** : Improved charset resolution.

* Issue **gchq#2447** : Correctly offset displayed date/time values by the configured user timezone.

* Issue **gchq#2457** : Display a tooltip when hovering the mouse over a grid column cell.

* Issue **gchq#2382** : Improve error message for invalid feed encodings.

* Issue **gchq#2387** : Fix reference data load/lookup failure handling.

* Issue **gchq#2422** : Change ref lookups to only do a lookup against a ref feeds that are known to contian the map being looked up against.

* Issue **gchq#2411** : Remove 10,000 limit on dashboard search of ref store.

* Issue **gchq#2389** : Add an API method for purging a single reference data stream.

* Issue **gchq#2379** : Change ref data lookups to truncate last access time to hourly.

* Uplift Dropwizard from 1.3.14 to 1.3.29.

* Issue **gchq#2424** : Stop session creation for rest calls. Remove unused SessionMap class.

* Change debug summary logging in Data Delete job to info level.

* Issue **gchq#2402** : Improved logging for error caused by invalid meta filter values.

* Issue **gchq#2404** : Added debug to help diagnose issue.

* Issue **gchq#2423** : Added error logging and configuration to handle buffer overflows when dealing with large search result values.

* Issue **gchq#2410** : Fixes for dashboard child selectors like `first()` and `last()`.

* Issue **gchq#2417** : Bad regex filter value no longer logged unnecessarily.

* Issue **gchq#2418** : LMDB environment is now only closed when the search results are no longer needed.

* Issue **gchq#2419** : Conditional formatting errors are now returned to the UI.

* Issue **gchq#2405, gchq#2407** : Errors caused by a thread interrupt when viewing a stream are no longer logged.

* Issue **gchq#2406** : Volume status update is now performed synchronously.

* Issue **gchq#2401, gchq#2408, gchq#2409** : Processing tasks no longer terminate by interrupting threads so error streams can now be written correctly.

* Issue **gchq#2398** : Fix to clear interrupt state for threads used by terminated tasks.

* Issue **gchq#2396** : Add `stroom:pointIsInsideXYPolygon` XSLT function.

* Allow HTTP request headers to be customized in HTTPAppender.

* Issue **gchq#2392** : Fix for token type to only allow `api`.

v7.0-beta.174

* Issue **gchq#2708** : Keep search results fresh by constantly pinging result caches for all active searches.

v7.0-beta.173

* Issue **gchq#2703** : Improve search trace log to identify multi query search issues.

* Issue **gchq#2705** : Attempt to fix SIDSEGV issue caused by too many LMDB readers.

v7.0-beta.172

* Issue **gchq#2688** : Add `metaTypes` to Proxy's `proxyRequestConfig`. Change data receipt to validate types against this new config property.

* Issue **gchq#2688** : Change defaults for Strooms `metaTypes` config property. Change type of `metaTypes` from String to a Set of Strings. Change data receipt to validate against this configured set.

* Issue **gchq#2694** : Improve error handling when search result store has reached max capacity.

* Issue **gchq#2689** : Add logging.

v7.0-beta.171

* Issue **gchq#2679** : Add unique index on `meta_processor.pipeline_uuid`.

* Issue **gchq#2596** : Disable the Ace editor settings menu (`ctrl-,`) as users should not have access to it and it does not work with modal dialogs.

* Issue **gchq#2139** : Add DB migration scripts to the ZIP distribution.

* Issue **gchq#2685** : Fix issue where searchable stores are limited to 1 million rows.

* Issue **gchq#2696** : Change the byte buffer pool to not block by default and make that configurable.

v7.0-beta.170

* Issue **gchq#2666** : Fix handling of missing reference stream file.

* Change error table to wrap error message text.

* Issue **gchq#2644** : Fix error handling when volume is full during proxy aggregation.

* Issue **gchq#2650** : Fix handling of negative numbers and negation in expression parser.

* Issue **gchq#2664** : Fix issue with search where a complete coprocessor was causing other coprocessor payloads to be read incorrectly.

v7.0-beta.169

* Issue **gchq#2648** : Add error message to data viewer when ID is invalid.

* Issue **gchq#2657** : Change classes to use provided config to stop the system using stale config values.

* Issue **gchq#2603** : Suppress expected buffer underflow exception.

* Make directly injected NotInjectable config throw an exception to prevent it happening.

v7.0-beta.168

* Issue **gchq#2652** : Add delay to stream event map creation so extraction opens fewer streams.

v7.0-beta.167

* Issue **gchq#2639** : Fix error message when invalid property values are used.

* Issue **gchq#2578** : Fix how application config is (re)loaded so that it copes with null or sparse branches.

* Issue **gchq#2361** : Fix create_account command so it allows creation of accounts with no password.

* Issue **gchq#2627** : Add processor filter and processor task id to meta.

* Issue **gchq#2628** : Add processor filter and processor task id to info log.

* Issue **gchq#2647** : Fix `Unrecognised permission assigned` error in logs.

* Issue **gchq#2629** : Change the way duplicate output is handled by deleting previous output from complete tasks at the end of processing.

* Issue **gchq#2603** : Add debug to diagnose buffer underflow.

v7.0-beta.166

* Issue **gchq#2637** : Fix hanging searches.