Make influxdb batch settings configurable #134758

jgaalen · 2025-01-05T10:49:45Z

Proposed change

This change makes the influxdb batch size and batch timeout configurable, rather than a fixed default of size: 100 and timeout: 1. It also fixes the timeout behaviour. Before, it waits for new events/values for the timeout (1s), but when values are received after 800ms, it waits for 1s again, causing situations where I had situations where it took more than 5 seconds before values are written.
This change makes it possible to write values quicker to influxdb, for a more realtime experience, or make larger batches and higher timeouts for lesser writes.

These values can be used in the influxdb yaml section:

  batch_buffer_size: 100
  batch_timeout: 0.1

Type of change

Dependency upgrade
Bugfix (non-breaking change which fixes an issue)
New integration (thank you!)
New feature (which adds functionality to an existing integration)
Deprecation (breaking change to happen in the future)
Breaking change (fix/feature causing existing functionality to break)
Code quality improvements to existing code or addition of tests

Additional information

This PR fixes or closes issue: -
This PR is related to issue: -
Link to documentation pull request: -

Checklist

The code change is tested and works locally.
Local tests pass. Your PR cannot be merged unless tests pass
There is no commented out code in this PR.
I have followed the development checklist
I have followed the perfect PR recommendations
The code has been formatted using Ruff (ruff format homeassistant tests)
[] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

The manifest file has all fields filled out correctly.
Updated and included derived files by running: python3 -m script.hassfest.
New or updated dependencies have been added to requirements_all.txt.
Updated by running python3 -m script.gen_requirements_all.
For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

I have reviewed two other open pull requests in this repository.

Improve timeout for new values

home-assistant · 2025-01-05T10:49:50Z

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

home-assistant · 2025-01-05T10:49:52Z

Hey there @mdegat01, mind taking a look at this pull request as it has been labeled with an integration (influxdb) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of influxdb can trigger bot actions by commenting:

@home-assistant close Closes the pull request.
@home-assistant rename Awesome new title Renames the pull request.
@home-assistant reopen Reopen the pull request.
@home-assistant unassign influxdb Removes the current integration label and assignees on the pull request, add the integration domain after the command.
@home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
@home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

jgaalen · 2025-01-07T07:09:20Z

What is necessary to get this PR going?

jgaalen · 2025-02-07T05:29:55Z

@mdegat01 So this PR is open for a while. What is that about?

Stale

github-actions · 2025-04-25T06:08:15Z

There hasn't been any activity on this pull request recently. This pull request has been automatically marked as stale because of that and will be closed if no further activity occurs within 7 days.
If you are the author of this PR, please leave a comment if you want to keep it open. Also, please rebase your PR onto the latest dev branch to ensure that it's up to date with the latest changes.
Thank you for your contribution!

jgaalen · 2025-04-25T06:11:47Z

@mdegat01 Hi, this PR is here now for a couple of months. Can you take a look please? Thanks

joostlek

I believe influxdb is an exception, but ideally we should not change YAML config anymore. There's a PR open to bring InfluxDB to the UI so I think it would be better to look at that PR and add it after that PR is merged

emontnemery · 2025-06-25T15:38:22Z

FTR, the other PR is this one: #134463

github-actions · 2025-08-24T16:06:15Z

There hasn't been any activity on this pull request recently. This pull request has been automatically marked as stale because of that and will be closed if no further activity occurs within 7 days.
If you are the author of this PR, please leave a comment if you want to keep it open. Also, please rebase your PR onto the latest dev branch to ensure that it's up to date with the latest changes.
Thank you for your contribution!

emontnemery · 2025-08-27T06:44:05Z

Not stale

github-actions · 2025-10-26T21:05:13Z

There hasn't been any activity on this pull request recently. This pull request has been automatically marked as stale because of that and will be closed if no further activity occurs within 7 days.
If you are the author of this PR, please leave a comment if you want to keep it open. Also, please rebase your PR onto the latest dev branch to ensure that it's up to date with the latest changes.
Thank you for your contribution!

emontnemery · 2025-10-27T07:00:20Z

The config flow PR is still not ready for merge. @RobBie1221 do you see a problem with merging this PR first?

RobBie1221 · 2025-10-27T08:38:07Z

No, I can deal with the conflicts.

mdegat01

Sorry for the delay. I'm good with this PR though searching influx documentation I notice a couple things:

It says the optimal batch size for Influx v2 is 5000 lines. So maybe we want to increase the default there.
Users of the cloud service need to be aware that it has it has hard limits here. In particular the free tier of Influx v2 cloud has a hard limit of 1MB/min. Since it only applies to free tier and cloud I don't know if we want to try and account for this in config schema but the corresponding documentation should note this.

jgaalen · 2025-10-27T15:21:09Z

Sorry for the delay. I'm good with this PR though searching influx documentation I notice a couple things:

It says the optimal batch size for Influx v2 is 5000 lines. So maybe we want to increase the default there.

To reach 5000 lines within the set delay, you probably need a hell of a lot lines. Basically the batch size is simply an upper limit of lines to add within the set timeout.

Users of the cloud service need to be aware that it has it has hard limits here. In particular the free tier of Influx v2 cloud has a hard limit of 1MB/min. Since it only applies to free tier and cloud I don't know if we want to try and account for this in config schema but the corresponding documentation should note this.

A good practise is to compress (gzip) the influx request payload to lower throughput and improve response times. For HA usually overkill.

cdce8p · 2025-11-04T12:48:26Z

homeassistant/components/influxdb/__init__.py

        """Return a batch of events formatted for writing."""
        queue_seconds = QUEUE_BACKLOG_SECONDS + self.max_tries * RETRY_DELAY
+        start_time = time.monotonic()
+        batch_timeout = self.batch_timeout()


This looks like an error. batch_timeout is now a property which returns a float so this will raise a TypeError. From the test logs

tests/components/influxdb/test_sensor.py: 25 warnings /home/runner/work/ha-core/ha-core/venv/lib/python3.13/site-packages/_pytest/threadexception.py:58: PytestUnhandledThreadExceptionWarning: Exception in thread influxdb Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.13.9/x64/lib/python3.13/threading.py", line 1043, in _bootstrap_inner self.run() ~~~~~~~~^^ File "/home/runner/work/ha-core/ha-core/homeassistant/components/influxdb/__init__.py", line 618, in run _, json = self.get_events_json() ~~~~~~~~~~~~~~~~~~~~^^ File "/home/runner/work/ha-core/ha-core/homeassistant/components/influxdb/__init__.py", line 554, in get_events_json batch_timeout = self.batch_timeout() TypeError: 'float' object is not callable

This reverts commit 0e2a460.

jgaalen added 3 commits January 5, 2025 10:09

Batch buffer size and timeout configurable

0696236

Improve timeout for new values

Fix

0ac3aeb

Fix

2e42eae

jgaalen requested a review from mdegat01 as a code owner January 5, 2025 10:49

home-assistant bot added bugfix cla-error integration: influxdb new-feature labels Jan 5, 2025

This comment was marked as resolved.

Sign in to view

home-assistant bot marked this pull request as draft January 5, 2025 10:49

home-assistant bot assigned mdegat01 Jan 5, 2025

home-assistant bot added the Quality Scale: legacy label Jan 5, 2025

jgaalen force-pushed the add_influxdb_batch_config branch from 3e0dc7e to 2e42eae Compare January 5, 2025 11:01

This comment was marked as resolved.

Sign in to view

home-assistant bot added cla-needed cla-recheck cla-signed and removed cla-recheck cla-needed labels Jan 5, 2025

Merge branch 'dev' into add_influxdb_batch_config

4060f0a

frenck added cla-recheck and removed cla-signed cla-error labels Feb 23, 2025

home-assistant bot added cla-signed and removed cla-recheck labels Feb 23, 2025

jgaalen marked this pull request as ready for review February 24, 2025 05:56

Merge branch 'dev' into add_influxdb_batch_config

02a6929

github-actions bot added the stale label Apr 25, 2025

github-actions bot removed the stale label Apr 25, 2025

joostlek reviewed Jun 23, 2025

View reviewed changes

github-actions bot added the stale label Aug 24, 2025

emontnemery removed the stale label Aug 27, 2025

github-actions bot added the stale label Oct 26, 2025

github-actions bot removed the stale label Oct 27, 2025

Merge branch 'dev' into add_influxdb_batch_config

d55fb5b

mdegat01 approved these changes Oct 27, 2025

View reviewed changes

emontnemery merged commit 0e2a460 into home-assistant:dev Nov 4, 2025
36 checks passed

cdce8p reviewed Nov 4, 2025

View reviewed changes

gjohansson-ST added a commit that referenced this pull request Nov 4, 2025

Revert "Make influxdb batch settings configurable (#134758)"

06bf3be

This reverts commit 0e2a460.

gjohansson-ST mentioned this pull request Nov 4, 2025

Revert "Make influxdb batch settings configurable" #155808

Open

Uh oh!

Make influxdb batch settings configurable #134758

Make influxdb batch settings configurable #134758

Conversation

jgaalen commented Jan 5, 2025

Proposed change

Type of change

Additional information

Checklist

Uh oh!

This comment was marked as resolved.

Uh oh!

home-assistant bot commented Jan 5, 2025

Uh oh!

home-assistant bot commented Jan 5, 2025

Uh oh!

This comment was marked as resolved.

Uh oh!

jgaalen commented Jan 7, 2025

Uh oh!

jgaalen commented Feb 7, 2025

Uh oh!

github-actions bot commented Apr 25, 2025

Uh oh!

jgaalen commented Apr 25, 2025

Uh oh!

joostlek left a comment

Choose a reason for hiding this comment

Uh oh!

emontnemery commented Jun 25, 2025

Uh oh!

github-actions bot commented Aug 24, 2025

Uh oh!

emontnemery commented Aug 27, 2025

Uh oh!

github-actions bot commented Oct 26, 2025

Uh oh!

emontnemery commented Oct 27, 2025

Uh oh!

RobBie1221 commented Oct 27, 2025

Uh oh!

mdegat01 left a comment

Choose a reason for hiding this comment

Uh oh!

jgaalen commented Oct 27, 2025

Uh oh!

Uh oh!

cdce8p Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants