Spread data spikes over missing days in counter graphs#6472
Spread data spikes over missing days in counter graphs#6472dwmw2 wants to merge 2 commits intodomoticz:developmentfrom
Conversation
|
For some reason this isn't working on my production box even though it was working in testing on the same type of meter device. |
|
Ah, I figured it out. My test server had gaps where Domoticz itself hadn't been running. My production server had been running but the meter data hadn't been coming in, so there were actual zeroes in the data and not just gaps: I'm confused by the presence of zeroes; surely we shouldn't be writing to We can tweak the graph code to treat those like empty data, but I'm less sure that we should. What do you think? |
When a device hasn't updated within the timeout period, UpdateMeter() already prevents entries from being added to the Meter table. The AddCalendarUpdateMeter() function should leave gaps in Meter_Calendar rather than explicitly inserting zeros, allowing the graph smoothing code to handle the missing data appropriately.
When there are gaps in the data (e.g., device offline for multiple days), the accumulated usage appears as a large spike on the day data resumes. Detect gaps and spread the accumulated value evenly across the missing days to prevent spikes from affecting graph scale. The fillMissingDays function is implemented in the shared counterLogSeriesSupplier module and used by all counter/meter device types (energy-used, energy-generated, gas, water, counter, P1 energy).
|
Fixed the code which introduced the zeroes. We deliberately don't introduce explicit zeroes in the |
a0a55cd to
e57a1b6
Compare
Just to be sure, this does not alter my solar power "Meter_Calendar" table? Sorry if unrelated, just asking... |
|
Good question. No, it should not affect those days because they should have an explicit zero rather than just an absence of data. |
|
Thx, good to know. I do understand your use case, but e.g. I do appreciate your time! |
|
If I may suggest, another way of looking at your use case might be to change the autoscaler behavior? |
|
As you know, there is still an issue with Enphase/Envoy and the latest beta/counterhelper. |
|
I don't think @dwmw2 is specifically having an issue with his Enphase/Envoy. That is not stated in his opening post. I do agree that I would not like to have that spike get masked. That also would have masked the underlying problem. My personal opinion is that Domoticz should not alter (It is fair to say that I manually needed to edit the database to fix my issue. |
It's Domoticz which creates those tables. But yes, again: the 'smoothing' here is only in the display, and the tables aren't modified to contain the averaged-out data. That does need a bug fix to avoid writing false explicit zeroes to the Meter_Calendar when there are no new data during that day, which is the only change to the Meter_Calendar handling. |
When there are gaps in the data (e.g., device offline for multiple days), the accumulated usage appears as a large spike on the day data resumes. This preprocessing function detects gaps and spreads the accumulated value evenly across the missing days, preventing spikes from affecting graph scale.
It used to look like this:
... now it looks like this: