Allow setting the time of a point manually. #304
Conversation
Point can be specified as either a number of nanoseconds, a python datetime object (with or without timezone) or a string in ISO datetime format. If a time is not specified, the Helper sets the time at the time of assembling the point fields so that multiple unique points with the same tags can be committed simultaneously without them failing to add due to all being assigned the same automatic time by the InfluxDB server. This fix is based upon the discussion in influxdata#130 but also includes the outstanding items for it to be merged. I'm happy to receive suggestions for further ways to add test coverage to this change. This also fixes influxdata#264 and fixes influxdata#259.
|
Hi @grundleborg! I've just opened a PR for the exact same feature. I didn't see that yours already existed, and didn't think to check because I had already told @aviau that I would work on this feature a few weeks ago. Had I seen this PR I would have discussed it with you beforehand. I've taken a slightly different approach as the one in your PR & #130 that avoids a few issues:
Take a look and let me know what you think 😃. Maybe @aviau can also weigh in. |
|
@matt-snider no problem, and quite happy for yours to be merged instead if it handles things more robustly. I needed the feature right away so couldn't wait for someone else to implement it, and I thought I might as well send a PR in since I didn't know how long it would be before anyone else actually dealt with it (and the original PR where the discussion was happening didn't seem to be going anywhere fast). |
|
Yep, I understand, especially considering the original PR was opened nearly a year ago... The important part is that we get this feature merged! 😄 |
|
Thank your for your contribution! |
Allow setting the time of a point manually. (Thanks @grundleborg !)
|
It does. You are setting a time when the point is created. #264 was because we didn't set a default time. |
|
Correct me if I'm wrong but won't all points receive approximately the same timestamp, as they are generated only when Edit: in #264 he also mentions that all points collapse down to one because the timestamps are equivalent. What I'm saying is that that can still occur here |
|
The timestamps were originally equivalent because InfluxDB was setting one when receiving the batch. I don't know how quickly _json_body() runs but I wouldn't expect timestamps to be the same. We can also move the timestamp generation to when the user creates the point from the serieshelper. This is still better than what we originally had. |
|
This is exactly the approach I took because it results in more accurate timestamps and rules out any chance of accidental loss of data (i.e. due to a race condition). Look I'm not advocating my approach over @grundleborg 's just because I want to have my code merge or anything like that. But if we're implementing automatic timestamps, shouldn't they be accurate to some degree? What about the case where I'm using a batch size (e.g. 500) and those points now all end up with the timestamp generated immediately before being sent? |
This reverts commit c25ec08, which this commit is part of PR influxdata#304.
* Revert Allow setting the time of a point manually This reverts commit c25ec08, which this commit is part of PR #304. * Allow time to be specified in SeriesHelper.__init__() * Extract SeriesHelper default timestamp into method for testability * Use datetime.utcnow() as default timestamp in SeriesHelper This is preferable to time.time() because _convert_timestamp() from influxdb.line_protocol will do precision handling and conversion if a datetime object is given. * Get existing tests working by mocking SeriesHelper._current_timestamp() * Add additional tests for SeriesHelper time field * Move _reset_() calls in TestSeriesHelper to tearDown() * Use mock.patch() instead of unittest.mock.patch() for py27 * Update SeriesHelper docstring
Point can be specified as either a number of nanoseconds, a python
datetime object (with or without timezone) or a string in ISO datetime
format.
If a time is not specified, the Helper sets the time at the time of
assembling the point fields so that multiple unique points with the same
tags can be committed simultaneously without them failing to add due to
all being assigned the same automatic time by the InfluxDB server.
This fix is based upon the discussion in #130 but also includes the
outstanding items for it to be merged. I'm happy to receive suggestions
for further ways to add test coverage to this change.
This also fixes #264 and fixes #259.