Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Search attribute Datetime need to support full ISO 8601 (and maybe more) #144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cretz opened this issue Sep 27, 2022 · 3 comments · Fixed by #179
Closed

Search attribute Datetime need to support full ISO 8601 (and maybe more) #144

cretz opened this issue Sep 27, 2022 · 3 comments · Fixed by #179
Labels
bug Something isn't working

Comments

@cretz
Copy link
Member

cretz commented Sep 27, 2022

Describe the bug

2022-09-27T22:28:42Z cannot be parsed by datetime.fromisoformat which we are using for search attributes. Make sure the full RFC3339Nano is supported.

@cretz cretz added the bug Something isn't working label Sep 27, 2022
@cretz cretz changed the title Search attribute Datetime types can be RFC3339Nano, not always ISO 8601 Search attribute Datetime need to support full ISO 8601 (and maybe more) Sep 27, 2022
@cretz
Copy link
Member Author

cretz commented Sep 27, 2022

From https://docs.python.org/3/library/datetime.html#datetime.datetime.fromisoformat:

Caution
This does not support parsing arbitrary ISO 8601 strings - it is only intended as the inverse operation of datetime.isoformat(). A more full-featured ISO 8601 parser, dateutil.parser.isoparse is available in the third-party package dateutil.

@Oracen
Copy link
Contributor

Oracen commented Oct 30, 2022

I've been experimenting with the new Scheduler system released in Temporal 1.18.0. Workers using the Python API break on the same problem described here. This can be replicated with the following code snippet, then setting up a schedule either in the web UI or tctl.

import asyncio
from datetime import datetime

from dateutil import parser
from temporalio import workflow
from temporalio.client import Client
from temporalio.worker import Worker


@workflow.defn
class Replication:
    @workflow.run
    async def run(self):
        pass


async def main():
    client = await Client.connect("localhost:7233")
    worker = Worker(client, workflows=[Replication], task_queue="replication")

    print("Starting worker")
    await worker.run()


if __name__ == "__main__":
    print("Before:", datetime.fromisoformat("2022-10-30T05:13:00"))
    try:
        print("Won't run:", datetime.fromisoformat("2022-10-30T05:13:00Z"))
    except ValueError:
        print("Exception thrown")
    print("After:", parser.isoparse("2022-10-30T05:13:00Z"))

    asyncio.run(main())

This produces the following stack trace:

Failed handling activation on workflow with run ID 435abaf1-36bc-49ff-b003-4d6a8d833315
Traceback (most recent call last):
  File "./.venv/lib/site-packages/temporalio/worker/workflow.py", line 174, in _handle_activation
    workflow = await self._create_workflow_instance(act)
  File "./.venv/lib/site-packages/temporalio/worker/workflow.py", line 321, in _create_workflow_instance
    search_attributes=temporalio.converter.decode_search_attributes(
  File "./.venv/lib/site-packages/temporalio/converter.py", line 704, in decode_search_attributes
    val = [datetime.fromisoformat(v) for v in val]
  File "./.venv/lib/site-packages/temporalio/converter.py", line 704, in <listcomp>
    val = [datetime.fromisoformat(v) for v in val]
ValueError: Invalid isoformat string: '2022-10-30T06:28:30Z'

I did some work on a fork that resolved down to a one-line fix + some unit tests for regression testing around the converter.decode_search_attributes function, linked here: https://github.com/Oracen/sdk-python

I'll open a PR with the fix now, hopefully the problem is as simple as it appears

@cretz
Copy link
Member Author

cretz commented Oct 31, 2022

👍 Saw the PR but wondering if we can avoid the additional dependency. Are there any snippets/approaches where we could do this using just the standard library?

(EDIT: Starting to look unreasonable to vendor this or avoid the dependency. I'll update the PR)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants