Thanks to visit codestin.com
Credit goes to github.com

Skip to content

bug: Timestream fails to write records: SQL syntax error at or near ")"' #12827

Open
@gornostal

Description

@gornostal

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When I do write records with measure_value set to 0 fails to write it and logs this error:

2025-07-02T16:22:03.436 ERROR --- [et.reactor-0] l.p.c.s.t.provider         : Failed to write records: {'S': 'ERROR', 'V': 'ERROR', 'C': '42601', 'M': 'syntax error at or near ")"', 'P': '950', 'F': 'scan.l', 'L': '1129', 'R': 'scanner_yyerror'}
Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/pg8000/legacy.py", line 251, in execute
    self._context = self._c.execute_simple(operation)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/pg8000/core.py", line 687, in execute_simple
    self.handle_messages(context)
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/pg8000/core.py", line 836, in handle_messages
    raise context.error
pg8000.exceptions.DatabaseError: {'S': 'ERROR', 'V': 'ERROR', 'C': '42601', 'M': 'syntax error at or near ")"', 'P': '950', 'F': 'scan.l', 'L': '1129', 'R': 'scanner_yyerror'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/pro/core/services/timestream/provider.py.enc", line 152, in write_records
    try:e(D);M+=1

Expected Behavior

It should be able to write 0 values

How are you starting LocalStack?

With a docker-compose file

Steps To Reproduce

Docker-compose

  timestream:
    image: localstack/localstack-pro
    ports:
      - 4566:4566
    environment:
      - SERVICES=timestream-query,timestream-write
      - DEBUG=1
      - LOCALSTACK_AUTH_TOKEN=ls-xxxxxxx

.NET Client

var record = new Amazon.TimestreamWrite.Model.Record
{
    Dimensions = dimensions,
    MeasureName = measureName,
    MeasureValue = paymentEvent.Amount.ToString("F2"),
    MeasureValueType = MeasureValueType.DOUBLE,
    Time = paymentEvent.Timestamp.ToUnixTimeMilliseconds().ToString(),
    TimeUnit = TimeUnit.MILLISECONDS,
};

Environment

- OS:Ubuntu 24.04
- LocalStack:
  image: localstack/localstack-pro
  LOCALSTACK_BUILD_GIT_HASH=37cd1bf17
  LOCALSTACK_BUILD_DATE=2025-07-02
  LOCALSTACK_BUILD_VERSION=4.5.1.dev140
  LocalStack Docker image sha: f80c740fbf259e14aa89112e3e86e8bffab5b9802beb1306d79d5155584bb174

Anything else?

I tried to debug. Got to the point where I see the correct SQL with measure_value set to non-zero:

DO $$
BEGIN
INSERT INTO PaymentEvents ("Id", "CardId", "CorrelationId", "Accepted", "Atm", "FailedCvv", "AuthType", "measure_name", "measure_value", "measure_value_type", "time", "_internal_version", "_internal_id") VALUES ('b90482a0-5402-408c-9bfb-979ca117526e','1','84110d034c7e475bb414379b5575b8f8','False','False','True','Authorisation','Authorisation',12.34,'DOUBLE','2025-07-02 15:50:39.423000000',0,'2025-07-02 15:50:39.423000000:Accepted.False,Atm.False,AuthType.Authorisation,CardId.1,CorrelationId.84110d034c7e475bb414379b5575b8f8,FailedCvv.True,Id.b90482a0-5402-408c-9bfb-979ca117526e:Authorisation')
ON CONFLICT ON CONSTRAINT enforce_unique_records DO
    UPDATE SET
        _internal_version =
            (CASE
                -- Idempotent call will just go through
                WHEN (
                    PaymentEvents._internal_version = EXCLUDED._internal_version
                    AND
                    PaymentEvents.measure_value = EXCLUDED.measure_value
                    ) THEN PaymentEvents._internal_version
                -- Greater version will overwrite the data
                WHEN (PaymentEvents._internal_version < EXCLUDED._internal_version) THEN EXCLUDED._internal_version
                WHEN (
                    PaymentEvents._internal_version > EXCLUDED._internal_version
                    ) THEN raise_lower_version_exception(PaymentEvents._internal_version ,EXCLUDED._internal_version)
                -- Otherwise we raise an exception with version
                ELSE raise_duplicate_exception(PaymentEvents._internal_version)
            END),
        -- If we didn't raise, we can just go on with the operation Ideally we would check if idempotent and keep
        -- the existing instead of replacing... but it might be more costly to do the idempotency check on all values?
        measure_value = EXCLUDED.measure_value;
END;
$$;

And incorrect one is generated when value is zero:

DO $$
BEGIN
INSERT INTO PaymentEvents ("Id", "CardId", "CorrelationId", "Accepted", "Atm", "FailedCvv", "AuthType", "measure_name", "measure_value", "measure_value_type", "time", "_internal_version", "_internal_id") VALUES ('b90482a0-5402-408c-9bfb-979ca117526e','1','84110d034c7e475bb414379b5575b8f8','False','False','True','Authorisation','Authorisation',12.34,'DOUBLE','2025-07-02 15:50:39.423000000',0,'2025-07-02 15:50:39.423000000:Accepted.False,Atm.False,AuthType.Authorisation,CardId.1,CorrelationId.84110d034c7e475bb414379b5575b8f8,FailedCvv.True,Id.b90482a0-5402-408c-9bfb-979ca117526e:Authorisation')
ON CONFLICT ON CONSTRAINT enforce_unique_records DO
    UPDATE SET
        _internal_version =
            (CASE
                -- Idempotent call will just go through
                WHEN (
                    PaymentEvents._internal_version = EXCLUDED._internal_version
                    AND

                    ) THEN PaymentEvents._internal_version
                -- Greater version will overwrite the data
                WHEN (PaymentEvents._internal_version < EXCLUDED._internal_version) THEN EXCLUDED._internal_version
                WHEN (
                    PaymentEvents._internal_version > EXCLUDED._internal_version
                    ) THEN raise_lower_version_exception(PaymentEvents._internal_version ,EXCLUDED._internal_version)
                -- Otherwise we raise an exception with version
                ELSE raise_duplicate_exception(PaymentEvents._internal_version)
            END),
        -- If we didn't raise, we can just go on with the operation Ideally we would check if idempotent and keep
        -- the existing instead of replacing... but it might be more costly to do the idempotency check on all values?
        measure_value = EXCLUDED.measure_value;
END;
$$;

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions