40 Pull requests merged by 17 people

• [AnomalyDetection] Add a notebook for anomaly detection with Z-Score

  #34459 merged May 7, 2025

• [AnomalyDetection] Add a notebook for using iforest for anomaly detection

  #34845 merged May 7, 2025

• Bump google.golang.org/api from 0.231.0 to 0.232.0 in /sdks

  #34876 merged May 7, 2025

• Replace deprecated model version.

  #34875 merged May 7, 2025

• [IcebergIO] Support column pruning

  #34856 merged May 7, 2025

• reorder opt-out review comment

  #34872 merged May 7, 2025

• Fix pr-bot workflows

  #34877 merged May 7, 2025

• Add yapf version upgrade doc to discussion-docs for 2025

  #34871 merged May 6, 2025

• Encode paneinfo with PaneInfoCoder. (#34824)

  #34864 merged May 6, 2025

• [release-2.65] Cherrypick PR #34867 to the release branch

  #34868 merged May 6, 2025

• fix: correct spanner column schema type parser

  #34867 merged May 6, 2025

• Improve clarity of PTransformRunnerFactory in Java SDK harness

  #34846 merged May 6, 2025

• Add label when PR gets reassigned

  #34866 merged May 6, 2025

• Update breaking changes with PaneInfoCoder change.

  #34865 merged May 6, 2025

• Fix Tour of Beam Go Integration tests

  #34854 merged May 6, 2025

• Revert "Bump @octokit/request and @octokit/rest in /scripts/ci/issue-report"

  #34857 merged May 6, 2025

• Use checkerframework Nullable/NonNull

  #34851 merged May 6, 2025

• Bump github.com/nats-io/nats-server/v2 from 2.11.2 to 2.11.3 in /sdks

  #34838 merged May 6, 2025

• Modify the assert to accept any instance of AvroCoder, including subclasses

  #34850 merged May 6, 2025

• Change Assert to Check instanceof AvroCoder

  #34843 merged May 5, 2025

• Cherrypick - Prism windowed value coder (#34830)

  #34842 merged May 5, 2025

• Fix Nexmark Dataflow V2 use runner v2

  #34823 merged May 5, 2025

• Update code-change-guide.md

  #34776 merged May 5, 2025

• use batch delete for GCS IO

  #34835 merged May 5, 2025

• Encode paneinfo with PaneInfoCoder.

  #34824 merged May 5, 2025

• Add OrderedList and Set state

  #34836 merged May 5, 2025

• Add powershell command 32307

  #34837 merged May 5, 2025

• Bump github.com/golang-cz/devslog from 0.0.11 to 0.0.13 in /sdks

  #34758 merged May 4, 2025

• Bump github.com/go-sql-driver/mysql from 1.9.1 to 1.9.2 in /sdks

  #34759 merged May 4, 2025

• Bump google.golang.org/api from 0.229.0 to 0.231.0 in /sdks

  #34784 merged May 4, 2025

• Prism windowed value coder

  #34830 merged May 4, 2025

• Fix loopback

  #34678 merged May 4, 2025

• Fix test retry on throttling

  #34786 merged May 2, 2025

• Add changes note about TFRecord support in beam yaml and rename integration test.

  #34635 merged May 2, 2025

• Add name to BQ File Loads lambda to ensure changes are update compat

  #34813 merged May 2, 2025

• Add name to BQ File Loads lambda to ensure changes are update compatible

  #34807 merged May 1, 2025

• Cherrypick Managed config validation revert

  #34805 merged May 1, 2025

• Fix issue link in trivial_inference.py

  #34803 merged May 1, 2025

• Revert "[ManagedIO] Fail expansion when encountering extra or unknown configuration"

  #34802 merged May 1, 2025

• Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.73 to 1.17.74 in /sdks

  #34785 merged May 1, 2025

27 Pull requests opened by 17 people

• Fix for emojis rendering issue for yaml examples (resolves #34770)

  #34795 opened May 1, 2025

• [IcebergIO] Add Iceberg SQL table provider and tests

  #34799 opened May 1, 2025

• yapf 0.43.0 migration

  #34801 opened May 1, 2025

• Revert "Unpin Dataflow legacy worker container for Nexmark test (#33224)"

  #34804 opened May 1, 2025

• [DO NOT MERGE] bigquery pico brainstorming

  #34806 opened May 1, 2025

• [BEAM-12164]: Make the spanner change stream connector metadata table ParentTokens column nullable.

  #34812 opened May 2, 2025

• Bump github.com/testcontainers/testcontainers-go from 0.36.0 to 0.37.0 in /sdks

  #34815 opened May 2, 2025

• use blob.exists to check the GCS file

  #34818 opened May 2, 2025

• Update Beam website to release 2.65.0

  #34821 opened May 2, 2025

• [IcebergIO] Support filter pushdown during reads

  #34827 opened May 3, 2025

• Add checkpoint during progress reporting.

  #34828 opened May 3, 2025

• Add Git commit SHA to artifacts using buildnumber-maven-plugin (#18227)

  #34833 opened May 4, 2025

• Bump github.com/nats-io/nats.go from 1.41.2 to 1.42.0 in /sdks

  #34839 opened May 5, 2025

• Update website documentation and add notebook example for python sdk RRIO

  #34841 opened May 5, 2025

• Update Iceberg table field documentation

  #34847 opened May 5, 2025

• fix direct vulnerabilities from jetty

  #34849 opened May 5, 2025

• Log warning instead of raising an exception for unsupported pickle option

  #34852 opened May 6, 2025

• Bump golang.org/x/text from 0.24.0 to 0.25.0 in /sdks

  #34853 opened May 6, 2025

• Enable certain Beam module compile with newer Java version

  #34858 opened May 6, 2025

• Fix parquet-avro vulnerability in io expansion service

  #34860 opened May 6, 2025

• [WIP] Update Error Prone for ThreadSafe analysis

  #34861 opened May 6, 2025

• Update trivial inference for Python 3.13

  #34870 opened May 6, 2025

• 34749 added cache for avro coder to reduce memory footprint

  #34873 opened May 6, 2025

• Cloudpickle deterministic

  #34874 opened May 7, 2025

• Fix flaky BoundedQueueExecutor test

  #34878 opened May 7, 2025

• Fix minor issues in various PTransformRunner classes

  #34881 opened May 7, 2025

• Reference managed-io page in our docs

  #34882 opened May 7, 2025

19 Issues closed by 8 people

• The pr-bot-pr-updates job is flaky

  #34731 closed May 7, 2025

• The pr-bot-new-prs job is flaky

  #34724 closed May 7, 2025

• [Feature Request]: Running Word-Count with Gradle for PowerShell

  #32307 closed May 7, 2025

• [Bug]: Reshuffle with default windows encodes PaneInfo with FastPrmitivesCoder

  #34826 closed May 6, 2025

• [Failing Test]: Tour of Beam Go Integration Test broken

  #34817 closed May 6, 2025

• The Generate issue report job is flaky

  #34855 closed May 6, 2025

• [Failing Test]: PostCommit Nexmark Dataflow Runner v2 actually runs on Dataflow legacy runner

  #34822 closed May 5, 2025

• [Bug]: Slow when deleting the temp files from BQ EXPORT

  #34834 closed May 5, 2025

• Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_inference_batch_latency_micro_secs:mean_inference_batch_latency_micro_secs

  #34810 closed May 5, 2025

• [Bug]: Prism failed on pipelines with reshuffle after windowing

  #34829 closed May 4, 2025

• [Bug]: External transforms cannot be instantiated with LOOPBACK mode.

  #34594 closed May 4, 2025

• [Bug]: ./gradlew :sdks:python:wordCount (Python environment setup check) fails for py3.9

  #34819 closed May 3, 2025

• [Failing Test]: TestGCSIORetry.test_retry_on_throttling failed due to unknown reasons

  #34736 closed May 2, 2025

• [Failing Test]: PostCommit Python Xlang IO Direct Failing

  #34796 closed May 2, 2025

• [Bug]: Managed transform incompatible schema

  #34797 closed May 2, 2025

• [Bug]: BQ File Loads breaks update compatability

  #34808 closed May 2, 2025

• The PostCommit Python Xlang IO Direct job is flaky

  #32809 closed May 2, 2025

• The PostCommit Python Xlang IO Dataflow job is flaky

  #33253 closed May 2, 2025

• The IcebergIO Managed Integration Tests on Dataflow job is flaky

  #34809 closed May 2, 2025

7 Issues opened by 7 people

• [Bug]: ValueError: Resource hint min_ram has invalid value None when setting resource_hints in StandardOptions

  #34879 opened May 7, 2025

• [Feature Request]: Add Support for Python 3.13

  #34869 opened May 6, 2025

• [Bug]: Spanner schemas with ARRAY<STRING(xxx)> fail to parse

  #34863 opened May 6, 2025

• [Bug]: Samza runner fails to run a pipeline with Custom Window followed by Reshuffle

  #34831 opened May 3, 2025

• [Feature Request]: make some of the dependencies optional/behind a feature flag

  #34816 opened May 2, 2025

• Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs

  #34811 opened May 1, 2025

• [Bug]: Workflow hangs during dependency installation with Pip 25.1

  #34798 opened May 1, 2025

54 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• add graceful restart mechanism for GetWorkStream to prevent DEADLINE_…

  #34367 commented on May 5, 2025 • 15 new comments

• Streamline non-cached state backed iterable.

  #34746 commented on May 7, 2025 • 6 new comments

• use WindmillChannelFactory to control what types of channels to generate

  #34653 commented on May 7, 2025 • 5 new comments

• complete implementation of open ai text embedding with test #new

  #34700 commented on May 7, 2025 • 4 new comments

• Concat protos in BQStorageWriteAPI - solve edge cases during mering of nested repeated fields

  #34436 commented on May 6, 2025 • 3 new comments

• Clean up GCP Resources (Pubsub)

  #34141 commented on May 7, 2025 • 2 new comments

• unbounded PCollections writes to files support in iobase derived IOs: AvroIO, ParquetIO , TextIO, TFRecordIO

  #34777 commented on May 7, 2025 • 1 new comment

• Improve failure message when passing Pipeline object / PBegin objects…

  #34716 commented on May 2, 2025 • 1 new comment

• Enabling long-running jobs to use federated STS assume role authentication for AWS resources.

  #34440 commented on May 5, 2025 • 0 new comments

• sdks/python: enrich data with CloudSQL

  #34398 commented on May 6, 2025 • 0 new comments

• Bump @octokit/plugin-paginate-rest, @actions/github and @octokit/rest in /scripts/ci/pr-bot

  #34377 commented on May 5, 2025 • 0 new comments

• [Java]Add Map Type Support to JsonToRow Transformer

  #34347 commented on May 1, 2025 • 0 new comments

• Update pypi documentation 30145

  #34329 commented on May 6, 2025 • 0 new comments

• Add Triton Inference Server Support

  #34252 commented on May 7, 2025 • 0 new comments

• feat:large-row-skip-in-bigtable | added experimental options to skip …

  #34245 commented on May 7, 2025 • 0 new comments

• [Java] Ensure Pipeline Execution Requires Configuration Options or Logs Warning

  #34220 commented on May 2, 2025 • 0 new comments

• [Bug][Prism]: Prism gets stuck when trying to flatten 2 unbounded pcollections

  #33815 commented on May 1, 2025 • 0 new comments

• SnowflakeIO: filter on db and schema when searching for existing table

  #34486 commented on May 7, 2025 • 0 new comments

• ✨ Upgrade sidepanel extension to JupyterLab 4.x compatibility [DO NOT MERGE]

  #34495 commented on May 2, 2025 • 0 new comments

• Add PyTorch DistilBERT Sentiment Analysis streaming pipeline for ML Benchmarks

  #34577 commented on May 6, 2025 • 0 new comments

• Python PTransform wrapper for AWS SQS

  #34581 commented on May 1, 2025 • 0 new comments

• Support customizing how built-in types are pickled for cloudpickle

  #34699 commented on May 6, 2025 • 0 new comments

• Attempt prism for pipelines with unbounded PCollections.

  #34721 commented on May 6, 2025 • 0 new comments

• 34749 reflect datum factory cache

  #34750 commented on May 6, 2025 • 0 new comments

• Fix PreCommit YAML Xlang Direct job

  #34762 commented on May 6, 2025 • 0 new comments

• Drain Mode as WindowedValue extension

  #34764 commented on May 2, 2025 • 0 new comments

• Make dill optional and fix coders.

  #34769 commented on May 5, 2025 • 0 new comments

• [DO NOT MERGE] Run all PostCommit and PreCommit Tests against Release Branch

  #34794 commented on May 2, 2025 • 0 new comments

• [Feature Request]: Allowed write to custom sink from unbounded source

  #25598 commented on May 1, 2025 • 0 new comments

• [Bug]: Support error handling in PyTransform

  #32332 commented on May 2, 2025 • 0 new comments

• Add documentation in Portable Runner to submit job in Java SDK

  #20617 commented on May 3, 2025 • 0 new comments

• [Bug]: apache_beam.io.gcp.bigquery.ReadFromBigQuery with EXPORT is slow when handling large amount of data

  #34792 commented on May 4, 2025 • 0 new comments

• [Bug]: YAML examples don't render emojis

  #34770 commented on May 5, 2025 • 0 new comments

• [Feature Request]: Deterministic serialization of DoFns

  #34410 commented on May 6, 2025 • 0 new comments

• [Bug]: AvroCoder uses a lot of memory because it keeps instantiating the same DatumReader and DatumWriter

  #34749 commented on May 6, 2025 • 0 new comments

• The PostCommit Python Dependency job is flaky

  #30799 commented on May 6, 2025 • 0 new comments

• [yaml]: Normalize BigtableIO

  #28672 commented on May 6, 2025 • 0 new comments

• [Feature Request]: Add a basic doc explaining Beam's security model

  #30911 commented on May 6, 2025 • 0 new comments

• Write just one file per window with WriteToFiles transform

  #20676 commented on May 6, 2025 • 0 new comments

• [Bug]: Java expansion service does not respect the namespace for all environments.

  #30876 commented on May 7, 2025 • 0 new comments

• The PreCommit Python Coverage job is flaky

  #30813 commented on May 7, 2025 • 0 new comments

• Replace StorageV1 client with GCS client - Draft

  #28733 commented on May 4, 2025 • 0 new comments

• Adding Google Storage Requester pays feature to Golang SDK.

  #33236 commented on May 1, 2025 • 0 new comments

• Remove type suppressions from MetricsPusher and MetricsSink

  #33918 commented on May 7, 2025 • 0 new comments

• Bump @octokit/request-error, @actions/github and @octokit/rest in /scripts/ci/pr-bot

  #33998 commented on May 5, 2025 • 0 new comments

• Switch to use registerFileSystemsOnce for SerializablePipelineOptions constructor

  #34028 commented on May 2, 2025 • 0 new comments

• [BEAM-6394] Add support to write protobuf data using ProtoParquetReader

  #34063 commented on May 7, 2025 • 0 new comments

• Fix Docker build error by adding fallback for python3.12-distutils

  #34144 commented on May 2, 2025 • 0 new comments

• Bump @octokit/plugin-paginate-rest and @octokit/rest in /scripts/ci/issue-report

  #34167 commented on May 2, 2025 • 0 new comments

• Rethrowing Exception from CassandraIO's ReadFn

  #34191 commented on May 6, 2025 • 0 new comments

• Fix ProtoCoder NoSuchMethodException

  #34194 commented on May 2, 2025 • 0 new comments

• [KafkaIO] Remove duplicate offset in range check

  #34201 commented on May 2, 2025 • 0 new comments

• Add support for top-level table properties table creation

  #34205 commented on May 6, 2025 • 0 new comments

• [Java] Add parsedData to Hl7v2Message and Update HL7v2IO Docs

  #34213 commented on May 6, 2025 • 0 new comments