Bugfix release

• Un-deprecate macro annotations to avoid magnolify serialization issues, deprecate typed avro and BQ methods by @kellen in #5846
• Check if org.apache.avro.SERIALIZABLE_CLASSES in scio-smb Parquet writes by @clairemcginty in #5847
• Null check on org.apache.avro.SERIALIZABLE_CLASSES prop by @clairemcginty in #5829

🚀 Enhancements

• scalafix for 0.15 by @kellen in #5822

📗 Documentation

• Add migration guide, deprecations for 0.15 by @kellen in #5845

🌱 Dependency Updates

• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.27.1 by @clairemcginty in #5833
• Update sbt-bom to 1.0.31 by @clairemcginty in #5834
• Update pprint to 0.9.6 by @clairemcginty in #5835
• Update sparkey to 3.5.0 by @clairemcginty in #5836
• Update scalacheck to 1.18.1 by @clairemcginty in #5840
• Update sbt-scoverage to 2.4.3 by @clairemcginty in #5843
• Update sbt-scalafix to 0.14.5 by @clairemcginty in #5830
• Upgrade Sparkey to 3.5.0 by @spkrka in #5827

Full Changelog: v0.15.0...v0.15.1

Scio 0.15 adds support for Iceberg, beam's managed transforms, and Avro 1.11's java-class annotations.

🚀 Enhancements

• Magnolify API by @kellen in #5286
• (fix #5067) Drop Java 8 Targeting by @clairemcginty in #5203
• Managed + Iceberg IO by @kellen in #5494
• Support writing extra metadata in scio-parquet by @clairemcginty in #5411
• Update IcebergIO read/write parameters by @kellen in #5823
• Add integration test for Magnolify TableRowType compatibility by @clairemcginty in #5818
• Support Avro 1.11 java-class annotations in scio-parquet by @clairemcginty in #5821
• Add ParquetMetadataDoFn by @kellen in #5820
• v0.15.0 by @kellen in #5609
• Remove stale comments, make some previously-deprecated methods private by @kellen in #5584

🏗️ Build Improvements

• Vendor Socco by @kellen in #5811
• Exclude dependabot from updating ci.yml by @kellen in #5813
• Move CI env definition by @kellen in #5826
• Remove developers list by @kellen in #5548
• Fix BT integration tests by @kellen in #5606
• Fix BT integration tests mark ii by @kellen in #5608
• Add headers, remove unused by @kellen in #5619

🐛 Bug Fixes

• Fix Iceberg integration test, defaults by @kellen in #5825
• Fix seed setting in samplers by @kellen in #5816
• Add check for java 17 SCollectionMatcher (fix #5483) by @kellen in #5605

🌱 Dependency Updates

• Upgrade to Avro 1.11.5 by @clairemcginty in #5799
• Update sbt to 1.11.7, sbt-mdoc to 2.8.0, sbt-java-formatter to 0.10.0… by @kellen in #5815
• Bump to magnolify 0.9.2 by @kellen in #5824
• Update scala-compiler, scala-library, ... to 2.13.17 by @clairemcginty in #5786
• Bump coursier/cache-action from 6 to 7 by @dependabot[bot] in #5797
• Update sbt-typelevel to 0.8.2 by @clairemcginty in #5808
• Update sbt-jmh to 0.4.8 by @clairemcginty in #5809
• Update mysql-connector-j to 9.5.0 by @clairemcginty in #5804
• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.27.0 by @clairemcginty in #5802
• Update jedis to 7.0.0 by @clairemcginty in #5810
• Update sbt-scoverage to 2.4.1 by @clairemcginty in #5807
• Update sbt-bom to 1.0.30 by @clairemcginty in #5803
• Update testcontainers-scala-elasticsearch, ... to 0.43.6 by @clairemcginty in #5801
• Update elasticsearch-java to 8.19.7 by @clairemcginty in #5800
• Update magnolify to 0.8.0, tensorflow to 1.0.0, caffeine to 3.2.0 by @kellen in #5604
• magnolify 0.9 by @kellen in #5812

Full Changelog: v0.14.20...v0.15.0

This release includes Beam 2.68.0 support. It also upgrades the major version of protobuf-java to 4.x.

🐛 Bug Fixes

• Fix serialization error when sortMergeCoGroup has secondary keys by @charles-tan in #5768

🏗️ Build Improvements

• Add test assertion for OverrideTypeProvider field name by @clairemcginty in #5766
• Fix mergeStrategy for scio-repl by @clairemcginty in #5794

🌱 Dependency Updates

• Upgrade to Beam 2.68 by @clairemcginty in #5769
• Upgrade protobuf-java and protoc to 4.x by @clairemcginty in #5701
• Update parquet-avro, parquet-column, ... to 1.16.0 by @clairemcginty in #5762
• Update jedis to 6.2.0 by @clairemcginty in #5763
• Update elasticsearch-java to 8.19.4 by @clairemcginty in #5770
• Update sbt, sbt-dependency-tree, ... to 1.11.6 by @clairemcginty in #5778
• Update jna to 5.18.1 by @clairemcginty in #5776
• Update scalafmt-core to 3.10.1 by @clairemcginty in #5791
• Update munit to 1.2.1 by @clairemcginty in #5789
• Update sbt-scalafix to 0.14.4 by @clairemcginty in #5782
• Update elasticsearch-java to 8.19.5 by @clairemcginty in #5783
• Update circe-core, circe-generic, ... to 0.14.15 by @clairemcginty in #5774
• Update metrics-core to 4.2.37 by @clairemcginty in #5775
• Update sbt-ghpages to 0.9.0 by @clairemcginty in #5771
• Update sbt-bom to 1.0.29 by @clairemcginty in #5772
• Bump netty to 4.1.128 by @clairemcginty in #5793

Full Changelog: v0.14.19...v0.14.20

v0.14.19

This release includes Beam 2.67.0 support.

🐛 Bug Fixes

🚀 Enhancements

• Add option to enable flow control for BigTable writes by @clairemcginty in #5747
• Unblock usage of gcs-connector 3.x by @clairemcginty in #5730
• Add experimental support for vectoredRead-compatible stream in scio-parquet by @clairemcginty in #5732

🏗️ Build Improvements

• Clean unused MiMa filters by @clairemcginty in #5729
• Bump actions/setup-java from 4 to 5 by @dependabot[bot] in #5748
• Bump actions/download-artifact from 3 to 5 by @dependabot[bot] in #5733

🌱 Dependency Updates

• Upgrade Beam to 2.67 by @clairemcginty in #5735
• Update sbt, sbt-dependency-tree, ... to 1.11.5 by @clairemcginty in #5752
• Update sbt-scalafix to 0.14.3 by @clairemcginty in #5685
• Update sbt-bom to 1.0.27 by @clairemcginty in #5738
• Update sbt-unidoc to 0.6.0 by @clairemcginty in #5750
• Update fansi to 0.5.1 by @clairemcginty in #5739
• Update mssql-jdbc to 13.2.0.jre11 by @clairemcginty in #5741
• Update mysql-connector-j to 9.4.0 by @clairemcginty in #5742
• Update sbt, sbt-dependency-tree, ... to 1.11.4 by @clairemcginty in #5744
• Update jedis to 6.1.0 by @clairemcginty in #5746
• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.25.3 by @clairemcginty in #5737
• Update elasticsearch-java to 8.19.2 by @clairemcginty in #5749
• Update scalafmt-core to 3.9.9 by @clairemcginty in #5745

Full Changelog: v0.14.18...v0.14.19

v0.14.18

This release includes Beam 2.66.0 support.

🐛 Bug Fixes

• (fix #5713) Fix BigQueryType#fromAvro on missing fields for Avro 1.11 by @clairemcginty in #5714

🚀 Enhancements

• Enable deterministic sampling with SCollection.sample by @crystall-spotify in #5709

📗 Documentation

• Update release docs by @clairemcginty in #5698
• Update Apache Beam version matrix in documentation by @charles-tan in #5727

🏗️ Build Improvements

• Allow manual trigger of integration tests in CI by @clairemcginty in #5706
• Warm up BQ service calls in IT tests by @clairemcginty in #5708
• Use unique name for manually triggered CI by @clairemcginty in #5707

🌱 Dependency Updates

• Bump beam to 2.66.0 by @charles-tan in #5715
• Update metrics-core to 4.2.32 by @clairemcginty in #5689
• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.25.1 by @clairemcginty in #5688
• Update sbt-avro, sbt-avro-compiler-bridge to 4.0.1 by @clairemcginty in #5687
• Update sbt, sbt-dependency-tree, ... to 1.11.3 by @clairemcginty in #5722
• Update circe-core, circe-generic, ... to 0.14.14 by @clairemcginty in #5711
• Update sbt-protoc to 1.0.8 by @clairemcginty in #5710
• Update sonatype host to central portal by @clairemcginty in #5717
• Update scalafmt-core to 3.9.8 by @clairemcginty in #5724
• Update sbt-mdoc to 2.7.2 by @clairemcginty in #5723
• Update mssql-jdbc to 12.10.1.jre11 by @clairemcginty in #5720
• Update elasticsearch-java to 8.18.3 by @clairemcginty in #5719

New Contributors

• @crystall-spotify made their first contribution in #5709

Full Changelog: v0.14.17...v0.14.18

🐛 Bug Fixes

• Fix Test IO for SMB multi-join with secondary key by @clairemcginty in #5676

🌱 Dependency Updates

• Update parquet-avro, parquet-column, ... to 1.15.2 by @clairemcginty in #5681
• Update mysql-connector-j to 9.3.0 by @clairemcginty in #5679
• Update jedis to 6.0.0 by @clairemcginty in #5684
• Update scalafmt-core to 3.9.5 by @clairemcginty in #5683
• Update munit to 1.1.1 by @clairemcginty in #5682
• Update circe-core, circe-generic, ... to 0.14.13 by @clairemcginty in #5680
• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.24.2 by @clairemcginty in #5678
• Update elasticsearch-java to 8.18.0 by @clairemcginty in #5677

Full Changelog: v0.14.16...v0.14.17

🐛 Bug Fixes

• Fix secondary key getter in ParquetBucketMetadata by @clairemcginty in #5667
• (fix #5665) BigQueryType#toAvro should compile for nested case classes with repeat field names by @clairemcginty in #5668

🚀 Enhancements

• Add collect method to SCollectionWithSideInput by @charles-tan in #5666

🌱 Dependency Updates

• Update sbt-mdoc to 2.7.1 by @clairemcginty in #5671
• Update elasticsearch-java to 8.17.5 by @clairemcginty in #5669

New Contributors

• @charles-tan made their first contribution in #5666

Full Changelog: v0.14.15...v0.14.16

This release includes Beam 2.64.0 support.

Note: This release reverts the change introduced in the 0.14.11 release that made Avro the default typed-BigQuery representation, due to regressions. Typed BQ reads and writes will once again default to using TableRow in the underlying translation layer, rather than GenericRecord, but can be configured to use Avro on an opt-in basis; for more information, see the typed BigQuery docs.

🐛 Bug Fixes

• Fix BigQueryType.toAvro performance bug + add BigQueryType benchmarks by @jto and @clairemcginty in #5655

🚀 Enhancements

• Parameterize BQ write format; set TableRow as default by @clairemcginty in #5643

📗 Documentation

• Fix link in Scio-Unit-Tests.md by @RickardZwahlen in #5641
• Update BigQuery Typed API docs by @clairemcginty in #5664

🏗️ Build Improvements

• Add integration tests for BQ OverrideTypeProvider by @clairemcginty in #5645

🌱 Dependency Updates

• Upgrade to Beam 2.64.0 by @clairemcginty in #5658
• Update parquet-avro, parquet-column, ... to 1.15.1 by @clairemcginty in #5650
• Update sbt, sbt-dependency-tree, ... to 1.10.11 by @clairemcginty in #5651
• Update sbt-mdoc to 2.6.5 by @clairemcginty in #5652
• Update scalafmt-core to 3.9.4 by @clairemcginty in #5653
• Update circe-core, circe-generic, ... to 0.14.12 by @clairemcginty in #5648
• Update scalac-options to 0.1.8 by @clairemcginty in #5654
• Update mssql-jdbc to 12.10.0.jre11 by @clairemcginty in #5647
• Update testcontainers-scala-elasticsearch, ... to 0.43.0 by @clairemcginty in #5646
• Update jna to 5.17.0 by @clairemcginty in #5649
• Update elasticsearch-java to 8.17.4 by @clairemcginty in #5660
• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.24.1 by @clairemcginty in #5661

New Contributors

• @RickardZwahlen made their first contribution in #5641

Full Changelog: v0.14.14...v0.14.15

🐛 Bug Fixes

• Fall back to TableRow representation for typed BQ writes containing a LocalDateTime field by @clairemcginty in #5623

📗 Documentation

• Document Scala Steward usage on Github site by @clairemcginty in #5639

🏗️ Build Improvements

• Update TypedBigQueryIT to enumerate all failure/success combinations by @clairemcginty in #5640

🌱 Dependency Updates

• Update sbt-scoverage to 2.3.1 by @clairemcginty in #5637
• Update scalafmt-core to 3.9.2 by @clairemcginty in #5636
• Update sbt, sbt-dependency-tree, ... to 1.10.10 by @clairemcginty in #5634
• Update cassandra-all to 3.11.19 by @clairemcginty in #5631
• Update elasticsearch-java to 8.17.3 by @clairemcginty in #5625
• Update sbt-scalafix to 0.14.2 by @clairemcginty in #5624
• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.23.1 by @clairemcginty in #5627
• Update shapeless to 2.3.13 by @clairemcginty in #5626
• Update neo4j-java-driver to 4.4.20 by @clairemcginty in #5633
• Update sbt-mdoc to 2.6.4 by @clairemcginty in #5635
• Update kantan.codecs to 0.6.0 by @clairemcginty in #5628

Full Changelog: v0.14.13...v0.14.14

Includes Beam 2.63.0 support and bugfixes for BigQuery Storage API typed writes.

📗 Documentation

• Prepare for 0.14.13 release by @clairemcginty in #5615

🐛 Bug Fixes

• Revert 0.14.12 patch by @clairemcginty in #5610
• Fix bq Avro format issue for nested records by @clairemcginty in #5611
• Use Avro 1.8-compatible APIs in BigQueryType macro by @clairemcginty in #5616
• Fix REPL assembly by @clairemcginty in #5614

🌱 Dependency Updates

• Upgrade to Beam 2.63.0 by @clairemcginty in #5613

🏗️ Build Improvements

• Be explicit about using Protobuf 3 in Scio by @clairemcginty in #5618

Full Changelog: v0.14.12...v0.14.13