A pipeline takes data sources, such as Firestore collections or collection groups, and applies + * a series of stages that are chained together. Each stage takes the output from the previous stage + * (or the data source) and produces an output for the next stage (or as the final output of the + * pipeline). + * + *
Expressions from {@link com.google.cloud.firestore.pipeline.expressions} can be used within + * each stages to filter and transform data through the stage. + * + *
NOTE: The chained stages do not prescribe exactly how Firestore will execute the pipeline. + * Instead, Firestore only guarantees that the result is the same as if the chained stages were + * executed in order. + * + *
Usage Examples: + * + *
{@code
+ * Firestore firestore; // A valid firestore instance.
+ *
+ * // Example 1: Select specific fields and rename 'rating' to 'bookRating'
+ * List results1 = firestore.pipeline()
+ * .collection("books")
+ * .select("title", "author", Field.of("rating").as("bookRating"))
+ * .execute()
+ * .get();
+ *
+ * // Example 2: Filter documents where 'genre' is "Science Fiction" and 'published' is after 1950
+ * List results2 = firestore.pipeline()
+ * .collection("books")
+ * .where(and(eq("genre", "Science Fiction"), gt("published", 1950)))
+ * .execute()
+ * .get();
+ * // Same as above but using methods on expressions as opposed to static functions.
+ * results2 = firestore.pipeline()
+ * .collection("books")
+ * .where(and(Field.of("genre").eq("Science Fiction"), Field.of("published").gt(1950)))
+ * .execute()
+ * .get();
+ *
+ * // Example 3: Calculate the average rating of books published after 1980
+ * List results3 = firestore.pipeline()
+ * .collection("books")
+ * .where(gt("published", 1980))
+ * .aggregate(avg("rating").as("averageRating"))
+ * .execute()
+ * .get();
+ * }
+ */
+@BetaApi
+public final class Pipeline {
+ private static Logger logger = Logger.getLogger(Pipeline.class.getName());
+ private final ImmutableListThis stage allows you to compute values on-the-fly based on existing data from previous + * stages or constants. You can use this to create new fields or overwrite existing ones (if there + * is name overlaps). + * + *
The added fields are defined using {@link Selectable} expressions, which can be: + * + *
Example: + * + *
{@code
+ * firestore.pipeline().collection("books")
+ * .addFields(
+ * Field.of("rating").as("bookRating"), // Rename 'rating' to 'bookRating'
+ * add(5, Field.of("quantity")).as("totalCost") // Calculate 'totalCost'
+ * );
+ * }
+ *
+ * @param fields The fields to add to the documents, specified as {@link Selectable} expressions.
+ * @return A new Pipeline object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline addFields(Selectable... fields) {
+ return new Pipeline(
+ this.db,
+ ImmutableList.The selected fields are defined using {@link Selectable} expressions, which can be: + * + *
If no selections are provided, the output of this stage is empty. Use {@link + * com.google.cloud.firestore.Pipeline#addFields(Selectable...)} instead if only additions are + * desired. + * + *
Example: + * + *
{@code
+ * firestore.pipeline().collection("books")
+ * .select(
+ * Field.of("name"),
+ * Field.of("address").toUppercase().as("upperAddress"),
+ * );
+ * }
+ *
+ * @param selections The fields to include in the output documents, specified as {@link
+ * Selectable} expressions.
+ * @return A new Pipeline object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline select(Selectable... selections) {
+ return new Pipeline(
+ this.db,
+ ImmutableList.If no selections are provided, the output of this stage is empty. Use {@link + * com.google.cloud.firestore.Pipeline#addFields(Selectable...)} instead if only additions are + * desired. + * + *
Example: + * + *
{@code
+ * firestore.collection("books")
+ * .select("name", "address");
+ *
+ * // The above is a shorthand of this:
+ * firestore.pipeline().collection("books")
+ * .select(Field.of("name"), Field.of("address"));
+ * }
+ *
+ * @param fields The name of the fields to include in the output documents.
+ * @return A new Pipeline object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline select(String... fields) {
+ return new Pipeline(
+ this.db,
+ ImmutableList.This stage allows you to apply conditions to the data, similar to a "WHERE" clause in SQL. + * You can filter documents based on their field values, using implementions of {@link + * FilterCondition}, typically including but not limited to: + * + *
Example: + * + *
{@code
+ * firestore.pipeline().collection("books")
+ * .where(
+ * and(
+ * gt("rating", 4.0), // Filter for ratings greater than 4.0
+ * Field.of("genre").eq("Science Fiction") // Equivalent to gt("genre", "Science Fiction")
+ * )
+ * );
+ * }
+ *
+ * @param condition The {@link FilterCondition} to apply.
+ * @return A new Pipeline object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline where(FilterCondition condition) {
+ return new Pipeline(
+ this.db, ImmutableList.This stage is useful for implementing pagination in your pipelines, allowing you to retrieve + * results in chunks. It is typically used in conjunction with {@link #limit(int)} to control the + * size of each page. + * + *
Example: + * + *
{@code
+ * // Retrieve the second page of 20 results
+ * firestore.pipeline().collection("books")
+ * .sort(Field.of("published").descending())
+ * .offset(20) // Skip the first 20 results
+ * .limit(20); // Take the next 20 results
+ * }
+ *
+ * @param offset The number of documents to skip.
+ * @return A new Pipeline object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline offset(int offset) {
+ return new Pipeline(
+ this.db, ImmutableList.This stage is particularly useful when you want to retrieve a controlled subset of data from + * a potentially large result set. It's often used for: + * + *
Example: + * + *
{@code
+ * // Limit the results to the top 10 highest-rated books
+ * firestore.pipeline().collection("books")
+ * .sort(Field.of("rating").descending())
+ * .limit(10);
+ * }
+ *
+ * @param limit The maximum number of documents to return.
+ * @return A new Pipeline object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline limit(int limit) {
+ return new Pipeline(
+ this.db, ImmutableList.This stage allows you to calculate aggregate values over a set of documents. You define the + * aggregations to perform using {@link ExprWithAlias} expressions which are typically results of + * calling {@link Expr#as(String)} on {@link Accumulator} instances. + * + *
Example: + * + *
{@code
+ * // Calculate the average rating and the total number of books
+ * firestore.pipeline().collection("books")
+ * .aggregate(
+ * Field.of("rating").avg().as("averageRating"),
+ * countAll().as("totalBooks")
+ * );
+ * }
+ *
+ * @param accumulators The {@link ExprWithAlias} expressions, each wrapping an {@link Accumulator}
+ * and provide a name for the accumulated results.
+ * @return A new Pipeline object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline aggregate(ExprWithAliasThis stage allows you to calculate aggregate values over a set of documents, optionally + * grouped by one or more fields or functions. You can specify: + * + *
Example: + * + *
{@code
+ * // Calculate the average rating for each genre.
+ * firestore.pipeline().collection("books")
+ * .aggregate(
+ * Aggregate
+ * .withAccumulators(avg("rating").as("avg_rating"))
+ * .withGroups("genre"));
+ * }
+ *
+ * @param aggregate An {@link Aggregate} object that specifies the grouping fields (if any) and
+ * the aggregation operations to perform.
+ * @return A new {@code Pipeline} object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline aggregate(Aggregate aggregate) {
+ return new Pipeline(
+ this.db, ImmutableList.This stage run through the results from previous stages to include only results with unique + * combinations of values for the specified fields and produce these fields as the output. + * + *
Example: + * + *
{@code
+ * // Get a list of unique genres.
+ * firestore.pipeline().collection("books")
+ * .distinct("genre");
+ * }
+ *
+ * @param fields The fields to consider when determining distinct values.
+ * @return A new {@code Pipeline} object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline distinct(String... fields) {
+ return new Pipeline(
+ this.db,
+ ImmutableList.This stage run through the results from previous stages to include only results with unique + * combinations of {@link Expr} values ({@link Field}, {@link Function}, etc). + * + *
The parameters to this stage are defined using {@link Selectable} expressions, which can be: + * + *
Example: + * + *
{@code
+ * // Get a list of unique author names in uppercase and genre combinations.
+ * firestore.pipeline().collection("books")
+ * .distinct(toUppercase(Field.of("author")).as("authorName"), Field.of("genre"))
+ * .select("authorName");
+ * }
+ *
+ * @param selectables The {@link Selectable} expressions to consider when determining distinct
+ * value combinations.
+ * @return A new {@code Pipeline} object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline distinct(Selectable... selectables) {
+ return new Pipeline(
+ this.db,
+ ImmutableList.This stage adds a "nearest neighbor search" capability to your pipelines. Given a field that + * stores vectors and a target vector, this stage will identify and return the inputs whose vector + * field is closest to the target vector, using the parameters specified in `options`. + * + *
Example: + * + *
{@code
+ * // Find books with similar "topicVectors" to the given targetVector
+ * firestore.pipeline().collection("books")
+ * .findNearest("topicVectors", targetVector,
+ * FindNearestOptions
+ * .withLimitAndMeasure(10, DistanceMeasure.cosine())
+ * .withOutputField("distance"));
+ * }
+ *
+ * @param fieldName The name of the field containing the vector data. This field should store
+ * {@link VectorValue}.
+ * @param vector The target vector to compare against.
+ * @param options Configuration options for the nearest neighbor search, such as the distance
+ * metric to use.
+ * @return A new {@code Pipeline} object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline findNearest(
+ String fieldName, double[] vector, FindNearest.FindNearestOptions options) {
+ return findNearest(Field.of(fieldName), vector, options);
+ }
+
+ /**
+ * Performs vector distance (similarity) search with given parameters to the stage inputs.
+ *
+ * This stage adds a "nearest neighbor search" capability to your pipelines. Given an + * expression that evaluates to a vector and a target vector, this stage will identify and return + * the inputs whose vector expression is closest to the target vector, using the parameters + * specified in `options`. + * + *
Example: + * + *
{@code
+ * // Find books with similar "topicVectors" to the given targetVector
+ * firestore.pipeline().collection("books")
+ * .findNearest(Field.of("topicVectors"), targetVector,
+ * FindNearestOptions
+ * .withLimitAndMeasure(10, DistanceMeasure.cosine())
+ * .withOutputField("distance"));
+ * }
+ *
+ * @param property The expression that evaluates to a vector value using the stage inputs.
+ * @param vector The target vector to compare against.
+ * @param options Configuration options for the nearest neighbor search, such as the distance
+ * metric to use.
+ * @return A new {@code Pipeline} object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline findNearest(
+ Expr property, double[] vector, FindNearest.FindNearestOptions options) {
+ // Implementation for findNearest (add the FindNearest stage if needed)
+ return new Pipeline(
+ this.db,
+ ImmutableList.This stage allows you to order the results of your pipeline. You can specify multiple {@link + * Ordering} instances to sort by multiple fields in ascending or descending order. If documents + * have the same value for a field used for sorting, the next specified ordering will be used. If + * all orderings result in equal comparison, the documents are considered equal, and the order is + * unspecified. + * + *
Example: + * + *
{@code
+ * // Sort books by rating in descending order, and then by title in ascending order for books
+ * // with the same rating, with density required and truncation disabled.
+ * firestore.pipeline().collection("books")
+ * .sort(
+ * Arrays.asList(Ordering.of("rating").descending(), Ordering.of("title")),
+ * Sort.Density.REQUIRED,
+ * Sort.Truncation.DISABLED,
+ * 10);
+ * }
+ *
+ * @param orders One or more {@link Ordering} instances specifying the sorting criteria.
+ * @param density Specifies the index density semantics. See {@link
+ * com.google.cloud.firestore.pipeline.stages.Sort.Density} for more information on density
+ * sorting.
+ * @param truncation Specifies the index truncation semantics. See {@link
+ * com.google.cloud.firestore.pipeline.stages.Sort.Truncation} for more information on
+ * truncation options.
+ * @return A new {@code Pipeline} object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline sort(ListThis stage allows you to order the results of your pipeline. You can specify multiple {@link + * Ordering} instances to sort by multiple fields in ascending or descending order. If documents + * have the same value for a field used for sorting, the next specified ordering will be used. If + * all orderings result in equal comparison, the documents are considered equal and the order is + * unspecified. + * + *
Example: + * + *
{@code
+ * // Sort books by rating in descending order, and then by title in ascending order for books with the same rating
+ * firestore.pipeline().collection("books")
+ * .sort(
+ * Ordering.of("rating").descending(),
+ * Ordering.of("title") // Ascending order is the default
+ * );
+ * }
+ *
+ * @param orders One or more {@link Ordering} instances specifying the sorting criteria.
+ * @return A new {@code Pipeline} object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline sort(Ordering... orders) {
+ return sort(Arrays.asList(orders), Sort.Density.UNSPECIFIED, Sort.Truncation.UNSPECIFIED);
+ }
+
+ /**
+ * Adds a generic stage to the pipeline.
+ *
+ * This method provides a flexible way to extend the pipeline's functionality by adding custom + * stages. Each generic stage is defined by a unique `name` and a set of `params` that control its + * behavior. + * + *
Example (Assuming there is no "where" stage available in SDK): + * + *
{@code
+ * // Assume we don't have a built-in "where" stage
+ * Map whereParams = new HashMap<>();
+ * whereParams.put("condition", Field.of("published").lt(1900));
+ *
+ * firestore.pipeline().collection("books")
+ * .genericStage("where", Lists.newArrayList(Field.of("published").lt(1900))) // Custom "where" stage
+ * .select("title", "author");
+ * }
+ *
+ * @param name The unique name of the generic stage to add.
+ * @param params A map of parameters to configure the generic stage's behavior.
+ * @return A new {@code Pipeline} object with this stage appended to the stage list.
+ */
+ @BetaApi
+ public Pipeline genericStage(String name, List