Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Releases: meilisearch/meilisearch

v1.45.0 🦒

26 May 08:38
bc80723

Choose a tag to compare

Meilisearch v1.45.0 mainly improves indexing performance when changing settings and also improves document fetch performance.

✨ Enhancement

  • Autobatch deletions by filter with additions by @Kerollmops in #6389
    While Meilisearch tries to improve indexing speed when users add documents and mix those additions with deletions via a filter, it is still an anti-pattern to interleave both. It is recommended to delete documents by ID whenever possible, as the engine is optimized to merge them, thereby drastically speeding up indexation.
  • Improve settings indexing performance & visibility
    We are introducing support for more settings in the nez settings indexer. If you find any bugs, please report them on GitHub. You can disable the new settings indexer by setting the environment variable like this: MEILI_EXPERIMENTAL_NO_EDITION_2024_FOR_SETTINGS=true or use the equivalent dedicated CLI parameter.
    • Support non-extracting parameters by @Kerollmops in #6393
      Changing one of the following settings is now directly handled by the new settings indexer: displayed fields, synonyms, the primary key, authorize typos, min word len one and two typos, max values per facet, sort facet values by, pagination max total hits, search cut off, chat, and foreign keys.
    • Support the global facet search by @Kerollmops in #6390
      Meilisearch can use the new settings indexer when the settings change the facet search root boolean parameter.

🪲 Bug Fixes

  • Fetching documents no longer blocks the actix worker @dureuill in #6402
  • Fix an internal error when changing the binary quantization by @Kerollmops in #6396
    We fixed an issue where users were changing the binary-quantized boolean in embedder configurations. The change corrupted the database, making it impossible to change the quantization in the future. Users had to create a binary-quantized embedder from scratch, or they could never change it again.

🔩 Miscellaneous

Full Changelog: v1.44.0...v1.45.0

v1.44.0

18 May 08:20
6294894

Choose a tag to compare

Meilisearch v1.44.0 adds remote federated facet search, indexing performance improvements, and other improvements and bugfixes. It also contains a couple of breaking changes, detailed below.

Breaking changes

  • When using the network experimental feature, with sharding enabled (leader is not null in the network configuration), POST /indexes/{indexUid}/facet-search calls now default to a remote federated facet search, fetching and merging results from all shards in the network.
  • The timeout for calling an external REST embedder at search time is now tied to the searchCutOffMs defined in the index, rather than fixed.
    • If you observe missing semantic results after upgrading, or HTTP 500 errors for pure semantic search, please increase the value of the search cutoff in the index settings.
    • By @dureuill in #6377

🌈 Improvements

Remote facet search

Meilisearch now has the ability to search across all shards of a network during facet search via the existing dedicated facet search route.

  • If you are using the network experimental feature and have a leader defined for your network, remote calls are now the default for calls to POST /indexes/{indexUid}/facet-search
  • The behavior can be controlled explicitly via the new useNetwork parameter of the facet search object.

By @dureuill in #6375

Reduce memory usage of the indexing

Reduces allocated memory when computing prefixes and speeds up some operations to avoid unnecessary deserialization.

If you still see high memory usage while the engine is post-processing, we recommend using the --experimental-reduce-indexing-memory-usage option.

By @Kerollmops in #6334

Improve GeoJSON indexing performance

This PR upgrades the cellulite library to a version that includes a GeoJSON indexing optimization. With this change, GeoJSON indexing avoids reprocessing documents that were already indexed in dense cells, and only handles newly added documents incrementally as they descend through the recursive cell tree.

By @YoEight in #6374

Human-formatted sizes and detailed DB sizes in stats

Adds two new query parameters to GET /indexes/{indexUid}/stats and GET /stats:

  • showInternalDatabaseSizes: boolean, optional, defaults to false. When present, the index stat objects in responses of the stat routes now contain an additional internalDatabaseSizes key, whose value is a dictionary of the internal database names and their current size in the index, like in the stats object of a batch.
  • sizeFormat: human or raw: optional, defaults to raw. When present and set to human, then all database sizes in responses of the stat routes will be returned as a string containing an appropriate unit (MiB, GiB, etc). When missing or set to raw, then the current behavior of expressing the size in bytes as a number is retained.

Note for integrations: The keys in internalDatabaseSizes are subject to change and should not be exposed as a strongly typed object. This is the same as the existing internalDatabaseSizes in batch stats.

Adding showInternalDatabaseSizes to an index stats

// curl -X GET "http://localhost:7700/indexes/movies/stats?showInternalDatabaseSizes=true"
{
  "numberOfDocuments": 31944,
  "rawDocumentDbSize": 20594688,
  "avgDocumentSize": 636,
  "isIndexing": false,
  "internalDatabaseSizes": {
    "wordPairProximityDocids": 100827136,
    "documents": 20594688,
    "wordPositionDocids": 18694144,
    "wordFidDocids": 10715136,
    "wordPrefixPositionDocids": 10256384,
    "wordDocids": 9453568,
    "wordPrefixFidDocids": 4603904,
    "wordPrefixDocids": 3424256,
    "main": 1425408,
    "externalDocumentsIds": 999424,
    "fieldIdWordCountDocids": 245760,
    "exactWordPrefixDocids": 16384,
    "celluliteMetadata": 16384
  },
  "numberOfEmbeddings": 0,
  "numberOfEmbeddedDocuments": 0,
  "fieldDistribution": {
    "genres": 31944,
    "id": 31944,
    "overview": 31944,
    "poster": 31944,
    "release_date": 31944,
    "title": 31944
  }
}

Adding showInternalDatabaseSizes and sizeFormat=human to global stats

// curl -X GET "http://localhost:7700/stats?showInternalDatabaseSizes=true&sizeFormat=human"
{
  "databaseSize": "351.38 MiB",
  "usedDatabaseSize": "350.64 MiB",
  "lastUpdate": "2026-04-16T13:07:02.74243Z",
  "indexes": {
    "comics": {
      "numberOfDocuments": 31944,
      "rawDocumentDbSize": "23.14 MiB",
      "avgDocumentSize": "751 B",
      "isIndexing": false,
      "internalDatabaseSizes": {
        "wordPairProximityDocids": "98.06 MiB",
        "documents": "23.14 MiB",
        "wordPositionDocids": "17.22 MiB",
        "wordFidDocids": "10.36 MiB",
        "wordPrefixPositionDocids": "9.47 MiB",
        "wordDocids": "8.97 MiB",
        "wordPrefixFidDocids": "4.36 MiB",
        "wordPrefixDocids": "3.27 MiB",
        "main": "1.36 MiB",
        "externalDocumentsIds": "944 KiB",
        "fieldIdWordCountDocids": "240 KiB",
        "exactWordPrefixDocids": "16 KiB",
        "celluliteMetadata": "16 KiB"
      },
      "numberOfEmbeddings": 0,
      "numberOfEmbeddedDocuments": 0,
      "fieldDistribution": {
        "genres": 31944,
        "id": 31944,
        "overview": 31944,
        "poster": 31944,
        "release_date": 31944,
        "title": 31944
      }
    },
    "movies": {
      "numberOfDocuments": 31944,
      "rawDocumentDbSize": "19.64 MiB",
      "avgDocumentSize": "636 B",
      "isIndexing": false,
      "internalDatabaseSizes": {
        "wordPairProximityDocids": "96.16 MiB",
        "documents": "19.64 MiB",
        "wordPositionDocids": "17.83 MiB",
        "wordFidDocids": "10.22 MiB",
        "wordPrefixPositionDocids": "9.78 MiB",
        "wordDocids": "9.02 MiB",
        "wordPrefixFidDocids": "4.39 MiB",
        "wordPrefixDocids": "3.27 MiB",
        "main": "1.36 MiB",
        "externalDocumentsIds": "976 KiB",
        "fieldIdWordCountDocids": "240 KiB",
        "exactWordPrefixDocids": "16 KiB",
        "celluliteMetadata": "16 KiB"
      },
      "numberOfEmbeddings": 0,
      "numberOfEmbeddedDocuments": 0,
      "fieldDistribution": {
        "genres": 31944,
        "id": 31944,
        "overview": 31944,
        "poster": 31944,
        "release_date": 31944,
        "title": 31944
      }
    }
  }
}

The call without any parameters is the same as in previous versions.

// curl -X GET "http://localhost:7700/stats"
{
  "databaseSize": 368443392,
  "usedDatabaseSize": 367673344,
  "lastUpdate": "2026-04-16T13:07:02.74243Z",
  "indexes": {
    "comics": {
      "numberOfDocuments": 31944,
      "rawDocumentDbSize": 24264704,
      "avgDocumentSize": 751,
      "isIndexing": false,
      "numberOfEmbeddings": 0,
      "numberOfEmbeddedDocuments": 0,
      "fieldDistribution": {
        "genres": 31944,
        "id": 31944,
        "overview": 31944,
        "poster": 31944,
        "release_date": 31944,
        "title": 31944
      }
    },
    "movies": {
      "numberOfDocuments": 31944,
      "rawDocumentDbSize": 20594688,
      "avgDocumentSize": 636,
      "isIndexing": false,
      "numberOfEmbeddings": 0,
      "numberOfEmbeddedDocuments": 0,
      "fieldDistribution": {
        "genres": 31944,
        "id": 31944,
        "overview": 31944,
        "poster": 31944,
        "release_date": 31944,
        "title": 31944
      }
    }
  }
}

By @dureuill in #6338

🦋 Fixes

  • Settings changes performed via a settings subroute (not the "all settings" route) are now correctly propagated to other remotes on the network. By @dureuill in #6380
  • The chat route no longer fails in some condition when using Mistral as a provider. By @renezander030 in #6327
  • Prevent accidentally renaming a remote when modifying the network configuration. By @dureuill in #6370

🔩 Miscellaneous changes

New Contributors

Full Changelog: v1.43.1...v1.44.0

v1.43.1

12 May 12:29
ba97d76

Choose a tag to compare

Meilisearch v1.43.1 contains a security fix for an authenticated SSRF vulnerability.

No exploitation was found on Meilisearch Cloud. Cloud users are not required to update.

We recommend that self-hosting users upgrade if they allow third parties to configure Meilisearch instances.

We thank Sion Park (@tldhs1144), who reported the issue and suggested a fix, for improving the security of Meilisearch ❤️

v1.43.0

04 May 09:43
475ed56

Choose a tag to compare

🌈 Enhancements

New settings indexer

The new settings indexer, a more efficient version of our internal indexer for index settings modifications, now supports filterable, sortable, facet search, and custom (asc/desc) attributes in addition to the already-supported searchable, exact, proximity precision, and embedders.

  • For Meilisearch Cloud users, the new settings indexer is disabled by default and can be enabled on a case-by-case basis for scaling purposes.
  • For OSS users, the new settings indexer can be disabled by setting the MEILI_EXPERIMENTAL_NO_EDITION_2024_FOR_SETTINGS to true.

By @Kerollmops in #6124

Improve facet search indexing performance

We reduced the time to generate the internal data structures for the facet search to work. We were previously doing a couple of full scans on the internal entries and replaced these with scans on specific entries, only the ones dedicated to the facet searchable fields.

By @Kerollmops in #6325

Improve task queue compaction integration

The GET /health route now returns HTTP 500 after a successful task queue compaction, to signal that Meilisearch should be restarted so that tasks can be enqueued again.

For Meilisearch Cloud users, this ensures that compacting the task queue will automatically restart the instance after the compaction.

🦋 Fixes

Fix lexicographic filters on strings

Fixes a bug where string facet values appearing in <, <=, >, >= and IN filters were not normalized before comparison to facet values.

This would cause some values in documents (e.g. 2026-01-01T00:00:00) to appear to be higher than their filter counterpart, due to being normalized (e.g. like 2026-01-01t00:00:00).

Thanks to @njaard for reporting the issue ❤️

By @dureuill in #6340

Fix typo tolerance v1.41 regression

Fixes the WordDelta::added_or_deleted_words function.

By @Kerollmops in #6350.

Thanks to @vladradin99 and @snurfer0 for reporting the issue in #6349 and #6324 ❤️

🔩 Miscellaneous

Thanks again @gounthar, @njaard, @vladradin99 and @snurfer0 for your contributions ❤️

Full Changelog: v1.42.1...v1.43.0

v1.42.1 🦑​

14 Apr 15:06

Choose a tag to compare

🪲 Bug fixes

Fix issues when using the legacy settings indexer

By @dureuill and @ManyTheFish in #6331

  • When using the experimental feature "multimodal", removing a fragment would cause an internal error in subsequent search requests
  • When using the experimental feature "multimodal", modifying fragments would sometimes cause an internal error at indexing time
  • regenerate: false would be ignored when modifying embedder settings
  • When a nested field was declared as searchable, it would not be indexed if its parents were not declared as searchable

🔩 Miscellaneous

  • Add new CI run that tests the stable settings indexer by @dureuill in #6331

v1.42.0 🦑​

13 Apr 09:08
02ff9a4

Choose a tag to compare

✨ Enhancement

Support search fallback on remote unavailability

By @Kerollmops in #6306

We introduce a new fallback system for the sharding and replication enterprise edition feature, along with a way to determine which remote is available. The engine can avoid machines that are unavailable for a period and resume querying them once they're back online.

The following snippet shows what the /network route looks like now that this PR exposes the remote statuses/availabilities.

"remotes": {
	"prod2": {
	  "url": "http://localhost:7702",
	  "searchApiKey": "mykey",
	  "writeApiKey": "mykey",
	  "status": "available"
	},
	"prod3": {
	  "url": "http://localhost:7703",
	  "searchApiKey": "mykey",
	  "writeApiKey": "mykey",
	  "status": "unavailable"
	}
}

🔬 Experimental: Document join Filtering

By @ManyTheFish in #6314

This enhancement extends the Cross-index document hydration introduced in v1.39.0 by allowing the user to filter on the foreign indexes to retrieve the documents.

📓 Note: This implementation doesn't support a remote sharding environment

foreignKeys experimental feature

TheforeignKeys experimental feature must be activated to use the foreign filters:

curl -X PATCH 'http://127.0.0.1:7700/experimental-features' \
  -H 'Content-Type: application/json' \
  --data-binary '{"foreignKeys": true}'

foreignKeys + filter index setting

To be able to use the foreign filters, the related field must be set as a foreignKey and as a filterableAttribute in /indexes/{index_uid}/settings:

{
	// new setting, an array of foreign keys that allows multiple foreign relationships between indexes
	"foreignKeys":  [
		{
			// the path in the JSON document containing foreign document ids
			"fieldName": "actors",
			
			// the UID of the foreign index containing the documents to fetch during hydration
	        "foreignIndexUid": "actors"
		}
	],
	// the actors field must be filterable on equality
	"filterableAttributes": [
		{
	    	"attributePatterns": ["actors"],
	     	"features": {
				"facetSearch": false,
				"filter": {
					"equality": true,
					"comparison": false
				}
			}
		}
	]
}

filtering using the _foreign filter

On the search route, a new _foreign verb has been introduced and should be used as follows:

{
	"q": "<query>",

	// filters on the movie index:
	// genres = action
    // AND
    // the foreign documents from the actor index match:  birthday STARTS WITH \"1958-\" AND popularity >= 3.5
	"filter": "genres = action AND _foreign(actors, birthday STARTS WITH \"1958-\" AND popularity >= 3.5)"
}

Note: nesting foreign filters is not supported and will return an error

Example of usage

Prerequisites

  • Meilisearch running on 127.0.0.1:7700 on the document-join-hydration branch.

Step 1: Enable Foreign Keys Feature

curl -X PATCH 'http://127.0.0.1:7700/experimental-features' \
  -H 'Content-Type: application/json' \
  --data-binary '{"foreignKeys": true}'

Step 2: Create Indexes

Create the actors index
curl -X POST 'http://127.0.0.1:7700/indexes' \
  -H 'Content-Type: application/json' \
  --data-binary '{"uid": "actors", "primaryKey": "id"}'
Create the movies index
curl -X POST 'http://127.0.0.1:7700/indexes' \
  -H 'Content-Type: application/json' \
  --data-binary '{"uid": "movies", "primaryKey": "id"}'

Step 3: Add Documents to the actors Index

curl -X POST 'http://127.0.0.1:7700/indexes/actors/documents' \
  -H 'Content-Type: application/json' \
  --data-binary '[
  {"id": 1, "name": "Tom", "familyName": "Hanks", "birthDate": "1956-07-09"},
  {"id": 2, "name": "Meryl", "familyName": "Streep", "birthDate": "1949-06-22"},
  {"id": 3, "name": "Leonardo", "familyName": "DiCaprio", "birthDate": "1974-11-11"},
  {"id": 4, "name": "Emma", "familyName": "Watson", "birthDate": "1990-04-15"}
]'

Step 4: Add Documents to the movies Index

curl -X POST 'http://127.0.0.1:7700/indexes/movies/documents' \
  -H 'Content-Type: application/json' \
  --data-binary '[
  {"id": 1, "title": "Forrest Gump", "description": "The presidencies of Kennedy and Johnson, the Vietnam War, the Watergate scandal and other historical events unfold from the perspective of an Alabama man with an IQ of 75.", "actors": [1]},
  {"id": 2, "title": "The Devil Wears Prada", "description": "A smart but sensible new graduate lands a job as an assistant to Miranda Priestly, the demanding editor-in-chief of a high fashion magazine.", "actors": [2, 4]},
  {"id": 3, "title": "Inception", "description": "A thief who steals corporate secrets through the use of dream-sharing technology is given the inverse task of planting an idea into the mind of a C.E.O.", "actors": [3]},
  {"id": 4, "title": "Cast Away", "description": "A FedEx executive undergoes a physical and emotional transformation after crash landing on a deserted island.", "actors": [1]}
]'

Step 5: Configure Foreign Keys on the movies Index

curl -X PATCH 'http://127.0.0.1:7700/indexes/movies/settings' \
  -H 'Content-Type: application/json' \
  --data-binary '{"foreignKeys": [{"fieldName": "actors", "foreignIndexUid": "actors"}], "filterableAttributes": [{"attributePatterns": ["actors"],"features": {"facetSearch": false,"filter": {"equality": true,"comparison": false}}}]}'

Step 6: Configure filterable on the actors Index

curl -X PATCH 'http://127.0.0.1:7700/indexes/actors/settings' \
  -H 'Content-Type: application/json' \
  --data-binary '{"filterableAttributes": [{"attributePatterns": ["birthDate"],"features": {"facetSearch": false,"filter": {"equality": true,"comparison": false}}}]}'

Step 7: Perform a Federated Search

curl -X POST 'http://127.0.0.1:7700/multi-search' \
  -H 'Content-Type: application/json' \
  --data-binary '{
  "queries": [
    {
      "indexUid": "movies",
      "q": "Forrest",
      "filter": "_foreign(actors, birthDate = \"1956-07-09\")"
    }
  ],
  "federation": {
    "limit": 20,
    "offset": 0
  }
}'

Expected Result

The federated search should return movie documents with the actors array automatically hydrated with full actor objects instead of just IDs:

{
  "hits": [
    {
      "id": 1,
      "title": "Forrest Gump",
      "description": "...",
      "actors": [
        {
          "id": 1,
          "name": "Tom",
          "familyName": "Hanks",
          "birthDate": "1956-07-09"
        }
      ],
      "_federation": {
        "indexUid": "movies",
        "queriesPosition": 0,
        "weightedRankingScore": 0.9848484848484849
      }
    }
  ],
  "processingTimeMs": 208,
  "limit": 20,
  "offset": 0,
  "estimatedTotalHits": 1
}

🪲 Bug fixes

  • Fix a race condition when writing network by @Kerollmops in #6300

    We fixed a race condition in network topology changes that could cause errors and prevent documents from being correctly indexed. Additionally, we fixed a bug in the networkTopologyChange task batching that was causing it to batch too many task types. We made sure it only batches import tasks, and only those, to avoid out-of-order task processing.

  • Throw document template errors when updating the chat settings by @Kerollmops in #6321

    We fixed an issue that prevented the engine from explicitly showing the possible document template errors users could encounter when updating the template in the chat settings. The engine now correctly checks for and throws template errors when they are detected.

  • Fix: Update Index tasks will be properly forwarded to remote nodes by @dureuill in #6299

  • Fix action mistake on the chat completions route by @Kerollmops in #6290

🔩 Miscellaneous

  • Use the latest version of heed with nested rtxns support by @Kerollmops in #6316

    This PR bumps the versions of crates that use heed to the latest version, v0.22.1. This version finally stabilized a long-standing piece of work we were doing with Howard Chu: nested read transactions. We no longer have to rely on unstable pre-releases, but rather on a clean, stable version of LMDB (still a fork, but a better one).

  • Add section to CONTRIBUTING.md to bump mini-dashboard version and testing section to right place by @curquiza in #6195

  • Make the no-agent AGENTS.ms more permissive by @Kerollmops in #6260

  • Remove deleted test commands by @Strift in #6283

  • Fix OpenAPI schema generation for chat completions route by @qdequele in #6274

  • Rename OpenAPI route names for search rules and compact by @qdequele in #6298

  • Update README with new features and demos by @qdequele in #6297

  • Prevent shell injection in benchmark workflows by @curquiza & @Kerollmopsin #6308 & #6318

  • Rename so...

Read more

v1.41.0 🏈

30 Mar 13:55
8dd5a95

Choose a tag to compare

✨ Enhancement

  • Support dynamic search rules with pinning by @YoEight in #6182
    Introduce the Dynamic Search Rules (DSR) experimental feature, making it easy to promote the right content for the right searches with flexible, condition-based pinning. In this first version, rules can be triggered by query-based conditions such as empty queries or literal substring matches, as well as time windows. Multiple documents can be pinned in a defined order, and pins still work with filtering, pagination, facet distribution, hybrid search, and federated search. This feature is behind the dynamicSearchRules experimental feature flag.

    Rules can be created or updated with PATCH /dynamic-search-rules/{uid} and removed with DELETE /dynamic-search-rules/{uid}. In this first version, a rule can define query- or time-based conditions and pin specific documents at fixed positions in the results list.

    Example of rule creation/update
      // PATCH /dynamic-search-rules/featured-wireless-headphones
      {
        "description": "Promote featured products for wireless headphone searches",
        "active": true,
        "conditions": [
          { "scope": "query", "contains": "headphone" }
        ],
        "actions": [
          {
            "selector": { "indexUid": "products", "id": "featured-headphones-001" },
            "action": { "type": "pin", "position": 0 }
          },
          {
            "selector": { "indexUid": "products", "id": "featured-headphones-002" },
            "action": { "type": "pin", "position": 1 }
          }
        ]
      }
    
  • Default to useNetwork: true in sharded instances by @dureuill in #6278

    This release contains breaking changes for users of the network experimental feature.

    Search requests now use the network by default when (replicated) sharding is enabled
    § Breaking changes

    When network.leader is set in the instance, useNetwork now defaults to true in search requests when omitted.

    So, when a network is configured, the following:

    Search request without an explicit useNetwork
    // POST /indexes/movies/search
    {
      "q": "Batman dark knight returns 1",
      "filter": "genres IN [Action, Adventure]",
      "facets": ["genres"],
      "limit": 5
      // No `useNetwork` is passed
    }

    is now equivalent to a federated search over multiples remotes so that all shards in the network are covered exactly once.

    Equivalent federated search request
    // POST /multi-search
    {
        "federation": {
            "limit": 5,
            "facetsByIndex": {
                "movies": [
                    "genres"
                ]
            },
            "merge": {}
        },
        // assuming 3 remotes 0, 1, 2 and 3 shards a, b, c such that
        // 0 owns a, b
        // 1 owns b, c
        // 2 owns c, a
        "queries": [
            {
                "indexUid": "movies",
                "federationOptions": {
                    "remote": "0"
                },
                "q": "Batman dark knight returns 1",
                "filter": ["genres IN [Action, Adventure]", "_shard = a"]
            },
            {
                "indexUid": "movies",
                "federationOptions": {
                    "remote": "0" // useNetwork picked remote 0 for both shard a and b
                },
                "q": "Batman dark knight returns 1",
    		     "filter": ["genres IN [Action, Adventure]", "_shard = b"]
            },
            {
                "indexUid": "movies",
                "federationOptions": {
                    "remote": "2"
                },
                "q": "Batman dark knight returns 1",
    		     "filter": ["genres IN [Action, Adventure]", "_shard = c"]
            }
        ]
    }

    Responding with documents from all the shards:

    Response
    {
      "hits": [
        {
          "id": 123025,
          "title": "Batman: The Dark Knight Returns, Part 1",
          "overview": "Batman has not been seen for ten years. A new breed of criminal ravages Gotham City, forcing 55-year-old Bruce Wayne back into the cape and cowl. But, does he still have what it takes to fight crime in a new era?",
          "genres": [
            "Action",
            "Animation",
            "Mystery"
          ],
          "poster": "https://image.tmdb.org/t/p/w500/kkjTbwV1Xnj8wBL52PjOcXzTbnb.jpg",
          "release_date": 1345507200,
          "_federation": {
            "indexUid": "mieli",
            "queriesPosition": 2,
            "weightedRankingScore": 0.9894586894586894,
            "remote": "2"
          }
        },
        {
          "id": 142061,
          "title": "Batman: The Dark Knight Returns, Part 2",
          "overview": "Batman has stopped the reign of terror that The Mutants had cast upon his city.  Now an old foe wants a reunion and the government wants The Man of Steel to put a stop to Batman.",
          "genres": [
            "Action",
            "Animation",
            "Mystery"
          ],
          "poster": "https://image.tmdb.org/t/p/w500/arEZYd6uMOFTILne9Ux0A8qctMe.jpg",
          "release_date": 1357171200,
          "_federation": {
            "indexUid": "mieli",
            "queriesPosition": 1,
            "weightedRankingScore": 0.9894558963186414,
            "remote": "0"
          }
        },
        {
          "id": 16234,
          "title": "Batman Beyond: Return of the Joker",
          "overview": "The Joker is back with a vengeance, and Gotham's newest Dark Knight needs answers as he stands alone to face Gotham's most infamous Clown Prince of Crime.",
          "genres": [
            "Animation",
            "Family",
            "Action",
            "Science Fiction"
          ],
          "poster": "https://image.tmdb.org/t/p/w500/7RlBs0An83fqAuKfwH5gKMcqgMc.jpg",
          "release_date": 976579200,
          "_federation": {
            "indexUid": "mieli",
            "queriesPosition": 1,
            "weightedRankingScore": 0.9427964918160996,
            "remote": "0"
          }
        },
        {
          "id": 155,
          "title": "The Dark Knight",
          "overview": "Batman raises the stakes in his war on crime. With the help of Lt. Jim Gordon and District Attorney Harvey Dent, Batman sets out to dismantle the remaining criminal organizations that plague the streets. The partnership proves to be effective, but they soon find themselves prey to a reign of chaos unleashed by a rising criminal mastermind known to the terrified citizens of Gotham as the Joker.",
          "genres": [
            "Drama",
            "Action",
            "Crime",
            "Thriller"
          ],
          "poster": "https://image.tmdb.org/t/p/w500/qJ2tW6WMUDux911r6m7haRef0WH.jpg",
          "release_date": 1216166400,
          "_federation": {
            "indexUid": "mieli",
            "queriesPosition": 1,
            "weightedRankingScore": 0.5784178187403994,
            "remote": "0"
          }
        },
        {
          "id": 49026,
          "title": "The Dark Knight Rises",
          "overview": "Following the death of District Attorney Harvey Dent, Batman assumes responsibility for Dent's crimes to protect the late attorney's reputation and is subsequently hunted by the Gotham City Police Department. Eight years later, Batman encounters the mysterious Selina Kyle and the villainous Bane, a new terrorist leader who overwhelms Gotham's finest. The Dark Knight resurfaces to protect a city that has branded him an enemy.",
          "genres": [
            "Action",
            "Crime",
            "Drama",
            "Thriller"
          ],
          "poster": "https://image.tmdb.org/t/p/w500/vzvKcPQ4o7TjWeGIn0aGC9FeVNu.jpg",
          "release_date": 1342396800,
          "_federation": {
            "indexUid": "mieli",
            "queriesPosition": 2,
            "weightedRankingScore": 0.5772657450076805,
            "remote": "2"
          }
        }
      ],
      "query": "Batman dark knight returns 1",
      "processingTimeMs": 173,
      "limit": 5,
      "offset": 0,
      "estimatedTotalHits": 47,
      "facetDistribution": {
        "genres": {
          "Action": 46,
          "Adventure": 15,
          "Animation": 34,
          "Comedy": 3,
          "Crime": 14,
          "Drama": 6,
          "Family": 15,
          "Fantasy": 8,
          "Horror": 1,
          "Mystery": 4,
          "Romance": 1,
          "Science Fiction": 14,
          "TV Movie": 4,
          "Thriller": 4,
          "Western": 1
        }
      },
      "facetStats": {},
      "requestUid": "019bbcf4-a609-7701-8d82-d370611adfb3",
      "remoteErrors": {}
    }

    This change allows to naturally query all the documents in a sharded context.
    When network.leader is not present (in particular, when no network was defined), then the behavior is identical to previous versions.

  • useNetwork picks local instance if it owns the shard by @dureuill in #6287
    To prevent unnecessary networ...

Read more

v1.40.0 🦈

23 Mar 10:30
7fbd466

Choose a tag to compare

This release introduced support for the distinct attribute in federated search, enabling cross-index distinct attributes with facet distribution support. Additionally, significant performance improvements were delivered, including faster federated search (approximately 100ms faster), optimized JSON document generation for better handling of large documents and a much better memory usage for large workloads.

✨ Enhancement

  • Support distinct in federated search by @dureuill in #6214

    The distinct attribute can now be passed to the federation object in federated search to apply a global, cross-index and cross-remote distinct computation to the results.

    Example of a federated search request with distinct
     {
       "federation": {
         "distinct": "genres", // ✨ NEW
         "facetsByIndex": { // recovering facet distribution is also supported with distinct
           "comics": [
             "genres"
           ],
           "movies": [
             "genres"
           ]
         },
         "mergeFacets": {} // merging facet distributions is also supported with distinct
       },
       "queries": [
         {
           "indexUid": "comics",
           "q": "batman",
           "attributesToRetrieve": ["title", "genres"],
           "useNetwork": true // distinct is also supported with network queries
         },
         {
           "indexUid": "movies",
           "q": "superman",
           "attributesToRetrieve": ["title", "genres"],
           "useNetwork": true
         }
       ]
     }
    Sample response to a federated search request with distinct
     {
       "hits": [
         {
           "title": "Batman",
           "genres": [
             "Family",
             "Adventure",
             "Comedy",
             "Science Fiction",
             "Crime"
           ],
           "_federation": {
             "indexUid": "comics",
             "queriesPosition": 0,
             "weightedRankingScore": 1.0,
             "remote": "ms2"
           }
         },
         {
           "title": "Batman",
           "genres": [
             "Fantasy",
             "Action"
           ],
           "_federation": {
             "indexUid": "comics",
             "queriesPosition": 0,
             "weightedRankingScore": 1.0,
             "remote": "ms1"
           }
         },
         {
           "title": "Batman & Bill",
           "genres": [
             "Documentary"
           ],
           "_federation": {
             "indexUid": "comics",
             "queriesPosition": 0,
             "weightedRankingScore": 0.9848484848484848,
             "remote": "ms1"
           }
         },
         {
           "title": "Superman: Red Son",
           "genres": [],
           "_federation": {
             "indexUid": "movies",
             "queriesPosition": 1,
             "weightedRankingScore": 0.9848484848484849,
             "remote": "ms0"
           }
         },
         {
           "title": "Superman, Spider-Man or Batman",
           "genres": [
             "Drama"
           ],
           "_federation": {
             "indexUid": "movies",
             "queriesPosition": 1,
             "weightedRankingScore": 0.9848484848484849,
             "remote": "ms0"
           }
         }
       ],
       "processingTimeMs": 15,
       "limit": 5,
       "offset": 0,
       "estimatedTotalHits": 11,
       "facetDistribution": {
         "genres": {
           "Action": 1,
           "Adventure": 1,
           "Comedy": 3,
           "Crime": 2,
           "Documentary": 1,
           "Drama": 1,
           "Family": 1,
           "Fantasy": 1,
           "Horror": 2,
           "Romance": 1,
           "Science Fiction": 1,
           "Thriller": 1,
           "Western": 1
         }
       },
       "facetStats": {},
       "requestUid": "019d05c7-ea65-77a1-8274-22a8ba9e26db",
       "remoteErrors": {}
     }

    Note the following to apply the distinct attribute at the federation level:

    1. Applying distinct at the query level at the same time as the federation level is disallowed and will return a HTTP 400 error.
    2. The chosen distinct field will apply to all remotes and indexes, so it must be a filterable attribute for all participating remotes and indexes.
    3. While Meilisearch attempts to compute the most accurate facet distribution, in distributed contexts this cannot be guaranteed as the distinct algorithm is not applied on all of the remote documents.
  • Improve performance of federated search by @dureuill in #6229

    Improves performance of federated search: about 100ms faster for all requests. Improves reliability of the HTTP server: the server will no longer be blocked when too many federated search requests are being processed.

  • Optimize the generation of JSON documents by @Kerollmops in #6257

    Addresses performance issues that users might encounter when requesting large documents. Additionally, performance is enhanced when users request only a small subset of fields from large documents.

  • Use the latest version of mimalloc to improve memory usage by @Kerollmops in #6201

    Updates mimalloc from v2 to v3, improving memory sharing between threads and significantly reducing memory usage on large workloads. It also overrides the allocator to use mimalloc at linking time, allowing LMDB, Meilisearch, and other C libraries to share their allocations for better overall memory efficiency. @Kerollmops wrote a blog post about the story behind this improvement.

  • Add POST /tasks/compact for task queue compaction by @YoEight in #6193

    Compacts the task queue database and reclaim space so new tasks can keep being enqueued, without deleting existing tasks. This feature is behind the taskQueueCompactionRoute experimental feature flag.

Warning

Once task queue compaction completes, all write operations are blocked until the server is restarted.

🔐 Security

  • Bump rustls-webpki from 0.103.8 to 0.103.10 in #6273
  • Bump tar from 0.4.44 to 0.4.45 in #6271
  • Bump actix-web-lab from 0.24.3 to 0.26.0 in #6218
  • Bump quinn-proto from 0.11.13 to 0.11.14 in #6217

🪲 Bug fixes

🔩 Miscellaneous

New Contributors

v1.39.0 🐳​

16 Mar 12:54
21a9178

Choose a tag to compare

What's Changed

✨ Enhancement

🔬 Cross-index document hydration

Add a new ForeignKeys setting allowing to hydrate documents with documents coming from other indexes.

📓 Note: This implementation doesn't support a remote sharding environment

foreignKeys experimental feature

A new experimental feature, foreignKeys, has been added to the /experimental-feature route that must be set to true to activate the hydration.

curl -X PATCH 'http://127.0.0.1:7700/experimental-features' \
  -H 'Content-Type: application/json' \
  --data-binary '{"foreignKeys": true}'

foreignKeys index setting

A new index setting, foreignKeys, has been added to the /indexes/{index_uid}/settings:

// new setting, an array of foreign keys that allows multiple foreign relationships between indexes
"foreignKeys":  [
	{
		// the path in the JSON document containing foreign document ids
		"fieldName": "actors",
		
		// the UID of the foreign index containing the documents to fetch during hydration
        "foreignIndexUid": "actors"
	}
]

Example

With this new feature, a document shaped as follows:

{
  "id": 1,
  "title": "Forrest Gump",
  // Document IDs in foreign index
  "actors": [
    1
  ]
}

Will be hydrated as follows in a search response:

{
  "id": 1,
  "title": "Forrest Gump",
  "actors": [
	{
	  "id": 1,
	  "name": "Tom",
	  "familyName": "Hanks",
	  "birthDate": "1956-07-09"
	}
  ]
}

By @ManyTheFish in #6047

Disable proxy response buffering on Server-Sent Events (SSE)

Add X-Accel-Buffering: no on POST /chats/{workspace_uid}/chat/completions when the streaming mode is activated.
By @YoEight in #6228

🪲 Bug fixes

  • Fix a memory leak in the indexation pipeline by @Kerollmops in #6212

    We fixed an important memory leak caused by an invalid use of bumpalo. If you've seen Meilisearch using more and more memory over time, this issue is no longer visible in the latest version. If you want to read more about the bumpalo-trap we felt in, here is a good summary. It looks like this leak was introduced in v1.12, so approximately a year ago...

  • Avoid losing tasks by applying deltas on the wtxn view by @Kerollmops in #6222

    We addressed a small performance regression introduced in v1.38.1 that affected sending updates to the engine during task deletion. We've restored the task deletion performance to match v1.38.0 levels and ensured no race conditions occur while still allowing updates during deletion.

🔩 Miscellaneous

Full Changelog: v1.38.0...v1.39.0

v1.38.2 🐍

11 Mar 11:36
238de27

Choose a tag to compare

Meilisearch v1.38.2 fixes a regression introduced in Meilisearch v1.38.0 where Meilisearch could sometimes stop processing tasks after an automatic task queue cleanup performed while more tasks where being added.

We recommend that all users in v1.38.0 and v1.38.1 migrate to v1.38.2.

Note that the dumpless upgrade process can take up to one minute before the HTTP server becomes available when migrating from a previous v1.38 version.

Full Changelog: v1.38.1...v1.38.2