Codestin Search App

Download from TypeDB Package Repository:

Distributions for 3.5.5

Pull the Docker image:

docker pull typedb/typedb:3.5.5

New Features

Bugs Fixed

Fix unreachable crashes in type seeder
For cases where the preconditions may not be satisfied by VariableCategory checks, we seed empty sets of types instead of having an "unreachable" panic.

Fixes the crash in #7607
Fix brew install
We now symlink bin/typedb to libexec/typedb instead of moving. This allows us to maintain the directory structure of the original distribution and allows the typedb script to correctly resolve the install directory.

Code Refactors

Write type-check errors specify variable name
Write type check errors now provide the variable name instead of id.

Other Improvements

Lower DB import/export logging to DEBUG

Download from TypeDB Package Repository:

Distributions for 3.5.4

Pull the Docker image:

docker pull typedb/typedb:3.5.4

New Features

Bugs Fixed

Use latest console with fixed optional argument parsing

Code Refactors

Other Improvements

Download from TypeDB Package Repository:

Distributions for 3.5.2

Pull the Docker image:

docker pull typedb/typedb:3.5.2

New Features

Bugs Fixed

Fix variable name unwrap when creating error message in insert executable
Fixes an unwrap when reading the variable name for returning an error message
SpillOverCache removes key before inserting
Fixes a bug causing in SpillOverCache where a key can be inserted in both the memory backed map, and the disk back mapped.
Fix relation indices recalculation on schema changes
Correctly regenerate and delete relation role player indices on schema changes. This fixes a bug when some query results could be ignored because of an incorrectly configured index used for optimization purposes.
Reenable Sentry crash reporting
Fix a bug when TypeDB Server did not send crash reports for diagnostics even when --diagnostics.reporting.errors was enabled.

Code Refactors

Replace serializable_response macro with available serde annotations
Replace custom implementation of Serialize for concepts in the HTTP API with the derived serialize and serde annotations.

Other Improvements

Download from TypeDB Package Repository:

Distributions for 3.5.1

Pull the Docker image:

docker pull typedb/typedb:3.5.1

New Features

Bugs Fixed

Only seek iterators in k-way merge when behind

Fix intermittent crashes with Key behind the stored item in a Peekable iterator when a roleplayer index constraint is part of a join.

Code Refactors

Other Improvements

Print non-typedb source errors

Print source of error when it's not a TypeDB error as well as part of the generated stack trace.

Download from TypeDB Package Repository:

Distributions for 3.5.0

Pull the Docker image:

docker pull typedb/typedb:3.5.0

New Features

Analyze endpoint
Introduce the analyze endpoint for the HTTP API, meant for compiling a query (without running it) and inspecting it - Currently, the structure & type-annotations are returned.
The endpoint is at: /<version>/transactions/<transaction-id>/analyze

Sample request:

{ "query": " 
  with fun title($book: book) -> string: 
  match $
      book has title $title;
      let $as_string = $title;
  return first $as_string;

  match
      $book isa book;
  fetch { 
      \"title\": title($book) 
  };"
}

Sample response: (Or see the end of #7537 for a full example)

{
    "structure":  {
        "query": PipelineStructure, // As returned in any query
        "preamble": [ FunctionStructure, ...],
     },
     annotations: {
          "query":  PipelineAnnotations,
          "preamble": [ FunctionAnnotations, ... ],
          "fetch": FetchAnnotations,
        }
     }
}

where,

PipelineStructure = { blocks: [...],  "pipeline": [...], "variables":{ ... }, "outputs": [...] }
PipelineAnnotations = {
  "annotationsByBlock": [
      {
          "variableAnnotations": { "0": VariableAnnotations, ... }
      }
}

FunctionStructure = { 
  "body": PipelineStructure 
  "arguments":  [...],
  "return": FunctionReturnStructure,
}

FunctionReturnStructure =   { "tag": "check"}  | 
  { "tag": "stream" | "single",    "variables": [...] }  |
  { "tag": "reduce",  "reducers": [ { "reducer": "count"|"sum"|..., [variable: ]  ] }  |


FunctionAnnotations = { 
  "arguments": [VariableAnnotations, ... ], 
  "returned": [VariableAnnotations, ...],
  "body": PipelineAnnotations
}

FetchAnnotations = {
   "<key>": ValueTypeAnnotations | FetchAnnotations
}

ValueTypeAnnotations = ["string", ...] // etc

VariableAnnotations = EITHER
  {
      "tag": "concept",
      "annotations": [
          {
              "kind": "attributeType",
              "label": "title",
              "valueType": "string"
          }, 
          ...
      ]
  }
OR 
  {
    "tag": "value",
    "value_types": [
        "string", "long", ....
    ]
}

Bugs Fixed

Fix crash when functions sorts empty stream
Fixes a crash when a functions tried to sort an empty stream.
Fix development mode configuration in config file
Fixes development mode configuration in config file
Support inserting attributes from values of other attributes
Support inserting attributes from values of other attributes
```
match $x isa src-attr;
insert $y isa dst-attr == $x;
```
fixes #7551
Enforce stratification through single return functions
Single return functions may not call themselves recursively. Addresses #7550

This will now fail:
```
  define
    fun last_number() -> integer:
    match
      let $number = last_number() + 1;
    return last $number;
```
Store answers in a hash table as well to avoid slow inserts in large table
Answer tables additionally store answers in a HashSet for constant time lookup, making constructing large tables linear instead of quadratic.
Variables referenced in only some branches of a disjunction are required inputs
Updates the planner to treat variables which are referenced in "some but not all" branches of a disjunction as "required inputs". This means a pattern binding the variable must be bound before the disjunction can be scheduled.
Lock attributes on updating connections

We fix an Isolation bug that is exposed under concurrent update (adding an ownership) + concurrent delete transactions. This is a relatively uncommon conflict, and is fixed by locking attributes that are edited in order to conflict with concurrent deletes of the same attribute. Under highly concurrent operations that include deletes, this change might manifest itself as more transaction conflicts STC2 (Storage Commit 2) errors, which can be resolved with a retry.

Code Refactors

Add cases for reduce enum variants
Adds the remaining ReduceInstruction cases to a match introduced in the previous commit.
Rename the command-line argument for authentication token expiration
Rename the command-line argument server.authentication.token_ttl_seconds to token-expiration-seconds to make it consistent with the config field

Other Improvements

Fix missing word in isolation error message

The isolation conflict message when deleting a required key reads "Transaction data a concurrent commit requires."
Refactor bazel build dependencies

We've identified some redundant dependencies in bazel build scripts, which are harmful to incremental/parallel build.
We refactored them as part of a research project on dependency reduction.
Revise response format of the analyze endpoint
We revise the response format for the analyze endpoint of the HTTP API, to align it better with TypeDB's representation, and simplify parsing & reconstructing the structure.

Major changes include:
- Or, Not and Try blocks are now included in the block constraints. Hence the conjunction tree structure is represented within the blocks. This avoids a relatively convoluted interplay between pipeline.structure.conjunctions and pipeline.structure.blocks in reconstructing the conjunction tree.
- A match stage under a pipeline structure now only needs to hold a single blockId instead of a tree representing the conjunction.
- Fetch annotations are more structured, for easier parsing in all languages. Previously, the dynamic keys required a map with a union of values to parse properly. Now it's simply a union of static types, and provides the user a schema for the returned JSON object.
```
export type FetchAnnotations =
    { tag:  "list", elements: FetchAnnotations } |
    { tag : "object", possibleFields: FetchAnnotationFieldEntry[] } |
    { tag : "value", valueTypes: ValueType[] };

export type FetchAnnotationFieldEntry = FetchAnnotations & { key: string };
```
For a more complete spec, see typedb/typedb-driver#783
Allow empty define

Since typedb/typeql#412 is implemented, empty define queries are allowed. This means that schema export that used to return an empty string for empty schemas can now return an empty define query, which is much easier to work with on the client side.

This addresses #7531
Work around brew install [email protected] breaking our CI on mac
Works around brew install [email protected] breaking CircleCI on mac.
Re-enable @typeql release dependency validation
Update README links
Skip decompressing WAL records unnecessarily

Download from TypeDB Package Repository:

Distributions for 3.5.0-rc0

Pull the Docker image:

docker pull typedb/typedb:3.5.0-rc0

New Features

Analyze endpoint
Introduce the analyze endpoint for the HTTP API, meant for compiling a query (without running it) and inspecting it - Currently, the structure & type-annotations are returned.
The endpoint is at: /<version>/transactions/<transaction-id>/analyze

Sample request:

{ "query": " 
  with fun title($book: book) -> string: 
  match $
      book has title $title;
      let $as_string = $title;
  return first $as_string;

  match
      $book isa book;
  fetch { 
      \"title\": title($book) 
  };"
}

Sample response: (Or see the end of #7537 for a full example)

{
    "structure":  {
        "query": PipelineStructure, // As returned in any query
        "preamble": [ FunctionStructure, ...],
     },
     annotations: {
          "query":  PipelineAnnotations,
          "preamble": [ FunctionAnnotations, ... ],
          "fetch": FetchAnnotations,
        }
     }
}

where,

PipelineStructure = { blocks: [...],  "pipeline": [...], "variables":{ ... }, "outputs": [...] }
PipelineAnnotations = {
  "annotationsByBlock": [
      {
          "variableAnnotations": { "0": VariableAnnotations, ... }
      }
}

FunctionStructure = { 
  "body": PipelineStructure 
  "arguments":  [...],
  "return": FunctionReturnStructure,
}

FunctionReturnStructure =   { "tag": "check"}  | 
  { "tag": "stream" | "single",    "variables": [...] }  |
  { "tag": "reduce",  "reducers": [ { "reducer": "count"|"sum"|..., [variable: ]  ] }  |


FunctionAnnotations = { 
  "arguments": [VariableAnnotations, ... ], 
  "returned": [VariableAnnotations, ...],
  "body": PipelineAnnotations
}

FetchAnnotations = {
   "<key>": ValueTypeAnnotations | FetchAnnotations
}

ValueTypeAnnotations = ["string", ...] // etc

VariableAnnotations = EITHER
  {
      "tag": "concept",
      "annotations": [
          {
              "kind": "attributeType",
              "label": "title",
              "valueType": "string"
          }, 
          ...
      ]
  }
OR 
  {
    "tag": "value",
    "value_types": [
        "string", "long", ....
    ]
}

Bugs Fixed

Support inserting attributes from values of other attributes
Support inserting attributes from values of other attributes
```
match $x isa src-attr;
insert $y isa dst-attr == $x;
```
fixes #7551
Enforce stratification through single return functions
Single return functions may not call themselves recursively. Addresses #7550

This will now fail:
```
  define
    fun last_number() -> integer:
    match
      let $number = last_number() + 1;
    return last $number;
```
Store answers in a hash table as well to avoid slow inserts in large table
Answer tables additionally store answers in a HashSet for constant time lookup, making constructing large tables linear instead of quadratic.
Variables referenced in only some branches of a disjunction are required inputs
Updates the planner to treat variables which are referenced in "some but not all" branches of a disjunction as "required inputs". This means a pattern binding the variable must be bound before the disjunction can be scheduled.
Lock attributes on updating connections

We fix an Isolation bug that is exposed under concurrent update (adding an ownership) + concurrent delete transactions. This is a relatively uncommon conflict, and is fixed by locking attributes that are edited in order to conflict with concurrent deletes of the same attribute. Under highly concurrent operations that include deletes, this change might manifest itself as more transaction conflicts STC2 (Storage Commit 2) errors, which can be resolved with a retry.

Code Refactors

Add cases for reduce enum variants
Adds the remaining ReduceInstruction cases to a match introduced in the previous commit.
Rename the command-line argument for authentication token expiration
Rename the command-line argument server.authentication.token_ttl_seconds to token-expiration-seconds to make it consistent with the config field

Other Improvements

Fix missing word in isolation error message

The isolation conflict message when deleting a required key reads "Transaction data a concurrent commit requires."
Refactor bazel build dependencies

We've identified some redundant dependencies in bazel build scripts, which are harmful to incremental/parallel build.
We refactored them as part of a research project on dependency reduction.
Revise response format of the analyze endpoint
We revise the response format for the analyze endpoint of the HTTP API, to align it better with TypeDB's representation, and simplify parsing & reconstructing the structure.

Major changes include:
- Or, Not and Try blocks are now included in the block constraints. Hence the conjunction tree structure is represented within the blocks. This avoids a relatively convoluted interplay between pipeline.structure.conjunctions and pipeline.structure.blocks in reconstructing the conjunction tree.
- A match stage under a pipeline structure now only needs to hold a single blockId instead of a tree representing the conjunction.
- Fetch annotations are more structured, for easier parsing in all languages. Previously, the dynamic keys required a map with a union of values to parse properly. Now it's simply a union of static types, and provides the user a schema for the returned JSON object.
```
export type FetchAnnotations =
    { tag:  "list", elements: FetchAnnotations } |
    { tag : "object", possibleFields: FetchAnnotationFieldEntry[] } |
    { tag : "value", valueTypes: ValueType[] };

export type FetchAnnotationFieldEntry = FetchAnnotations & { key: string };
```
For a more complete spec, see typedb/typedb-driver#783
Allow empty define

Since typedb/typeql#412 is implemented, empty define queries are allowed. This means that schema export that used to return an empty string for empty schemas can now return an empty define query, which is much easier to work with on the client side.

This addresses #7531
Work around brew install [email protected] breaking our CI on mac
Works around brew install [email protected] breaking CircleCI on mac.
Re-enable @typeql release dependency validation
Update README links
Skip decompressing WAL records unnecessarily

Download from TypeDB Package Repository:

Distributions for 3.4.4

Pull the Docker image:

docker pull typedb/typedb:3.4.4

New Features

Implement missing reducers: min/max, decimal

We implement min/max reducers for every totally ordered value type (NOTE: does not include Duration as it's only partially ordered). We also implement statistical reducers (mean/std/median) for Decimal, decaying to Double in std.
Implement optionality in read queries

We implement a huge new feature: optionality in read queries! This means you can now do partial matches.

Basics

In the following query, all person instances will be returned, and if any name exists, the name will be added to the answer. However, not having an answer will not cause that person instance to be eliminated:
```
# Find any person, and if they have a name add that into the answer
match 
  $p isa person;
  try { $p has name $name; };
```
The answers will conceptually conform to a type like [ $p=person, $name=Option<name>].

Optionals only return when the full pattern inside the optional matches:
```
# Return any people, and use the ages only of people with name "john", else do not match anything additional
match 
  $p isa person;
  try { $p has name "john"; $p has age $age; };
```
Multiple try clauses are matched to the maximum degree at simultaneously for each non-optional answer component:
```
# If every person can have at most 1 name and 1 age: return every person, plus their name if it exists, plus their age if it exists
match
  $p isa person;
  try { $p has name $name; };
  try { $p has age $age; };
```
Note that this will produce 1 answer per person, not a permutation of each person, with and without their names and ages. If there are higher cardinalities of name or age, you get permutations of each possible name and age filled but neither filled.

Optional variables cannot be re-used across sibling clauses due to ambiguous outcomes:
```
### ILLEGAL - optional $name cannot be re-used across sibling try clauses
match 
  $p isa person;
  try { $p has name $name; }; 
  try { $x isa person, has name $name; };
```
Nesting and Pipelining

Optionals can also not be nested within disjunctions (or) or negations (not) - in negations, these operations would be no-ops, and disjunctions are currently limited to only returning variables that are guaranteed to be returned from any branch, ie. they cannot currently return optional values at all.

Optional variables can be used across stages of a query pipeline. Any pattern in a query using an optional variable that is not set will "fail" immediately (return 0 answers). However, by nesting a pattern containing an unset optional variable inside tryclause, this behaviour is absorbed since only thetry` clause will fail:
```
match
  try { $name isa name "John"; };
match
  try { $p isa person, has name $name; };
```
This will return exactly [$name=None, $p=None ] or [$name=Some(name), $p=Some(person) ] or [$name=Some(name), $p=None]

In contrast, by using the optional $name outside any optional scope, the following query will eliminate any answers that don't have $name set and only return answers containing [$name=name, $person=person]
```
match
  try { $name isa name "John"; };
match
  $p isa person, has name $name;
```

Bugs Fixed

Only seek on iterators that are behind in merge

Fix a crash when during a seek on a KMergeBy, which is merging multiple sorted tuple iterators, seek is called on iterators that are already ahead of the seek target, an invalid operation. It is valid to seek the iterator as a whole, as the merge may still be behind the seek point even if some constituent iterators are ahead.

Code Refactors

Separate out optional & negation type inference
Since negated & optional patterns cannot influence the type-annotations of the containing conjunction, we need not interleave their type-inference with that of the parent conjunction. Hence, We refactor the type-inference procedure so that type-inference on a conjunction & its disjunctions is completed before we begin the process on negated & optional subpatterns.

Other Improvements

Update warning printing on bootup
Update behaviour repository and make step checking more strict

Download from TypeDB Package Repository:

Distributions for 3.4.1

Pull the Docker image:

docker pull typedb/typedb:3.4.1

New Features

Enable RocksDB Prefix bloom filters
Enable RocksDB bloom filters, reducing avoidable disk-reads in large databases.
Refactor HTTP API output to include the structure of the pipeline
Adds the structure of the pipeline to the query structure returned by the HTTP API. Previously, we would just return a flat list of blocks without any information of how these blocks were connected (through stages and connectives such as or/not)

Bugs Fixed

Handle trailing partial writes in WAL

We implement the ability to discard partially written trailing WAL records.
Improve intersection cost estimate, granular query plan cache eviction

We improve the cost estimate of an intersection by factoring in the work required to produce each element of the iterators involved. Previously the cost estimate assumed that said cost was limited to fetching a single key on disk, however when a post-filter is applied to a constraint iterator, we may have to fetch multiple keys to produce one that is accepted by all filters.

Additionally, the query plan cache now takes into account the exact types used in the query to make eviction decisions on a plan-by-plan basis.

Code Refactors

Rename config and CLI fields to align with the naming conventions
Perform the following renames in config files:
- server.http-enabled -> server.http.enabled
- server.http-address -> server.http.address
- server.encryption.certificate_key -> server.encryption.certificate-key
Perform the following renames in CLI arguments:
- logging.logdir -> logging.directory
Simple renaming

We rename VariableValue::Empty to None, and rename VariableDependency to VariableBindingMode
Simplify PartialCostHash and beam search implementation
We simplify the hash used to detect redundant states during the plan-search, loosening the definition of equivalent plans for more aggressive pruning.
Reduce usages of assert! that don't need to be assert!
Allow assert! only in unrecoverable cases where we must crash the server.
Use refactored BDD layout

Other Improvements

Add HTTP authentication action diagnostics counts
Add diagnostics reporting for authentication actions performed through the HTTP endpoint.

Previously, due to its excessive usage for every request and low value, we ignored this method. However, it's the most efficient metric to understand the usage of the HTTP endpoint of the server, so it is a valuable piece of usage metrics for us.

Download from TypeDB Package Repository:

Distributions for 3.4.0

Pull the Docker image:

docker pull typedb/typedb:3.4.0

New Features

Introduce database export and import
Add database export and database import operations. Unlike in TypeDB 2.x, these operations are run through TypeDB GRPC clients such as the Rust driver or TypeDB Console, which solves a number of issues with networking and encryption, especially relevant for users of TypeDB Enterprise. With this, it becomes an official part of the TypeDB GPRC protocol.
Both operations are performed through the network, but the server and the client can be used on the same host.

Each TypeDB database can be represented as two files:
1. A text file with its TypeQL schema description: a complete define query for the whole schema.
2. A binary file with its data.
This format is an extension to the TypeDB 2.x's export format. See more details below for version compatibility information.

Database export

Database export allows a client to download database schema and data files from a TypeDB server for future import to the same or higher TypeDB version.

The files are created on the client side. While the database data is being exported, parallel queries are allowed, but none of them will affect the exported data thanks to TypeDB's transactionality.
However, the database will not be available for such operations as deletion.

Exported TypeDB 3.x databases cannot be imported into servers of older versions.

Database import

Database import allows a client to upload previously exported database schema and data files to a TypeDB server. It is possible to assign any new name to the imported database.

While the database data is being imported, it is not recommended to perform parallel operations against it.
Interfering actions may lead to import errors or database corruption.
Import supports all exported TypeDB 2.x and TypeDB 3.x databases. It can be used to migrate between TypeDB versions with breaking changes. Please visit our docs for more information.

Bugs Fixed

Remove "no open transaction" errors from HTTP transaction close endpoint
Remove the last possible and very rare case of the "no open transaction" error after calling /transactions/:transaction-id/close when the transaction is closed in parallel without answering the current request.

Fix variable id allocator overflow crash
Return a representation error instead of a crash for queries overflowing the variable number limit. Fixes #7482.

Rust driver's error example:

called `Result::unwrap()` on an `Err` value: Server(
[REP52] The query is too big and exceeds the limit of 65535 declared and anonymous variables. Please split it into parts and execute separately for better performance.
Caused: [QEX7] Error in provided query.
Near 16385:97
-----
    $p15460 isa person, has name "XdelwJ-5275984974011080378", has id 13954001766678618073, has bio "Bio: 0R9641h09e3kGdTbHN4t";
    (friend: $p5460, friend: $p15460) isa friendship, has bio "Friendship#1568388083:Tn1VORIS-ZUwdem";
--> $p5461 isa person, has name "k6HFQW-10247636131927110182", has id 14336185473951253852, has bio "Bio: OPPgVhYFdncrGBQ0xVv6";
                                                                                                    ^
    $p15461 isa person, has name "Fll5rq-125886840531597973", has id 10540830226431637304, has bio "Bio: KoJly7fs0HEPDiHiUBoz";
    (friend: $p5461, friend: $p15461) isa friendship, has bio "Friendship#3113679880:m6GqBBwt-144TBF";
-----)

Fix forcing development mode in the config
Fix the compilation time forcing of development mode for local and snapshot builds

Code Refactors

Commit statistics to WAL more frequently

Rebooting a large database can take a long time, due to having to synchronise statistics from the last durably written checkpoint in the WAL. This statistics record was written on a percentage change of the statistics, which meant a lot of transactions could be committed before the statistics are checkpointed again. These all needed to be replayed to generate the newest statistics values on reboot.

Instead, each database now checkpoints the statistics either every 1k commits, or after am aggregate delta of 10k counts in the statistics, whichever comes first. This helps bound the amount of WAL entries replayed (& therefore memory required) for synchronising statistics on reboot.

Note: There is still a gap in this implementation, which occurs when there are 999 large commits happen but the aggregate delta is 0 (for example, by adding/deleting data continuously in large blocks) - this scenario would still require loading a large amount of the WAL into memory on replay.
Open-up server state component for extension

The ServerState component has been converted into a trait, with the actual implementation moved into a new component LocalServerState. This change is motivated by the need of allowing additional server state implementations with extended functionalities.
Update planner cost model for joins to reflect seeks being implemented
Updates the cost for a join between iterators to reflect seeks being introduced (#7367). The sum of the iterator sizes was accurate when we had to scan through the iterator. With seeks, we can skip impossible values. The number of seeks is on the order of the size of the smaller iterator.
Centralise server's state management

Server's state and state mutation has been centralised into a single struct, ServerState. This centralised state management structure makes it possible to reuse and also extend state management in TypeDB Cluster.

Other Improvements

Send commit responses on commit requests instead of silently closing the stream
We utilize the already existing GRPC protocol's "commit" transaction response to explicitly mark that commit requests are successfully executed without errors. Previously, TypeDB clients only relied on either an error or a closure of the transaction stream, understanding the state of the transaction only based on its sent requests.
Update console artifact with the upgraded db import protocol
Update README.md's style, contributions, and build instructions
Update the contributors list and minor styling moments to correspond to the current state of TypeDB.
Implement check operation in function return
Implement check operation in function return.
Ignore importing databases with empty schemas
Ignore empty imported schemas to allow importing databases without schemas. This also unlocks the importing of empty exported databases, ensuring the symmetry of export and import operations.
Update zlib dependency
Support build on Apple Clang 17+ by updating dependencies (details: typedb/typedb-dependencies#577).

Download from TypeDB Package Repository:

Distributions for 3.4.0-rc0

Pull the Docker image:

docker pull typedb/typedb:3.4.0-rc0

New Features

Introduce database export and import
Add database export and database import operations. Unlike in TypeDB 2.x, these operations are run through TypeDB GRPC clients such as the Rust driver or TypeDB Console, which solves a number of issues with networking and encryption, especially relevant for users of TypeDB Enterprise. With this, it becomes an official part of the TypeDB GPRC protocol.
Both operations are performed through the network, but the server and the client can be used on the same host.

Each TypeDB database can be represented as two files:
1. A text file with its TypeQL schema description: a complete define query for the whole schema.
2. A binary file with its data.
This format is an extension to the TypeDB 2.x's export format. See more details below for version compatibility information.

Database export

Database export allows a client to download database schema and data files from a TypeDB server for future import to the same or higher TypeDB version.

The files are created on the client side. While the database data is being exported, parallel queries are allowed, but none of them will affect the exported data thanks to TypeDB's transactionality.
However, the database will not be available for such operations as deletion.

Exported TypeDB 3.x databases cannot be imported into servers of older versions.

Database import

Database import allows a client to upload previously exported database schema and data files to a TypeDB server. It is possible to assign any new name to the imported database.

While the database data is being imported, it is not recommended to perform parallel operations against it.
Interfering actions may lead to import errors or database corruption.

Import supports all exported TypeDB 2.x and TypeDB 3.x databases. It can be used to migrate between TypeDB versions with breaking changes. Please visit our docs for more information.

Bugs Fixed

Remove "no open transaction" errors from HTTP transaction close endpoint
Remove the last possible and very rare case of the "no open transaction" error after calling /transactions/:transaction-id/close when the transaction is closed in parallel without answering the current request.

Fix variable id allocator overflow crash
Return a representation error instead of a crash for queries overflowing the variable number limit. Fixes #7482.

Rust driver's error example:

called `Result::unwrap()` on an `Err` value: Server(
[REP52] The query is too big and exceeds the limit of 65535 declared and anonymous variables. Please split it into parts and execute separately for better performance.
Caused: [QEX7] Error in provided query.
Near 16385:97
-----
    $p15460 isa person, has name "XdelwJ-5275984974011080378", has id 13954001766678618073, has bio "Bio: 0R9641h09e3kGdTbHN4t";
    (friend: $p5460, friend: $p15460) isa friendship, has bio "Friendship#1568388083:Tn1VORIS-ZUwdem";
--> $p5461 isa person, has name "k6HFQW-10247636131927110182", has id 14336185473951253852, has bio "Bio: OPPgVhYFdncrGBQ0xVv6";
                                                                                                    ^
    $p15461 isa person, has name "Fll5rq-125886840531597973", has id 10540830226431637304, has bio "Bio: KoJly7fs0HEPDiHiUBoz";
    (friend: $p5461, friend: $p15461) isa friendship, has bio "Friendship#3113679880:m6GqBBwt-144TBF";
-----)

Fix forcing development mode in the config
Fix the compilation time forcing of development mode for local and snapshot builds

Code Refactors

Commit statistics to WAL more frequently

Rebooting a large database can take a long time, due to having to synchronise statistics from the last durably written checkpoint in the WAL. This statistics record was written on a percentage change of the statistics, which meant a lot of transactions could be committed before the statistics are checkpointed again. These all needed to be replayed to generate the newest statistics values on reboot.

Instead, each database now checkpoints the statistics either every 1k commits, or after am aggregate delta of 10k counts in the statistics, whichever comes first. This helps bound the amount of WAL entries replayed (& therefore memory required) for synchronising statistics on reboot.

Note: There is still a gap in this implementation, which occurs when there are 999 large commits happen but the aggregate delta is 0 (for example, by adding/deleting data continuously in large blocks) - this scenario would still require loading a large amount of the WAL into memory on replay.
Open-up server state component for extension

The ServerState component has been converted into a trait, with the actual implementation moved into a new component LocalServerState. This change is motivated by the need of allowing additional server state implementations with extended functionalities.
Update planner cost model for joins to reflect seeks being implemented
Updates the cost for a join between iterators to reflect seeks being introduced (#7367). The sum of the iterator sizes was accurate when we had to scan through the iterator. With seeks, we can skip impossible values. The number of seeks is on the order of the size of the smaller iterator.
Centralise server's state management

Server's state and state mutation has been centralised into a single struct, ServerState. This centralised state management structure makes it possible to reuse and also extend state management in TypeDB Cluster.

Other Improvements

Send commit responses on commit requests instead of silently closing the stream
We utilize the already existing GRPC protocol's "commit" transaction response to explicitly mark that commit requests are successfully executed without errors. Previously, TypeDB clients only relied on either an error or a closure of the transaction stream, understanding the state of the transaction only based on its sent requests.
Update console artifact with the upgraded db import protocol
Update README.md's style, contributions, and build instructions
Update the contributors list and minor styling moments to correspond to the current state of TypeDB.

Releases: typedb/typedb

TypeDB 3.5.5

New Features

Bugs Fixed

Code Refactors

Other Improvements

Uh oh!

TypeDB 3.5.4

New Features

Bugs Fixed

Code Refactors

Other Improvements

Uh oh!

TypeDB 3.5.2

New Features

Bugs Fixed

Code Refactors

Other Improvements

Uh oh!

TypeDB 3.5.1

New Features

Bugs Fixed

Code Refactors

Other Improvements

Uh oh!

TypeDB 3.5.0

New Features

Bugs Fixed

Code Refactors

Other Improvements

Contributors

Uh oh!

TypeDB 3.5.0-rc0

New Features

Bugs Fixed

Code Refactors

Other Improvements

Contributors

Uh oh!

TypeDB 3.4.4

New Features

Basics

Nesting and Pipelining

Bugs Fixed

Code Refactors

Other Improvements

Uh oh!

TypeDB 3.4.1

New Features

Bugs Fixed

Code Refactors

Other Improvements

Uh oh!

TypeDB 3.4.0

New Features

Database export

Database import

Bugs Fixed

Code Refactors

Other Improvements

Uh oh!

TypeDB 3.4.0-rc0

New Features

Database export

Database import

Bugs Fixed

Code Refactors

Other Improvements

Uh oh!