v2.0.0

Tue, 17 Sep 2024 00:00:00 GMT

After numerous alpha and beta releases over months of development, we are releasing [SurrealDB v2.0.0!](https://surrealdb.com/blog/challenge-accepted-announcing-surrealdb-2-0) 🎉. The changes for this release pertain to those in between the final `2.0.0-beta-3` and other alpha versions. _**NOTE:**_ Because of changes in the underlying way SurrealDB stores data, steps are required to migrate to `2.0.0` from previous versions. Please use the following steps for any existing databases: * If you are moving from `1.x` to `2.x`, [use the `surreal fix` command](https://surrealdb.com/docs/surrealdb/installation/upgrading/migrating-data-to-2x#moving-from-1x-to-200-beta). Features: * [Enable FoundationDB transaction timeout customization](https://github.com/surrealdb/surrealdb/pull/4166) * [Implement missing modulo operator in new parser](https://github.com/surrealdb/surrealdb/pull/4182) * [Restructure the parser around the concept of token gluing.](https://github.com/surrealdb/surrealdb/pull/4081) * Added [`DEFINE ACCESS`](https://surrealdb.com/docs/surrealql/statements/define/access) statement to grant access to resources. * Deprecated `DEFINE SCOPE` in favor of [`DEFINE ACCESS ... TYPE RECORD`](https://surrealdb.com/docs/surrealql/statements/define/access/record). * Deprecated `DEFINE TOKEN` in favor of [`DEFINE ACCESS ... TYPE JWT`](https://surrealdb.com/docs/surrealql/statements/define/access/jwt). * Deprecated `DEFINE TOKEN ON SCOPE` in favor of [`DEFINE ACCESS ... TYPE RECORD ... WITH JWT`](https://surrealdb.com/docs/surrealql/statements/define/access/record). * Added customizable algorithm and key when issuing tokens for [records users](https://surrealdb.com/docs/surrealdb/security/authentication#record-users) with the `WITH ISSUER` clause. * Added customizable [token and session duration](https://surrealdb.com/docs/surrealdb/security/authentication#expiration) to [`DEFINE USER`](https://surrealdb.com/docs/surrealql/statements/define/user) and [`DEFINE ACCESS`](https://surrealdb.com/docs/surrealql/statements/define/access). * Removed [session expiration](https://surrealdb.com/docs/surrealdb/security/authentication#expiration) when not explicitly defined. * Changed the `INFO` statement to redact secrets defined via `DEFINE ACCESS`. * Changed HTTP request headers expected by SurrealDB to require the `surreal-` prefix. * Removed the `--auth` flag in the CLI to enable it by default. Added [`--unauthenticated`](https://surrealdb.com/docs/surrealdb/cli/start#unauthenticated-mode). * Removed the `--enable-auth-level` flag in CLI to enable the behavior by default. Defaults to root. * Changed authentication to expect level via the [`--auth-level`](https://surrealdb.com/docs/surrealdb/cli/sql#--auth-level-option) flag or `surreal-auth-*` headers. * Changed the default `--bind` address in the CLI from `0.0.0.0` to `127.0.0.1`. * Added the `--no-identification-headers` flag in the CLI to limit information leakage. * Added [`type::range`](https://surrealdb.com/docs/surrealql/functions/database/type#typerange) function. * Added [`TEMPFILES`](https://surrealdb.com/docs/surrealql/statements/select#the-tempfiles-clause) clause to the `SELECT` statement. * Introduce resource creation checks for `DEFINE` statement to avoid duplicates. * Added [string functions](https://surrealdb.com/docs/surrealql/functions/database/string) for IP `string::is::ip`,`string::is::ipv4`,`string::is::ipv6`. * Added [string functions](https://surrealdb.com/docs/surrealql/functions/database/string) for HTML: `string::html::encode`, `string::html::sanitize`. * Added new [math functions](https://surrealdb.com/docs/surrealql/functions/database/math): `math::acos`, `math::acot`, `math::asin`, `math::atan`, `math::clamp`, `math::cos`, `math::cot`, `math::deg2rad`, `math::lerp`, `math::lerpangle`, `math::ln`, `math::log`, `math::log2`, `math::log10`, `math::rad2deg`, `math::sign`, `math::sin`, `math::tan`. * Support batch import with `INSERT` statement. This also extends to relationships between tables using the `INSERT RELATION` statement * [Allow bridge connectivity in all docker images built](https://github.com/surrealdb/surrealdb/pull/4364) * [Added define JWT access at the root level](https://surrealdb.com/docs/surrealql/statements/define/access/jwt#root) * [Allow authenticating with a token from the CLI](https://surrealdb.com/docs/surrealdb/cli/sql#--token-option) * Added [Asynchronous building for indexes](https://surrealdb.com/docs/surrealql/statements/define/indexes#using-concurrently-clause) * Improved [HNSW persistence](https://github.com/surrealdb/surrealdb/pull/4214) * Added [Range value support](https://github.com/surrealdb/surrealdb/pull/4506) * Added ability to [SELECT with version](https://github.com/surrealdb/surrealdb/pull/4510), [CREATE with version](https://github.com/surrealdb/surrealdb/pull/4548) and [INSERT with version](https://github.com/surrealdb/surrealdb/pull/4583) on SurrealKV * Added [ENFORCED](https://github.com/surrealdb/surrealdb/pull/4579) keyword to DEFINE TABLE * SCHEMAFULL tables are now [TYPE NORMAL by default](https://github.com/surrealdb/surrealdb/pull/4558) * Added [literal types](https://github.com/surrealdb/surrealdb/pull/4557) to the query language * Added the [.chain() method](https://github.com/surrealdb/surrealdb/pull/4513) * Added [typed LET statements](https://github.com/surrealdb/surrealdb/pull/4476) * Added [typed function returns](https://github.com/surrealdb/surrealdb/pull/4475) * Added [record::exists()](https://github.com/surrealdb/surrealdb/pull/4602) function * Added [new functions](https://github.com/surrealdb/surrealdb/pull/4608) `value::diff()` and `value:::patch()` * [Array function changes and additions](https://github.com/surrealdb/surrealdb/pull/4612): `array::all()`, `array::any()`, `array::filter_index()`, `array::find_index()` functions can now take a value or a closure. Also added `array::filter()` and `array::find()` functions. * Upgraded [SurrealKV to v0.3](https://github.com/surrealdb/surrealdb/pull/4301) * More flexible [create API](https://github.com/surrealdb/surrealdb/pull/4797) for Rust SDK * [Query depth limits can now be configured](https://github.com/surrealdb/surrealdb/pull/4787) * [Added GraphQL config](https://github.com/surrealdb/surrealdb/pull/4704) * Indexing operators [can now take expressions](https://github.com/surrealdb/surrealdb/pull/4756) * Support for [mixed number types on range queries](https://github.com/surrealdb/surrealdb/pull/4747) * [Improved capabilities logic with specific allows able to override general denies](https://github.com/surrealdb/surrealdb/pull/4763) * [“false” string no longer eagerly evaluated as false](https://github.com/surrealdb/surrealdb/pull/4775) * [time::epoch](https://github.com/surrealdb/surrealdb/pull/4773) constant now available * Added [geo::is::valid() function](https://github.com/surrealdb/surrealdb/pull/4711) * Method call syntax [now works on integers](https://github.com/surrealdb/surrealdb/pull/4762) * Improved [GraphQL record fetching](https://github.com/surrealdb/surrealdb/pull/4682) * [Added `AUTHENTICATE` clause on `RECORD` access](https://surrealdb.com/docs/surrealql/statements/define/access/record#with-authenticate-clause) * Enabled [publishing SurrealDB ports in Docker for bridge networking connectivity](https://github.com/surrealdb/surrealdb/pull/4276) * [Allow generating `ULID` and `UUID` based on a timestamp](https://surrealdb.com/docs/surrealql/functions/database/rand#randuuidv4-from-timestamp) * [Allow unsetting namespace and database on RPC protocol](https://github.com/surrealdb/surrealdb/pull/4373) * [Moved fetch resolution to a compute method](https://github.com/surrealdb/surrealdb/pull/4312) * Added [`ALTER TABLE`](https://surrealdb.com/docs/surrealql/statements/alter) statement * Moved [`AUTHENTICATE`](https://surrealdb.com/docs/surrealql/statements/define/access#with-authenticate-clause) clause to the root of `DEFINE ACCESS` statement to allow the clause to be used in every access method * Added [`OVERWRITE`](https://github.com/surrealdb/surrealdb/pull/4465) clause for defined resources * Added [`array::windows()`](https://surrealdb.com/docs/surrealql/functions/database/array#arraywindows) function * Added [support for idiom destructuring in SurrealQL](https://github.com/surrealdb/surrealdb/pull/4443) * Added [`string::is::record`](https://github.com/surrealdb/surrealdb/pull/4462) function * Added [Method chaining for Database functions](https://github.com/surrealdb/surrealdb/pull/4469) * Added [anonymous functions](https://github.com/surrealdb/surrealdb/pull/4474) * Added [typed function returns](https://github.com/surrealdb/surrealdb/pull/4475) * Function return types [now shown in `INFO` statement](https://github.com/surrealdb/surrealdb/pull/4643) * Type `any` can now [act as an optional type](https://github.com/surrealdb/surrealdb/pull/4659) * Added [`time::is::leap_year()`](https://surrealdb.com/docs/surrealql/functions/database/time#timeisleap_year) function * Added environment variables to configure [HTTP endpoint limits](https://surrealdb.com/docs/surrealdb/cli/env) & [JavaScript runtime limits](https://surrealdb.com/docs/surrealdb/cli/env) * Introduced [savepoint capabilities](https://github.com/surrealdb/surrealdb/pull/4707) * Added [experimental `VERSION` clause to `INFO FOR DB` and `INFO FOR TABLE`](https://github.com/surrealdb/surrealdb/pull/4692) * Added [additional chained methods to number values](https://github.com/surrealdb/surrealdb/pull/4741) Performance improvements: * [SELECT/count optimization on table range](https://github.com/surrealdb/surrealdb/pull/4786) * Improvements to [node membership processing and garbage collection](https://github.com/surrealdb/surrealdb/pull/4780) * Improvements to [select count table scan](https://github.com/surrealdb/surrealdb/pull/4285) * Avoid [potentially some methods twice](https://github.com/surrealdb/surrealdb/pull/4731) * Stabilise and use `parser2` for parsing. * KNN filtering with limit and KNN distance function. Bug fixes: * Fixed [incorrectly parsed identifier characters](https://github.com/surrealdb/surrealdb/pull/4522) * Fixed [`record` casting from string](https://github.com/surrealdb/surrealdb/pull/4496) * Connection ID [now stored inside session data](https://github.com/surrealdb/surrealdb/pull/4536) * Fixed [certain identifier characters not being parsed correctly](https://github.com/surrealdb/surrealdb/pull/4522) * Fixed [remote fields added to output result](https://github.com/surrealdb/surrealdb/pull/4515) inside SELECT \* queries * [Revise `RecordAccess` structure to aid v2 alpha upgrade](https://github.com/surrealdb/surrealdb/pull/4361) * Fixed [GHSA-gh9f-6xm2-c4j2](https://github.com/surrealdb/surrealdb/security/advisories/GHSA-gh9f-6xm2-c4j2): Improper Authentication when Changing Databases as Scope User * Fixed [argument check for `array::clump`](https://github.com/surrealdb/surrealdb/pull/4167) * Fixed [aborting task leading to an error](https://github.com/surrealdb/surrealdb/pull/4195) * Fixed [non-enforced field permissions on WHERE statements](https://github.com/surrealdb/surrealdb/pull/4800) * Fixed [failing search when search analyzer lacks tokenizers](https://github.com/surrealdb/surrealdb/pull/4798) * Fixed [core dump when performing INSERT](https://github.com/surrealdb/surrealdb/pull/4785) * Fixed [discarding without error when using RELATE statement on edge that already exists](https://github.com/surrealdb/surrealdb/pull/4785) * Fixed [modification of arrays with JSON Patch semantics](https://github.com/surrealdb/surrealdb/pull/4784) * Fixed [selecting directly from a graph edge failing to select all primary edges](https://github.com/surrealdb/surrealdb/pull/4783) * Fixes to [issues with composite indexes](https://github.com/surrealdb/surrealdb/pull/4746) * Fixed [INSERT not working in subqueries](https://github.com/surrealdb/surrealdb/pull/4760) * Fixed [KNN brute force error when an index exists](https://github.com/surrealdb/surrealdb/pull/4143). * Fixed [Redefining HNSW index error](https://github.com/surrealdb/surrealdb/pull/4142). * [Check that a field exists before index creation](https://github.com/surrealdb/surrealdb/pull/4148). * Fixed [missing serialisation for `math::inf`](https://github.com/surrealdb/surrealdb/pull/4132). * Fixed [polygon serialization](https://github.com/surrealdb/surrealdb/pull/4127). * Fixed [panic and infinite loop in parser](https://github.com/surrealdb/surrealdb/pull/4149). * Fixed [URL parsing function when a port is present](https://github.com/surrealdb/surrealdb/pull/4087). * Fixed [display formats for access methods](https://github.com/surrealdb/surrealdb/pull/4220) * Fixed [parser error to allow `ns` as an alias for `NAMESPACE`](https://github.com/surrealdb/surrealdb/pull/4195) * [Added missing datetime prefix](https://github.com/surrealdb/surrealdb/pull/4244) * [Removed duplicated badge for docker pulls count in `README.md`](https://github.com/surrealdb/surrealdb/pull/4215) * Fixed [export for sql::filter::Filter type in surrealdb-core](https://github.com/surrealdb/surrealdb/pull/4156) * [Updated deprecated version of Node.js](https://github.com/surrealdb/surrealdb/pull/4251) * Fixed [ELSE IF pretty formatting](https://github.com/surrealdb/surrealdb/pull/4267) * Changed [geometry point values to floats in CBOR](https://github.com/surrealdb/surrealdb/pull/4266) * Fixed [authentication checks on root user websocket RPC session](https://github.com/surrealdb/surrealdb/pull/4288) * Ensured [a transaction is always available when live queries are evaluated](https://github.com/surrealdb/surrealdb/pull/4330) * Reduced [read lock duration while processing connection messages](https://github.com/surrealdb/surrealdb/pull/4328) * Ensured [execution halts after the `RETURN` statement](https://github.com/surrealdb/surrealdb/pull/4298) * Ensured [the server shuts down properly](https://github.com/surrealdb/surrealdb/pull/4295) * [Fixed querying complex edge table](https://github.com/surrealdb/surrealdb/pull/4391) * [Removed unnecessary backticks from `INFO FOR` statement output](https://github.com/surrealdb/surrealdb/pull/4438) * [Fixed Live Query Panic issue](https://github.com/surrealdb/surrealdb/pull/4437) * [Fixed validation on schemafull `RELATE`](https://github.com/surrealdb/surrealdb/pull/4226) * [Fixed division by zero bug in `math::mean()`](https://github.com/surrealdb/surrealdb/pull/4402) function * Improved [`string::is::datetime` and `string::is::uuid`](https://github.com/surrealdb/surrealdb/pull/4462) * [Parse like (`~`) operator correctly](https://github.com/surrealdb/surrealdb/pull/4460) * [Removed faulty flatten in graph traversal](https://github.com/surrealdb/surrealdb/pull/4463) * [Fixed `record` casting from string](https://github.com/surrealdb/surrealdb/pull/4496) * [Removed `ASSERT` for type `option` and value `NONE`](https://github.com/surrealdb/surrealdb/pull/4483) * Fixed [`aud` field on a `JWT` claim decoding](https://github.com/surrealdb/surrealdb/pull/4488) * Removed [unneeded return in matching](https://github.com/surrealdb/surrealdb/pull/4492) * Disabled [default audience validation for JWKS](https://github.com/surrealdb/surrealdb/pull/4495) * Fixed [errors in supply chain security workflow](https://github.com/surrealdb/surrealdb/pull/4497) * Enabled [check for single and multiple token audiences](https://github.com/surrealdb/surrealdb/pull/4493) * Removed [panic on invalid range scan](https://github.com/surrealdb/surrealdb/pull/4641) * Added `table` [to reserved keywords](https://github.com/surrealdb/surrealdb/pull/4655) * Concurrent indexing [no longer fails if record updated during build](https://github.com/surrealdb/surrealdb/pull/4629) * Unreachable code [now returns an error](https://github.com/surrealdb/surrealdb/pull/4665) * `DEFINE EVENT` [defaults to `true` when no WHEN clause specified](https://github.com/surrealdb/surrealdb/pull/4662) * Fixed bug [in which `NONE` and `NULL` both < 1 and > 1](https://github.com/surrealdb/surrealdb/pull/4658) * Fixed [inconsistency on queries using equality with or without index](https://github.com/surrealdb/surrealdb/pull/4666) * `Create` on rpc which returned an array [now returns an object](https://github.com/surrealdb/surrealdb/pull/4667). Similar change [made to Rust SDK](https://github.com/surrealdb/surrealdb/pull/4688) * Rust SDK [fix retrieving multiple columns from the same response](https://github.com/surrealdb/surrealdb/pull/4678) * [Improved consistency for `surrealdb.query()` and `surrealdb.value()` in JavaScript runtime](https://github.com/surrealdb/surrealdb/pull/4675) * Fixed [`ENFORCED` keyword not displaying in INFO statement](https://github.com/surrealdb/surrealdb/pull/4635) * Fixed [GHSA-64f8-pjgr-9wmr](https://github.com/surrealdb/surrealdb/security/advisories/GHSA-64f8-pjgr-9wmr): Untrusted Query Object Evaluation in RPC API * Improved [`math::mean()` to support returning floats](https://github.com/surrealdb/surrealdb/pull/4390) * Improved [namespace/database validation for GraphQL](https://github.com/surrealdb/surrealdb/pull/4718) * Ensured [`TIMEOUT` clauses are processed correctly](https://github.com/surrealdb/surrealdb/pull/4721) * Fixed [importing from `1.x`](https://github.com/surrealdb/surrealdb/pull/4725) * Deprecated [missing format inference on RPC protocol](https://github.com/surrealdb/surrealdb/pull/4729) Workflow improvements: * [Deprecated `file://`](https://github.com/surrealdb/surrealdb/pull/4505) * Added [`surreal fix`](https://github.com/surrealdb/surrealdb/pull/4605) command * Exports [no longer use explicit transactions](https://github.com/surrealdb/surrealdb/pull/4610) * [/lib path renamed to /sdk](https://github.com/surrealdb/surrealdb/pull/4561) * [Rename the next workflow to alpha](https://github.com/surrealdb/surrealdb/pull/4175) * [Improved structured output for `INFO` statement](https://github.com/surrealdb/surrealdb/pull/4217) * [Added database upgrade test from `1.5.x` to `2.0.x`](https://github.com/surrealdb/surrealdb/pull/4286) * [Improved `InfoStructure` implementation on `Ident` to return the raw value](https://github.com/surrealdb/surrealdb/pull/4280) * Improved [placeholder text when filling a bug report](https://github.com/surrealdb/surrealdb/pull/4309) * [Seperate the supply chain security actions to its workflow](https://github.com/surrealdb/surrealdb/pull/4349) * [Improve Cargo local testing](https://github.com/surrealdb/surrealdb/pull/4347) * [Limit Cargo build jobs during tests with coverage](https://github.com/surrealdb/surrealdb/pull/4353) * [Improved type safety in router API](https://github.com/surrealdb/surrealdb/pull/4406) * [Improved hashing mechanism to aid MTree benchmarking](https://github.com/surrealdb/surrealdb/pull/4408) * [Removed unnecessary cloning of `DefineStatementAnalyzer`](https://github.com/surrealdb/surrealdb/pull/4379) * [Simplified string formatting](https://github.com/surrealdb/surrealdb/pull/4260) * [Refactored transaction, caching, and key-value store interfaces](https://github.com/surrealdb/surrealdb/pull/4257) * [Imported entry enum in `src/sql/mod.rs`](https://github.com/surrealdb/surrealdb/pull/4375) * [Extracted duplicated code for storing interfaces to initialize\_store](https://github.com/surrealdb/surrealdb/pull/4272) * [Resolved a TODO: map\_err to inspect error](https://github.com/surrealdb/surrealdb/pull/4271) * [Improved pull request template](https://github.com/surrealdb/surrealdb/pull/4440) * [Added Improvements to kvs module layer](https://github.com/surrealdb/surrealdb/pull/4418) * [Updated rustc version to `1.80.0`](https://github.com/surrealdb/surrealdb/pull/4436) * [Enabled the `ml` feature in nightly and alpha releases](https://github.com/surrealdb/surrealdb/pull/4447) * [Removed limit to Cargo jobs](https://github.com/surrealdb/surrealdb/pull/4449) * [Restored `bench-sdk-surrealkv`](https://github.com/surrealdb/surrealdb/pull/4450) * [Upgraded to http/hyper 10](https://github.com/surrealdb/surrealdb/pull/3726) * [Upgrade upstream `jsonwebtoken` crate](https://github.com/surrealdb/surrealdb/pull/4472) * [Bumped SurrealKV version to 0.3.1](https://github.com/surrealdb/surrealdb/pull/4467) * [Added aliases for complex feature flag combinations](https://github.com/surrealdb/surrealdb/pull/4479) * [Improvemed RocksDB range scans](https://github.com/surrealdb/surrealdb/pull/4478) * [Improved Serialisation for `sql::Value`](https://github.com/surrealdb/surrealdb/pull/4255) * Improved [parser error type](https://github.com/surrealdb/surrealdb/pull/4652) * Added Support for [SurrealCS](https://github.com/surrealdb/surrealdb/pull/4458) * Bumped [async graphql version](https://github.com/surrealdb/surrealdb/pull/4687) * Improved [transparent API](https://github.com/surrealdb/surrealdb/pull/4651) * Fixed [cargo dependency loading issue](https://github.com/surrealdb/surrealdb/pull/4660) * Improved [the logic for migrating IDs in `surreal fix`](https://github.com/surrealdb/surrealdb/pull/4715) API improvements: * [Decoupled capabilities from the public API](https://github.com/surrealdb/surrealdb/pull/4173) Surrealkv API improvements: * [Changed disk format for compaction in Surrealkv](https://github.com/surrealdb/surrealkv/pull/45) * [Removed global static Datastore](https://github.com/surrealdb/surrealdb/pull/4377) * Added [versioning to `SELECT * FROM table` queries](https://github.com/surrealdb/surrealdb/pull/4494) Parser improvements: * [Added support for the `s` string prefix in parser](https://github.com/surrealdb/surrealdb/pull/4282) * [Fixed parser panic when a dot follows a string in idiom-allowed contexts](https://github.com/surrealdb/surrealdb/pull/4287) * Ensured [variables after the `FETCH` clause are parsed](https://github.com/surrealdb/surrealdb/pull/4086) * Ensured [UUIDs are parsed correctly](https://github.com/surrealdb/surrealdb/pull/4304) * [Fixed floating point parsing bug](https://github.com/surrealdb/surrealdb/pull/4337) * [Updates reblessive dependency to newest version](https://github.com/surrealdb/surrealdb/pull/4352) * [Removed flume as a dependency in favour of async-channels](https://github.com/surrealdb/surrealdb/pull/4367) * [Fixed warnings and simplified connection trait](https://github.com/surrealdb/surrealdb/pull/4366) Breaking changes: * [Functions beginning with `meta::` renamed to `record::`](https://github.com/surrealdb/surrealdb/pull/4602) * Functions `string::endsWith()` and `string::startsWith()` [renamed to `string::ends_with()` and `string::starts_with()`](https://github.com/surrealdb/surrealdb/pull/4612) * Changed `UPDATE` statement behaviour of creating records in favour of new [`UPSERT` statement](https://surrealdb.com/docs/surrealql/statements/upsert). Experimental features: * Added [`BEARER` access type and its basic grant management](https://github.com/surrealdb/surrealdb/pull/4302)