Engineering Updates | Dec. 2022
⚙️

Engineering Updates | Dec. 2022

Tableland Engineering team will be releasing a number of updates, including changes in SQL features, gateway changes, and a new and improved SDK.

Engineering Updates | Dec. 2022

A quick update on some recent changes and what’s coming for Tableland developers.

Over the past few months, the Tableland core engineering team has been hard at work, thinking through how to better enable our ecosystem of developers to build on the foundation that we have laid out. One area that we found ripe for optimization was the developer experience: docs, APIs, usage patterns, and all the other touch points between you the developer, and Tableland the protocol. As a result of this deep work, a number of changes to the design, implementation, and inner workings of Tableland have been made… and are nearing public launch!

SQL Specification

Tableland’s web3-native flavor of SQL has some important updates coming up! These changes are going to make the developer experience both more pleasant to work with and more robust. Developers can expect a few new constraints, and several new language features. Starting in the new year, there will be…

  • No JOINs between tables that cross the testnet and mainnet divides,
  • No REAL or ANY values (blog post on this one coming out soon),
  • More powerful writes with SELECT subquery support

Why no queries across testnets and mainnets? There is some additional context below in Gateway Changes, but the gist of it is that testnets and mainnets really should not interact. Plain and simple!

One change that might surprise some developers is that support for REAL and ANY data types has been dropped. This is due to problems inherent in floating point math, and is a problem that exists in any blockchain. Numbers need to be deterministic, and floating point math is, by definition, an approximation (and can even change based on a computer’s architecture!). So this means that the standard SQL REAL data type most definitely cannot be supported on Tableland. Unfortunately, since ANY is sort of a catch-all and could include REAL under the hood, support for this datatype must also be deprecated.

These deprecations from the SQL spec come with a silver lining: support for subqueries within writes is coming! The previous iteration of Tableland SQL only allowed subqueries within a JOIN. But now, support for subqueries in writes is also being added. A few caveats:

  • Only support INSERT queries, not UPDATE yet
  • Cannot support UNION, JOIN, and deeper subqueries (only flattened SELECT with direct table access)
  • Only allow references to tables on the same chain
  • Note that the keywords HAVING and GROUP BY are not part of this feature and will be blocked until future research shows there is no unintended (non-deterministic) behavior.

Gateway Changes

Gateways will be changing to reflect specific testnet and mainent chains:

  • testnets.tableland.network ⇒ Testnet chains only (notice the s on testnets)
  • tableland.network ⇒ Mainnet chains only

The former testnet.tableland.network will be sunset. There will be a period of time where this gateway will redirect to the proper gateways above, so developers should have ample time to make the proper changes. These gateways are live today, so developers can actively already make the switch!

One major reason this separation is taking place is to lay the groundwork for our upcoming Tableland mainnet (2023). The Tableland validator nodes were previously running a single node for both testnet and mainnet chains, which wouldn’t be ideal from a validator’s perspective since it would increase the node’s requirements.

Additionally, testnet to mainnet queries shouldn’t really be possible. It doesn’t make sense for a mainnet application to be relying on testnet data, and vice versa. With the gateway and node separation, this ensures data that lives on tesnets and mainnets are logically separated, down to the core Tableland protocol itself.

tokenURI & Associated Code Changes

The above gateway changes mean that any reference to testnet.tableland.network should be removed and replaced with the gateways noted above. NFT collections that use Tableland point their metadata to the Tableland gateway using methods like tokenURI and contractURI. These should point to the correct gateway based the testnet or mainnet chain environment. If any smart contract method or storage variable points to the Tableland testnet (singular, not testnets) network, it should also be updated accordingly.

Those are the most high impact areas to be aware. Of course, any off-chain code should also be updated, accordingly (e.g., projects calling the Tableland gateway via an off-chain REST API).

If you have launched a mainnet project that is not upgradable nor has methods to re-set references to the testnet.tableland.network gateway, please open a #support-request in the Tableland Discord.

SDK

The new (upcoming) Tableland SDK will be Cloudflare D1 compatible, and it will automatically read from the correct gateway as defined above! It will also wrap just about all of the above changes into an easy to use interface that means… you barely have to think about where you target your queries.

The Tableland SDK was initially built with single purpose methods. For example, to create a table, you would use create ; to write to a table, write; to read, read. When you look at the broader SQL ecosystem (outside of web3), there are a number of great SDKs that make the process of writing SQL in JavaScript a smooth experience. Cloudflare D1 was one in particular; it is extremely intuitive and simplifies all of those purpose specific methods into a single prepare, making it much easier to interact with the Tableland network. Our team is already finding the new APIs much more intuitive, and quick to develop with, and we think you will too!

Sneak Peak

const { meta: create } = await db
  .prepare("CREATE TABLE demo (id integer, desc text);")
  .run();
const { name } = await create.txn.wait();
const { meta: insert } = await db
  .prepare(
    `INSERT INTO ${name} (id, desc)
   VALUES (1, 'howdy'), (2, 'hi'), (3, 'hello'), (4, 'world');`
  )
  .run();
await insert.txn.wait();
const stmt = db
	.prepare(`SELECT * FROM ${name} WHERE id < ?;`)
	.bind(3);
const { results } = await stmt.all();
console.log(results)
// [
//   { id: 3, desc: "hello" },
//   { id: 4, desc: "world" },
// ]