What’s wrong with web3 metadata?
⚙️

What’s wrong with web3 metadata?

Reviewing NFT metadata options, including hosted solutions, distributed file systems (like IPFS), on-chain, and Tableland tables.

What’s wrong with web3 metadata?

A review of the current state of NFT metadata paradigms.

By @Dan Buchholz

The NFT ecosystem exploded over the past few years. Non-fungible tokens are a flexible primitive that range from art to gaming to reputation, and so many more use cases (such as SQL TABLES). One of the biggest challenges has always been the metadata; there are a number of different ways to tackle metadata, but each comes with tradeoffs. One of the many use cases Tableland enables is streamlining NFT metadata storage while also enabling attribute mutability (if desired) in the same stroke. But, why is this needed? How are developers doing this today?

Current metadata workflows

Developers are faced with three common options for storing NFT metadata:

  1. Distributed / decentralized file storage
  2. Hosted / centralized server
  3. On-chain

File storage

The defacto solution for “decentralized” metadata is IPFS. Keep in mind that IPFS is a distributed file system. It is great for storing files using content addressing, which uniquely identifies a file and not its location. Pin the file to the IPFS network, and you have greater guarantees of it’s long-term persistance. For true persistance, you need to use an actual decentralized network with built-in incentives for storing the data. This is where solutions like Filecoin and Arweave come into play.

Notice the emphasis on the word file. This is intentional. File storage solutions are amazing and built for, well, storing files. However, they lack query features. Imagine you upload a JSON metadata file to IPFS and get its CID in return. Okay—now tell me what the value is at the image key, without downloading the file itself, reading / parsing the file as JSON, and then getting the value. Or, take two different JSON metadata files of unrelated collections, and compose their metadata together.

Now, imagine another scenario: you want to mutate an NFT’s existing metadata that is stored at an immutable CID. You can’t. This forces developers to get creative using tactics like tokenURI CID switching. For example:

// Data is pinned to IPFS and hardcoded; this is more so dynamic NFT emualation
string[] IpfsUri = [
    "https://ipfs.io/ipfs/QmYaTsyxTDnrG4toc8721w62rL4ZBKXQTGj9c9Rpdrntou/seed.json",
    "https://ipfs.io/ipfs/QmYaTsyxTDnrG4toc8721w62rL4ZBKXQTGj9c9Rpdrntou/purple-sprout.json",
    "https://ipfs.io/ipfs/QmYaTsyxTDnrG4toc8721w62rL4ZBKXQTGj9c9Rpdrntou/purple-blooms.json"
];

// In the mint function, you use `_setTokenURI` for the 0th value: `IpfsUri[0]`
function safeMint(address to) public {
    uint256 tokenId = _tokenIdCounter.current();
    _tokenIdCounter.increment();
    _safeMint(to, tokenId);
    _setTokenURI(tokenId, IpfsUri[0]);
}

// In some other contract logic, you trigger a change to set the URI to IpfsUri[1] or IpfsUri[2]

These complexities led to teams hosting their metadata using centralized servers.

Hosted server

With centralized solutions (AWS, GCloud, etc.), developers can easily store and mutate metadata. You can even use a combination of the example above where images are stored on IPFS or similar, and the CIDs are stored in the hosted server. It’s a hybrid approach. The primary benefit here is that by running your own database, you have full control over querying the data as well as mutating the values.

If you were to run your own SQL server, you could easily write a SQL query that changes an NFT’s metadata or makes read queries across your database.

Your database.

This obviously limits how NFTs can interact and also detracts entirely from the web3 value proposition. If you are storing data in a centralized server, how does another developer read your data? Can someone else make a read query across your collection? Can some actor outside of your core team, like the NFT’s owner, easily change a piece of data (with your permssion, of course)?

Using a hosted server siloes data and prevents true composability and interoperability. There are a number of projects, such as Valhalla, that currently implement this approach because there isn’t a great way to dynamically change metadata in a web3 native way. (Well, unless you’re familiar with Tableland 😉.)

On-chain

The last pattern developers turn to is writing metadata to contract storage (on-chain maxis, rejoice!). Here, developers put all of the metadata on-chain, most commonly, stored as an SVG. When you make a call to the tokenURI method, the response is this SVG. Take a look at the 1337 5ku11s contract and call this method with a miscellaneous token value. It will return a string that looks something like data:image/svg+xml;base64,PHN2....

This is quite cool! The NFT is interoperable by default. Depending on the contract’s design, the attributes can be composable to where other projects can build on top of the base NFT—such as making derivative collections that, for example, simply toggle traits on/off. Plus, if the contract is immutable (i.e., not using any upgradability patterns), this NFT will exist as long as the chain itself lives. With the file storage or centralized storage options, there is no such guarantee. Persisted file solutions at least provide similar guarantees but within their own network, but again, there are some clear drawbacks to hosting the metadata itself there.

One key consideration here is cost. Upon contract deployment, you must store these large SVG strings directly in contract storage. If you were to try and make the metadata mutable, that means you’d have to overwrite values in contract storage, with is costly…especially, if you’re on Ethereum mainnet.

A better way

Tableland offers the best of all of these approaches.

  1. Store image files on IPFS, Filecoin, or similar
  2. Hosted data in a decentralized fashion—including pointers to the CID
  3. Data is secured on-chain and natively composable using off-chain queries

First and foremost, Tableland is a decentralized SQL database. The process is quite simple:

image

Developers (or contracts) interact with the Tableland “registry” contract to create a table. To write data to the table, there is also an on-chain interaction that writes a SQL statement to event logs. Then, the decentralized network of Tableland validator nodes listen to these SQL events and materialize the instructions in their own instance of SQLite.

Dynamic metadata

Because the table creates and mutations are written to event logs, the base chain’s security and execution model are inherently leveraged. In other words, if Tableland were to suddenly disappear (note: Tableland has no plans to do so!), the entire SQL history is replayable—SQL tables can easily be recreated as long as the base chain lives. Note that the cost is significantly lower in comparison to mutating on-chain storage variables.

One final point: access control. Recall that the centralized approach allows for mutabilty in which one drawback was open access control. Data is siloed to those who can access it. With metadata in Tableland, all data is always openly readable, so composability happens by default. But there’s an added benefit. Since all SQL mutations happen on-chain, the access control rules can also be configured as such.

Let’s take an even simple example: you have a collection and want to allow only the NFT’s owner to update the metadata, such as the background color. Developers can achieve this with smart contract rules that check NFT ownership, and upon the owner trying to mutate a metadata value, these rules impose the SQL constraints to ensure only permissioned data can be changed. You would do some on-chain checks to verify ownership and then allow for a statement like so to be executed:

UPDATE my_nft_table SET background_color="<user_input>" WHERE token_id="<owned_token>"

Blue chips & collections

Is this something that blue chip NFT collections are thinking about today? Definitely—BAYC founder Wylie Aronow has some thoughts about why evolving characters (e.g., traits right in the NFT) have been a compelling idea in previous types of art, even before NFTs: https://youtu.be/qWsXTMBgNEk?t=32. And listen to J1mmy.eth describe his realization about the lack of “on-chain” aspects after purchasing CryptoKitties: https://youtu.be/GrBvHSGtokc?t=209

These are perfect examples about how thought leaders in the space are looking to advance NFT metadata and create new experiences.

Where to next?

The possibilities are endless. NFT collections can implement mutability without the tradeoffs that come with centralized hosting, file storage solutions, or on-chain metadata. However, this functionality isn’t restricted to just NFT collections but is applicable to gaming (leaderboards or reading game state in UIs), utility NFTs (like reputation), web3 tooling (like offering a web3 database deployment option), and anything that else that belongs in a decentralized database!