Variable security data sharding

Disclaimer: everything here is pure speculative showerthought with many oversimplifications, none of this is to be taken seriously, I only hope there’s some food for thought…

The beautiful idea of danksharding is this: only builders - which is an honest-minority assumption anyway - will need to run expensive hardware. Validators, users and everyone else will have very meagre costs while rollups scale to millions of TPS over time. The issue is, this ambitious vision will take time. Depending on who I ask, anywhere between 2 and 5 years. The crypto space has always suffered from extreme planning fallacy, and while things are definitely improving, I refuse to believe any roadmaps till I at least see full-featured prototypes in action.

EIP-4844 is a huge step forward, and I believe it’ll be more than enough for all applications and valuable transactions that require high security for the foreseeable future, enabling rollups to expand 100x-1,000x from current activity. But there may potentially be different classes of data-hungry applications that may require “millions of TPS” but don’t need high security.

Meanwhile, it’s pretty clear developers aren’t going to be waiting around for full danksharding. Today, 7 out of the top 15 projects on L2Beat are not rollups but validiums and optimistic chains which use external data layers. (I’m assuming you’re familiar with these constructions and aware about why they are still superior to sidechains and the like) StarkEx (now 6 validiums!), Arbitrum and Metis already have external data layers in production; and StarkNet, zkSync, Polygon and others are all building in-house data layers, not to mention the likes of EigenDA or Celestia. These hybrid solutions require neither EIP-4844 nor full danksharding to keep transactions fees dirt cheap and are already here today.

One approach would be to let them do their thing, users will choose the lower security validium options (but still higher than alt-L1s) as and when they need. The magic of volitions is users can select on a per-transaction or per-user basis.

But the other approach is - how can we improve over external data layers?

While this is far from a representative sample, 20% of Ethereum validators today are way over-specced. At the same time, 45% prefer to have very conservative bandwidth/traffic requirements. With EIP-4844, we’re optimizing for the 45%, but the remaining 55% continue to run underutilized. So, this idea is about leveraging this spare bandwidth by adding simple data sharding where the validator set is split. I have zero engineering knowledge, but I’m taking Dankrad’s word that this is “trivial”.

With EIP-4844, we have a new data layer. I’ll call this data shard #0 (DS0). DS0 is mandatory to be secured by the full Ethereum validator set.

From here, we can have more data shards - DS1, DS2…etc. which have optional opt-in from validators. So, the 30% in the 2TB-10TB camp as above can run 2 or 3 data shards. As a result, while DS0 is secured by 100% of ethereum’s validator set, DS1 is 55%, DS2 is 50% and so on. Now, we have those way overspecced validators comfortable with 20TB+ traffic or 100 Mbps+ bandwidth (who have evidently mistaken themselves for Solana validators) running - the 20% - capable of running many more data shards. So, you have the last data shard - DS16 - only offers 10% of Ethereum security. Of course, I make it sound arbitrary, but there’ll probably be some clear divisions according to how the beacon chain committees are structured. But the gist of it is DS0 comes with 100% security - so all rollups will be hre. DS1 with 50% security, DS10 with 25% security, DS16 with 10% security and so on, for a new type of construction that’s somewhere between a full rollup (DS0/100% security) and a validium/optimistic chain.

In reality, as we have seen with the many validiums and data layers going live, the security required for each application, user and usecase is a spectrum. Not everything needs to be secured by hundreds of billions of dollars in economic security.

But here’s the thing - if we assume 30% of ETH supply staked, even the “lowest security” DS16 is still backed by $5 billion in economic security even in this bear market - which is still in the ballpark of top alt-L1 level security. It is also important to remember that this security spectrum is only for data availability. Validity proofs & fraud proofs are still verified with 100% security! So, even a DS16 semi-rollup/semi-validium has significantly higher security properties than a top alt-L1.

This may still be a significant step forward over external data layer options. For sure, high-value financial transactions will opt to settle on DS0, but some medium value NFTs may be just fine with DS8, while for zero value gaming applications even DS16 is probably overkill. As mentioned above, in a volition type setup, each application can choose their level of security as appropriate.

What about rollup users? They simply need to run the data shard(s) relevant to them, so the system requirements will not be different from EIP-4844. This is why even though there’s a security compromise, there’s no decentralization compromise relative to EIP-4844. (Unless it’s a gigarollup that settles over multiple data shards - but that’s something its users will be well aware of)

To be clear, I don’t expect any of this to happen, these are just the ravings of a wildly non-technical amateur blogger. The path of least resistance is EIP-4844 comes in 2023, and once it’s saturated over the years the data-hungry usecases spill over to the many external data layers coming only - data is all set to be highly commodified and abundant. Eventually, years down the line, danksharding goes live gradually for the optimal solution. But maybe someone will read this and find a good idea to fill the large gap between 4844 and full sharding…

Subscribe to polynya
Receive the latest updates directly to your inbox.
Mint this entry as an NFT to add it to your collection.
This entry has been permanently stored onchain and signed by its creator.