Flipped btree encoding to allow variable redund blocks

This should have been done as a part of the earlier tag reencoding work,
since having the block at the end was what allowed us to move the
redund-count out of the tag encoding.

New encoding:

  [-- 32-bit csum   --]
  [-- leb128 weight --]
  [-- leb128 trunk  --]
  [-- leb128 block  --]

Note that since our tags have an explicit size, we can store a variable
number of blocks. The plan is to use this to eventually store redundant
copies for error correction:

  [-- 32-bit csum   --]
  [-- leb128 weight --]
  [-- leb128 trunk  --]
  [-- leb128 block  --] -.
  [-- leb128 block  --]  +- n redundant blocks
  [-- leb128 block  --]  |
           ...          -'

This does have a significant tradeoff, we need to know the checksum size
to access the btree structure. This doesn't seem like a big deal, but
with the possibility of different checksum types may be an annoying
issue.

Note that FCRC was also flipped for consistency.
This commit is contained in:
Christopher Haster
2023-06-19 13:17:52 -05:00
parent a5e7ff7be0
commit 799e7cfc81
3 changed files with 30 additions and 28 deletions

View File

@ -93,10 +93,10 @@ def fromtag(data):
return tag>>15, tag&0x7fff, weight, size, 2+d+d_
def frombtree(data):
w, d1 = fromleb128(data)
trunk, d2 = fromleb128(data[d1:])
block, d3 = fromleb128(data[d1+d2:])
crc = fromle32(data[d1+d2+d3:])
crc = fromle32(data)
w, d1 = fromleb128(data[4:])
trunk, d2 = fromleb128(data[4+d1:])
block, d3 = fromleb128(data[4+d1+d2:])
return w, trunk, block, crc
def popc(x):