contrib: fix dead null-byte check in linearize-data.py #36157

pull aman21-droid wants to merge 1 commits into bitcoin:master from aman21-droid:fix/linearize-null-byte-check changing 1 files +1 −1
  1. aman21-droid commented at 6:56 AM on September 3, 2026: none

    linearize-data.py hangs on any block file that ends in zero padding.

    Reproduce:

    mkdir -p /tmp/lin/blocks && cd /tmp/lin
    python3 -c "open('blocks/blk00000.dat','wb').write(bytes(4096))"
    printf '%064d\n' 0 > hashlist.txt
    printf 'input=blocks\nhashlist=hashlist.txt\noutput_file=bootstrap.dat\ngenesis=%064d\n' 0 > linearize.cfg
    python3 /path/to/bitcoin/contrib/linearize/linearize-data.py linearize.cfg
    

    Before, it never returns:

    Read 1 hashes
    Input file blocks/blk00000.dat
    <hangs>
    

    After:

    Read 1 hashes
    Input file blocks/blk00000.dat
    Input file blocks/blk00001.dat
    Premature end of block data
    

    Indexing a bytes object returns an int in Python 3, so inhdr[0] == "\0" is always false and the zero-padding check never fires. The script instead falls through to the magic-bytes search, which advances one byte per iteration; at the end of the file read(8) returns 7 bytes and seek(-7) moves back to the same offset, so it loops there forever and the Premature end of block data path is never reached.

    Block files are preallocated in BLOCKFILE_CHUNK_SIZE (16 MiB) chunks and only truncated once finalized, so the most recent blk*.dat normally ends in padding.

    This is reached whenever the hashlist references blocks that are not in the blocks directory. The most common case is a pruned datadir, which the script explicitly supports - getFirstBlockFileId() starts at the first surviving blk file for exactly that reason. The intended response is "Premature end of block data", but that path is never reached.

    This is the same one-line change as #32978, which was closed because no reproduction was provided. Happy to add a test to test/functional/feature_loadblock.py instead if that is preferred.

  2. contrib: fix dead null-byte check in linearize-data.py
    Indexing a bytes object returns an int in Python 3, so `inhdr[0] == "\0"`
    is always false and the check for zero padding at the end of a block file
    never fires.
    
    Block files are preallocated in 16 MiB chunks and are only truncated once
    finalized, so the most recent blk*.dat normally ends in zero padding. On
    reaching it the script scans forward one byte at a time and then loops
    forever at the end of the file, rather than moving to the next file and
    reporting "Premature end of block data".
    ea4fed5aaa
  3. DrahtBot added the label Scripts and tools on Sep 3, 2026
  4. DrahtBot commented at 6:56 AM on September 3, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/36157.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process. A summary of reviews will appear here.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-09-09 07:56 UTC