txorphanage: bound orphan memory by storing transactions serialized #36015

pull brunoerg wants to merge 6 commits into bitcoin:master from brunoerg:2026-08-orphanage-mem-bug changing 6 files +317 −131
  1. brunoerg commented at 9:11 PM on August 18, 2026: contributor

    The orphanage limits the "usage" of the orphans it stores, per peer and globally, to bound the amount of memory an attacker can make us hold on to. It uses weight as a proxy for that memory, on the assumption that weight is "often higher than the actual memory usage of the transaction".

    That assumption does not hold for a deserialized transaction. Every witness stack element is an individually heap-allocated vector, costing its 24-byte slot in the stack vector plus a 32-byte minimum allocation, while only weighing 2WU. A transaction of 199,000 1-byte witness elements weighs 398,247WU (i.e. it is of standard weight, and witness standardness cannot be checked while the inputs are missing), but uses 11.1MB of memory: 28 times what is accounted for it, and one such orphan can be retained per peer.

    Rather than change the accounting metric, keep orphans in serialized form, deserializing them again on the paths that hand them back out. Serialized, a transaction's memory usage is bounded by its weight, so the existing weight-based accounting becomes a true upper bound on memory and the worst case is the peers' combined allowances.

    Admission, eviction and accounting behavior are unchanged: no transaction that was previously accepted is refused, so orphan resolution (and thus 1p1c package relay) keeps working for standard-weight transactions whose witnesses consist of many small elements, such as BitVM-style transactions.

  2. DrahtBot commented at 9:11 PM on August 18, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/36015.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    Concept ACK w0xlt, l0rinc, Crypt-iQ
    Stale ACK jeanpablojp

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #35919 (p2p: avoid orphanage abort at high peer counts by l0rinc)
    • #35569 (Encapsulation for CTransaction by purpleKarrot)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

    LLM Linter (✨ experimental)

    Possible typos and grammar issues:

    • // The assignments ensure that every transaction is in some peer's last 2 transactions, and is thus remains in the orphanage until the end of LimitOrphans. -> // The assignments ensure that every transaction is in some peer's last 2 transactions, and thus remains in the orphanage until the end of LimitOrphans. [“is thus remains” is grammatically broken]

    <sup>2026-08-25 17:20:29</sup>

  3. w0xlt commented at 9:15 PM on August 18, 2026: contributor

    Concept ACK.

  4. brunoerg marked this as a draft on Aug 18, 2026
  5. DrahtBot added the label CI failed on Aug 18, 2026
  6. DrahtBot commented at 10:42 PM on August 18, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task Windows native, fuzz, VS: https://github.com/bitcoin/bitcoin/actions/runs/32186423022/job/95870987922</sub> <sub>LLM reason (✨ experimental): CI failed due to an assertion failure in the fuzz test txorphan (txorphan.cpp:182), causing the fuzzer to exit with code 1.</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  7. l0rinc commented at 11:08 PM on August 18, 2026: contributor

    Concept ACK The failing orphanage tests seem related (the musl one is the new test, the fuzz ones need a rebase, iwyu is the bench) and the change mixes refactors and hardening - can you check if it's possible to simplify and do refactors that aren't strictly related in follow-ups instead? Especially since #35923 is related and complements this change, and #35919 is also touching the same area.

  8. brunoerg force-pushed on Aug 18, 2026
  9. brunoerg commented at 11:22 PM on August 18, 2026: contributor

    The failing orphanage tests seem related (the musl one is the new test, the fuzz ones need a rebase, iwyu is the bench) and the change mixes refactors and hardening - can you check if it's possible to simplify and do refactors that aren't strictly related in follow-ups instead?

    My bad on that, forgot to push since my latest local change. Just did it.

  10. brunoerg force-pushed on Aug 19, 2026
  11. DrahtBot removed the label CI failed on Aug 19, 2026
  12. brunoerg marked this as ready for review on Aug 19, 2026
  13. brunoerg commented at 11:33 AM on August 19, 2026: contributor

    Ready for review.

  14. jeanpablojp commented at 3:03 PM on August 21, 2026: contributor

    tACK d48b8afcaee6d08cc5e0c00262d0b7defdc6009f

    Reverted txorphanage back to master and the new functional test fails like it should, "no orphan was evicted". With 10 peers sending one memory-heavy orphan each, master keeps all 10 and the head none.

    I also checked the two numbers in the MAX_ORPHAN_TX_USAGE comment and got 1.24x and 1.47x, matching.

    Left a comment on send_orphan() in the new test, nothing blocking.

  15. in test/functional/p2p_orphan_memory_accounting.py:85 in d48b8afcae
      80 | +    def send_orphan(self, peer, tx):
      81 | +        """Announce tx by wtxid, then serve it when it is requested."""
      82 | +        node = self.nodes[0]
      83 | +        peer.send_and_ping(msg_inv([CInv(t=MSG_WTX, h=int(tx.wtxid_hex, 16))]))
      84 | +        node.bumpmocktime(TXREQUEST_TIME_SKIP)
      85 | +        peer.wait_until(lambda: peer.last_message.get("getdata"))
    


    jeanpablojp commented at 3:03 PM on August 21, 2026:

    This wait_until matches the getdata from the previous call, since last_message isn't cleared. From the second send_orphan() on, the getdata doesn't hold the current tx's wtxid, so the tx goes out unrequested and the inv/getdata path is only exercised on the first call.

    wait_for_getdata([int(tx.wtxid_hex, 16)]) fixes it, which is what p2p_orphan_handling.py does. Swapped it here and it still passes.

  16. DrahtBot requested review from l0rinc on Aug 21, 2026
  17. instagibbs commented at 9:28 AM on August 24, 2026: member

    I'm pretty worried that the new behavior would cause subtle user breakage, epsecially considering something BitVM-like (with lots of 20b and 1b elements in witness data i.e. winternitz sigs).

    Rather than change the metrics entirely, could you consider an alternative where the stored transactions are in serialized form, and deserialized only when required? https://github.com/instagibbs/bitcoin/tree/2026-08-orphanage-serialized-weight

    It's significantly less code, doesn't require the weight metric to be swapped out (it would truly be an overestimation now modulo small constant), limiting the scope of the change. The one cost is the serialize<->deserialize that would be added on round-tripping. From my benchmarks it looks like a non-adversarial large tx would take ~1.5ms on ReconsiderTx, and a BitVM-like one ~3ms. The other costs look negligible.

    Let me know what you think. I'm happy for you to take the code, or to open my own PR.

  18. jeanpablojp commented at 9:05 PM on August 24, 2026: contributor

    @instagibbs I was curious so I built your branch, and I'm sharing some numbers I got in my tests.

    The timing below is for rebuilding the transaction, one input with N items of k bytes.

    witness stack         weight    deserialize   with teardown
    100 x 80B              8,343        0.07 ms         0.08 ms
    1 x 380,000B         380,248         2.4 ms          2.4 ms
    8,000 x 20B          168,245         2.2 ms          2.8 ms
    199,000 x 1B         398,247          28 ms           42 ms
    

    Your ~3ms checks out. It's the item count driving this and not the bytes, and it's paid once, both of the paths a peer can trigger consume the orphan.

    The limit here is 600,000 bytes of memory, not weight. Varying the item size, this is the weight at which it's reached.

    item size    stored up to    % of the range to 400,000 WU lost
      1 byte       21,661 WU                  94.6%
     20 bytes     175,133 WU                  56.2%
     64 bytes     375,035 WU                   6.2%
     65 bytes     330,047 WU                  17.5%
    

    64 to 65 is a MallocUsage step, so the threshold tracks the allocator's buckets and not the transaction.

    I tested on x86-64, best case of several runs.

  19. instagibbs commented at 11:01 AM on August 25, 2026: member

    Note that it's a pretty expensive "attack" in that to cause deserialization in the orphanage you'd have to enter in a valid parent tx into the mempool.

  20. brunoerg commented at 12:11 PM on August 25, 2026: contributor

    I'm pretty worried that the new behavior would cause subtle user breakage, epsecially considering something BitVM-like (with lots of 20b and 1b elements in witness data i.e. winternitz sigs).

    Rather than change the metrics entirely, could you consider an alternative where the stored transactions are in serialized form, and deserialized only when required? https://github.com/instagibbs/bitcoin/tree/2026-08-orphanage-serialized-weight

    It's significantly less code, doesn't require the weight metric to be swapped out (it would truly be an overestimation now modulo small constant), limiting the scope of the change. The one cost is the serialize<->deserialize that would be added on round-tripping. From my benchmarks it looks like a non-adversarial large tx would take ~1.5ms on ReconsiderTx, and a BitVM-like one ~3ms. The other costs look negligible.

    Let me know what you think. I'm happy for you to take the code, or to open my own PR.

    Good point. I haven't tried that approach, but it seems simpler and achieves the same. I'll take a look at your branch, but you can open the PR and move on, no problem. Happy to review.

  21. instagibbs commented at 12:12 PM on August 25, 2026: member

    @brunoerg I'm a little busy, please take it on

  22. iwyu: fix includes in node/txorphanage.h
    Drop includes the header does not use, add the ones it does, and forward
    declare FastRandomContext. The IWYU job checks the files a change touches,
    and the following commits modify this header.
    fa511e751d
  23. scripted-diff: rename DEFAULT_RESERVED_ORPHAN_WEIGHT_PER_PEER
    This constant is the default for TxOrphanage::m_reserved_usage_per_peer,
    returned by ReservedPeerUsage() and compared against UsageByPeer(), all
    of which are named after the "usage" they bound (of type
    TxOrphanage::Usage). Name the constant after that same quantity rather
    than after weight, the metric that happens to measure it, so that it
    matches the field it initializes and the rest of the usage-based API.
    
    The orphanage keeps accounting usage by weight, so this is a pure
    naming change with no change in behavior.
    
    -BEGIN VERIFY SCRIPT-
    sed -i 's/DEFAULT_RESERVED_ORPHAN_WEIGHT_PER_PEER/DEFAULT_RESERVED_ORPHAN_USAGE_PER_PEER/g' $(git grep -l DEFAULT_RESERVED_ORPHAN_WEIGHT_PER_PEER)
    -END VERIFY SCRIPT-
    42da36544a
  24. txorphanage: add GetOrphanUsage() and cache it per announcement
    The orphanage's notion of an orphan's "usage" is currently duplicated in
    the unit tests, the benchmarks and the fuzz targets, all of which call
    GetTransactionWeight() to predict what the orphanage will account. Move
    that knowledge into a single function so that a change of metric only
    has to happen in one place, and so that all users agree on it.
    
    Also cache the value in the Announcement instead of recomputing it on
    every operation. Announcements hold an immutable CTransactionRef, so the
    value never changes; caching it guarantees that PeerDoSInfo::Add() and
    ::Subtract() always use the same number, and makes Erase() (and therefore
    LimitOrphans()) independent of the transaction's size.
    
    No behavior change.
    cabbc4af20
  25. txorphanage: add GetParentTxids() and use it for orphan resolution candidates
    AddTxAnnouncement() only needs the deduplicated prevout txids of an
    announced orphan to consider a peer as an orphan resolution candidate,
    but obtains them by retrieving the full transaction with GetTx(). Serve
    them from the orphanage directly, so that handling an announcement of a
    known orphan does not depend on how the orphanage stores the
    transaction. The following commit stores orphans in serialized form,
    which would otherwise make every such announcement deserialize the
    orphan.
    d068c577b6
  26. txorphanage: store orphans serialized, making weight bound their memory
    The orphanage limits the "usage" of the orphans it stores, per peer and
    globally, to bound the amount of memory an attacker can make us hold on
    to, using weight as a proxy for that memory on the assumption that it
    is "often higher than the actual memory usage of the transaction".
    
    That assumption does not hold for a deserialized transaction: every
    witness stack element is an individually heap-allocated vector, costing
    its 24-byte slot in the stack vector plus a 32-byte minimum allocation,
    while only weighing 2WU. A transaction of 199,000 1-byte witness
    elements weighs 398,247WU (i.e. it is of standard weight, and witness
    standardness cannot be checked while the inputs are missing), but uses
    11.1MB of memory: 28 times what is accounted for it, and one such
    orphan can be retained per peer.
    
    Keep orphans in serialized form instead, deserializing them again on
    the paths that hand them back out, all of which feed into full
    (re)validation or RPC whose cost dwarfs a deserialization. Serialized,
    a transaction's memory usage is bounded by its weight, so the existing
    weight-based accounting becomes a true upper bound on memory and the
    worst case is the peers' combined allowances. Admission, eviction and
    accounting behavior are unchanged: no transaction that was previously
    accepted is refused, so orphan resolution (and thus 1p1c package relay)
    keeps working for standard-weight transactions whose witnesses consist
    of many small elements, such as BitVM-style transactions.
    
    The remaining per-orphan overhead not covered by weight (the entry in
    m_orphans, the entries in m_outpoint_to_orphan_wtxids, and the cached
    prevouts and hashes) is bounded by the latency score limits.
    493561a342
  27. test: cover storing a witness-heavy orphan at its serialized cost
    A standard-weight transaction of 199,000 1-byte witness elements, which
    would use over 10 times its weight in memory if stored deserialized, is
    stored (accounted its weight, which now bounds its memory), fits within
    the announcer's reservation alongside normal orphans, and round-trips
    through the orphanage intact.
    c377e4af30
  28. brunoerg force-pushed on Aug 25, 2026
  29. brunoerg commented at 5:24 PM on August 25, 2026: contributor

    Force-pushed addressing @instagibbs' approach.

  30. instagibbs commented at 7:45 PM on August 25, 2026: member

    title and OP will need updating :+1:

  31. brunoerg renamed this:
    txorphanage: account memory usage instead of weight
    txorphanage: bound orphan memory by storing transactions serialized
    on Aug 25, 2026
  32. brunoerg commented at 8:15 PM on August 25, 2026: contributor

    title and OP will need updating 👍

    Done.

  33. jeanpablojp commented at 11:00 AM on August 26, 2026: contributor

    Built the merge with master and tested again.

  34. in src/node/txorphanage.cpp:763 in c377e4af30
     757 | @@ -666,9 +758,9 @@ std::vector<CTransactionRef> TxOrphanageImpl::GetChildrenFromSamePeer(const CTra
     758 |          --it_upper;
     759 |          if (!Assume(it_upper->m_announcer == peer)) break;
     760 |          // Check if this tx spends from parent.
     761 | -        for (const auto& input : it_upper->m_tx->vin) {
     762 | -            if (input.prevout.hash == parent_txid) {
     763 | -                children_found.emplace_back(it_upper->m_tx);
     764 | +        for (const auto& prevout : it_upper->m_tx_data->m_prevouts) {
     765 | +            if (prevout.hash == parent_txid) {
     766 | +                children_found.emplace_back(it_upper->m_tx_data->MakeTxRef());
    


    jeanpablojp commented at 11:00 AM on August 26, 2026:

    Every matching child is rebuilt before Find1P1CPackage looks at the first one. With 94 orphans of 199,000 1-byte items, one per peer, all announced by a single peer, that's 1,047.6 MB live against the 37.4 MB accounted for it, where one at a time would be 11.1 MB.

    The trigger costs no fee, min relay fee not met already returns TX_RECONSIDERABLE.

  35. in src/test/orphanage_tests.cpp:749 in c377e4af30
     744 | +    BOOST_CHECK_LE(orphanage->UsageByPeer(0), orphanage->ReservedPeerUsage());
     745 | +
     746 | +    // The transaction handed back out deserializes to the original.
     747 | +    const auto ptx_out{orphanage->GetTx(ptx->GetWitnessHash())};
     748 | +    BOOST_REQUIRE(ptx_out != nullptr);
     749 | +    BOOST_CHECK(ptx_out->GetWitnessHash() == ptx->GetWitnessHash());
    


    jeanpablojp commented at 11:00 AM on August 26, 2026:

    This test passes without the change. Reverting just src/node/txorphanage.cpp to before 493561a34285, the whole suite still passes, since the only difference the API exposes is the identity of the returned object.

        BOOST_CHECK(ptx_out != ptx);
        BOOST_CHECK(ptx_out->GetWitnessHash() == ptx->GetWitnessHash());
    

    Fails without the change and passes with it.

  36. Crypt-iQ commented at 7:24 AM on August 29, 2026: contributor

    Concept ACK

  37. l0rinc referenced this in commit 7045787c0c on Sep 3, 2026
  38. l0rinc referenced this in commit 9d159b717a on Sep 3, 2026
  39. instagibbs commented at 9:43 PM on September 8, 2026: member

    When finding potential 1P1C packages, we're now deserializing potentially many orphans, which on my machine can cause ~1s of cpu time at ~1GB of memory usage on top of normal. astra slop for your consideration:

    https://github.com/instagibbs/bitcoin/commit/847d2dc2a0dc76340e269f3f60977b8a5d6bab2a


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-09-09 07:56 UTC