Accurately account for mempool index memory #26614

pull hebasto wants to merge 9 commits into bitcoin:master from hebasto:221130-account changing 7 files +204 −21
  1. hebasto commented at 10:49 PM on November 30, 2022: member

    Picked #18086:

    This introduces a C++ allocator class (memusage::AccountingAllocator) which enables containers that accurately account for all their memory allocations in a tracker variable.

    This is then used to replace the heuristics in the mempool to guess memory usage.

  2. test: Make `mempool_tests/MempoolSizeLimitTest` allocation-neutral 90e4fc7faf
  3. hebasto added the label Mempool on Nov 30, 2022
  4. hebasto added the label Resource usage on Nov 30, 2022
  5. DrahtBot commented at 10:49 PM on November 30, 2022: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #25325 (Add pool based memory resource by martinus)
    • #19909 (refactor: Remove unused CTxMemPool::clear() helper by MarcoFalke)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  6. hebasto cross-referenced this on Nov 30, 2022 from issue Accurately account for mempool index memory by sipa
  7. hebasto force-pushed on Nov 30, 2022
  8. hebasto cross-referenced this on Nov 30, 2022 from issue test: Make `mempool_tests/MempoolSizeLimitTest` allocation-neutral by hebasto
  9. DrahtBot cross-referenced this on Dec 1, 2022 from issue Add pool based memory resource by martinus
  10. DrahtBot cross-referenced this on Dec 1, 2022 from issue refactor: Remove unused CTxMemPool::clear() helper by maflcko
  11. hebasto force-pushed on Dec 1, 2022
  12. Add `AccountingAllocator` to help with memusage tracking
    Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
    3d585bd71d
  13. test: Add `AccountingAllocator` tests
    Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
    5e2f44f957
  14. Account `CTxMemPool::mapTx` accurately 28f2c42e0f
  15. Account `CTxMemPool::mapDeltas` accurately a66e9114b7
  16. Account `CTxMemPool::vTxHashes` accurately 141cbe3755
  17. Add allocator support to `indirectmap` 8cc3b95b2e
  18. Account `CTxMemPool::mapNextTx` accurately b267080105
  19. Account `DisconnectedBlockTransactions::queuedTx` accurately 64b0d3c230
  20. hebasto force-pushed on Dec 1, 2022
  21. hebasto marked this as ready for review on Dec 1, 2022
  22. fanquake commented at 10:55 AM on December 20, 2022: member

    It would be good for the PR description to outline:

    • What is currently wrong with our memory accounting?
      • How inaccurate is it?
    • How (much more) accurate is it after this change?
      • Is the accuracy affected by things like standard library (version), compiler used etc.
    • How does someone reviewing this PR check the accounted memory usage, before and after?

    Maybe the above is all completely obvious from the changes, but I think a PR like this would benefit from having a PR description that isn't just copy-paste from the original 2-year-old PR. Especially since this PR is also missing all the discussion/questions that happened in the original PR (anything relevant could be recapped in the description here).

  23. DrahtBot added the label Needs rebase on Jan 4, 2023
  24. DrahtBot commented at 8:52 AM on January 4, 2023: contributor

    <!--cf906140f33d8803c4a75a2196329ecb-->

    🐙 This pull request conflicts with the target branch and needs rebase.

  25. hebasto closed this on Mar 21, 2023
  26. martinus cross-referenced this on May 25, 2023 from issue util: generalize accounting of system-allocated memory in pool resource by LarryRuane
  27. bitcoin locked this on Mar 20, 2024
Contributors
hebastoDrahtBotfanquake
Labels
MempoolResource usageNeeds rebase
Linked (view graph)
#18086 Accurately account for mempool index memory#26615 test: Make `mempool_tests/MempoolSizeLimitTest` allocation-neutral#27748 util: generalize accounting of system-allocated memory in pool resource
github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-20 06:53 UTC