rpc: Faster getblock using PureBlock #17529

pull maflcko wants to merge 4 commits into bitcoin:master from maflcko:1911-PureTx changing 20 files +115 −63
  1. maflcko commented at 3:58 AM on November 20, 2019: member

    This adds and uses a new transaction type PureTransaction. The underlying class has no transaction hash compiled in. So it is safe to use (and faster) wherever it compiles.

    The existing CMutableTransaction retains the behavior to calculate the tx hash on demand. Use of CMutableTransaction makes compile-time complexity analysis impossible.

    The existing CTransaction retains the behavior to cache the hash on initialization.

  2. maflcko cross-referenced this on Nov 20, 2019 from issue Skip tx-rehashing on historic blocks by maflcko
  3. maflcko force-pushed on Nov 20, 2019
  4. DrahtBot added the label Build system on Nov 20, 2019
  5. DrahtBot added the label Consensus on Nov 20, 2019
  6. DrahtBot added the label RPC/REST/ZMQ on Nov 20, 2019
  7. DrahtBot added the label Tests on Nov 20, 2019
  8. DrahtBot added the label Validation on Nov 20, 2019
  9. DrahtBot commented at 8:44 AM on November 20, 2019: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #25694 (refactor: Make CTransaction constructor explicit by MarcoFalke)
    • #25665 (BResult improvements, allow returning separate value on failure by ryanofsky)
    • #25232 (rpc: Faster getblock API by AaronDewes)
    • #23599 ([WIP] DRAFT NOMERGE Tidy up RPCTxSerializationFlags by MarcoFalke)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  10. maflcko removed the label Build system on Nov 20, 2019
  11. maflcko removed the label Consensus on Nov 20, 2019
  12. maflcko removed the label Tests on Nov 20, 2019
  13. maflcko commented at 4:12 PM on November 20, 2019: member

    So I did some measurements: While this is a clear performance win, it seems the performance is dominated by shuffling the data around multiple times:

    Before: disk -> deserialize -> hash -> serialize -> hex string -> json

    After: disk -> deserialize -> ____ -> serialize -> hex string -> json

    Hashing takes 10%-25% on intel CPUs. Didn't try arm.

    master: Screenshot from 2019-11-20 10-47-50

    faaa142412:

    Screenshot from 2019-11-20 11-01-35

  14. maflcko commented at 4:22 PM on November 20, 2019: member

    I guess it would be smarter to see if the rpc serialization flags are matching the ones of the block on disk and then just serve it raw. See #13151

  15. maflcko added the label Brainstorming on Nov 20, 2019
  16. DrahtBot added the label Needs rebase on Dec 16, 2019
  17. laanwj commented at 12:17 PM on December 17, 2019: member

    I guess it would be smarter to see if the rpc serialization flags are matching the ones of the block on disk and then just serve it raw. See #13151

    If so, it would definitely be a simpler/smaller change!

  18. luke-jr cross-referenced this on Feb 28, 2021 from issue RPC/Blockchain: Optimise getblock for simple disk->hex case by luke-jr
  19. refactor: Add PureTransaction and PureBlock (no tx hash)
    Those new types are currently unused. The underlying class has no
    transaction hash compiled in. So it is safe to use wherever it compiles.
    
    The existing CMutableTransaction retains the behavior to calculate the
    tx hash on demand. Use of CMutableTransaction makes compile-time
    complexity analysis impossible.
    
    The existing CTransaction retains the behavior to cache the hash on
    initialization.
    39f89b31f9
  20. bench: Add PureBlock deserialization benchmark 31bf7f635d
  21. zmq: Send blocks faster by serializing PureBlock b60e452aef
  22. rpc: Faster getblock by using PureBlock 853cea80b9
  23. maflcko force-pushed on Jul 25, 2022
  24. DrahtBot removed the label Needs rebase on Jul 25, 2022
  25. DrahtBot cross-referenced this on Jul 25, 2022 from issue refactor: Make CTransaction constructor explicit by maflcko
  26. DrahtBot cross-referenced this on Jul 25, 2022 from issue refactor: Add util::Result failure values, multiple error and warning messages by ryanofsky
  27. DrahtBot cross-referenced this on Jul 25, 2022 from issue rpc: Faster getblock API by AaronDewes
  28. DrahtBot cross-referenced this on Jul 26, 2022 from issue [WIP] DRAFT NOMERGE Tidy up RPCTxSerializationFlags by maflcko
  29. DrahtBot commented at 12:17 PM on July 26, 2022: contributor

    <!--cf906140f33d8803c4a75a2196329ecb-->

    🐙 This pull request conflicts with the target branch and needs rebase.

    <sub>Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".</sub>

  30. DrahtBot added the label Needs rebase on Jul 26, 2022
  31. achow101 commented at 6:04 PM on October 12, 2022: member

    Closing this as it has not had any activity in a while. If you are interested in continuing work on this, please leave a comment so that it can be reopened.

  32. achow101 closed this on Oct 12, 2022

  33. bitcoin deleted a comment on Dec 8, 2022
  34. maflcko deleted the branch on Dec 8, 2022
  35. maflcko cross-referenced this on Nov 16, 2023 from issue rpc: Remove deprecated -rpcserialversion by maflcko
  36. bitcoin locked this on Dec 8, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-20 06:54 UTC