refactor: Avoid UniValue copies #25429

pull maflcko wants to merge 3 commits into bitcoin:master from maflcko:2206-uni-copy-๐Ÿ‘พ changing 30 files +179 −174
  1. maflcko commented at 3:17 PM on June 20, 2022: member

    UniValue copies have been a source of performance related bugs and regressions, so it would be good to remove the copy constructors.

    This is the first change toward the goal. This change uses a const UniValue& where possible.

    See also:

  2. maflcko commented at 3:21 PM on June 20, 2022: member

    Review note: const will compile, but kill performance when std::move or RVO is intended.

  3. DrahtBot added the label Refactoring on Jun 20, 2022
  4. theStack commented at 7:51 PM on June 20, 2022: contributor

    Concept ACK

  5. fanquake commented at 9:28 PM on June 20, 2022: member
  6. DrahtBot commented at 11:56 PM on June 20, 2022: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK theStack, hebasto
    Stale ACK martinus, fanquake

    If your review is incorrectly listed, please react with ๐Ÿ‘Ž to this comment and the bot will ignore it on the next update.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #26988 ([rpc]: Add test-only RPC addrmaninfo for new/tried table address count by stratospher)
    • #26840 (refactor: importpubkey, importprivkey, importaddress, importmulti, and importdescriptors rpc by KolbyML)
    • #26780 (rpc: simplify scan blocks by furszy)
    • #26621 (refactor: Continue moving application data from CNode to Peer by dergoegge)
    • #26612 (refactor: RPC: pass named argument value as string_view by stickies-v)
    • #26426 (WIP: Fix coinstatsindex overflow issue by fjahr)
    • #25974 (test, build: Separate read_json function into its own module by hebasto)
    • #25390 (sync: introduce a thread-safe generic container and use it to remove a bunch of "GlobalMutex"es by vasild)
    • #25344 (New outputs argument for bumpfee/psbtbumpfee by rodentrabies)
    • #15294 (refactor: Extract RipeMd160 by Empact)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  7. martinus commented at 5:34 PM on June 21, 2022: contributor

    Nice! concept ACK. How about removing the copy constructor and copy assignment from the UniValue class, as done here? Fixing all copies would make the PR much bigger though.

  8. maflcko commented at 5:56 AM on June 22, 2022: member

    I am actually not sure how to fix all copies. Sometimes the std lib will copy if asked to:

    std::vector<UniValue> a;
    std::vector<UniValue> b;
    a.insert(a.end(), b.begin(), b.end());
    
  9. laanwj commented at 10:14 AM on June 22, 2022: member

    I guess you could add an explicit 'clone' method (like rust) and use that when it's really necessary to make a copy. It will probably make some things much more verbose though.

  10. martinus commented at 3:55 PM on June 22, 2022: contributor

    a.insert(a.end(), b.begin(), b.end());

    In that case you can use

    a.insert(a.end(), std::make_move_iterator(b.begin()), std::make_move_iterator(b.end()));
    

    But I there might still be case where copy constructor is necessary, I remember that I had difficulties in my old PR as well

  11. fanquake commented at 12:11 PM on June 23, 2022: member

    Concept ACK

  12. fanquake approved
  13. fanquake commented at 5:30 PM on June 28, 2022: member

    ACK bbbbcfdb8e87f3151229ebc8c7b812a420d89b14 - as a first step.

  14. maflcko commented at 9:47 AM on June 29, 2022: member

    While the changes are correct, I'll try to make them easier to review and more complete. For now, please review actual UniValue-copy bugfixes, like https://github.com/bitcoin/bitcoin/pull/25464

  15. maflcko marked this as a draft on Jun 29, 2022
  16. maflcko cross-referenced this on Jul 12, 2022 from issue rpc: Reduce Univalue push_backV peak memory usage in listtransactions by maflcko
  17. DrahtBot cross-referenced this on Jul 15, 2022 from issue refactor: univalue test cleanups by fanquake
  18. DrahtBot cross-referenced this on Jul 26, 2022 from issue refactor: Make const references to avoid unnecessarily copying objects and enable two clang-tidy checks by aureleoules
  19. DrahtBot cross-referenced this on Jul 29, 2022 from issue rpc: treat univalue type check error as RPC_TYPE_ERROR, not RPC_MISC_ERROR by furszy
  20. DrahtBot added the label Needs rebase on Aug 19, 2022
  21. maflcko force-pushed on Sep 1, 2022
  22. maflcko force-pushed on Sep 1, 2022
  23. maflcko force-pushed on Sep 1, 2022
  24. DrahtBot removed the label Needs rebase on Sep 1, 2022
  25. DrahtBot cross-referenced this on Sep 8, 2022 from issue refactor: Run type check against RPCArgs (1/2) by maflcko
  26. DrahtBot added the label Needs rebase on Sep 21, 2022
  27. maflcko force-pushed on Dec 21, 2022
  28. DrahtBot removed the label Needs rebase on Dec 21, 2022
  29. maflcko cross-referenced this on Dec 21, 2022 from issue clang-tidy: Add more `performance-*` checks and related fixes by hebasto
  30. DrahtBot cross-referenced this on Dec 22, 2022 from issue rpc: remove optional from fStateStats fields by fanquake
  31. DrahtBot cross-referenced this on Dec 22, 2022 from issue validate package transactions with their in-package ancestor sets by glozow
  32. DrahtBot cross-referenced this on Dec 22, 2022 from issue validation, bugfix: provide more info in *MempoolAcceptResult by glozow
  33. DrahtBot cross-referenced this on Dec 22, 2022 from issue refactor: Continue moving application data from CNode to Peer by dergoegge
  34. DrahtBot cross-referenced this on Dec 22, 2022 from issue WIP: Fix coinstatsindex overflow issue by fjahr
  35. DrahtBot cross-referenced this on Dec 22, 2022 from issue test, build: Separate `read_json` function into its own module by hebasto
  36. DrahtBot cross-referenced this on Dec 22, 2022 from issue sync: introduce a thread-safe generic container and use it to remove a bunch of "GlobalMutex"es by vasild
  37. DrahtBot cross-referenced this on Dec 22, 2022 from issue New `outputs` argument for `bumpfee`/`psbtbumpfee` by rodentrabies
  38. DrahtBot cross-referenced this on Dec 22, 2022 from issue doc: Fix incorrect sendmany RPC doc by maflcko
  39. DrahtBot cross-referenced this on Dec 22, 2022 from issue refactor: Extract RipeMd160 by Empact
  40. hebasto cross-referenced this on Dec 27, 2022 from issue refactor: Add `performance-no-automatic-move` clang-tidy check by hebasto
  41. hebasto commented at 3:53 PM on December 27, 2022: member

    Concept ACK.

  42. DrahtBot cross-referenced this on Jan 1, 2023 from issue rpc: simplify scan blocks by furszy
  43. DrahtBot cross-referenced this on Jan 7, 2023 from issue refactor: importpubkey, importprivkey, importaddress, importmulti, and importdescriptors rpc by KolbyML
  44. DrahtBot cross-referenced this on Jan 11, 2023 from issue refactor: Split util/system into exception, shell, and fs-specific files by Empact
  45. DrahtBot added the label Needs rebase on Jan 11, 2023
  46. maflcko referenced this in commit 9887fc7898 on Jan 11, 2023
  47. sidhujag referenced this in commit 3b0597af60 on Jan 11, 2023
  48. maflcko force-pushed on Jan 11, 2023
  49. DrahtBot removed the label Needs rebase on Jan 11, 2023
  50. DrahtBot cross-referenced this on Jan 13, 2023 from issue RPC: make RPCResult::MatchesType return useful errors by ajtowns
  51. DrahtBot added the label Needs rebase on Jan 17, 2023
  52. maflcko force-pushed on Jan 19, 2023
  53. DrahtBot removed the label Needs rebase on Jan 19, 2023
  54. DrahtBot cross-referenced this on Jan 19, 2023 from issue refactor: RPC: pass named argument value as string_view by stickies-v
  55. DrahtBot added the label Needs rebase on Jan 20, 2023
  56. refactor: Avoid UniValue copies 9530e4f515
  57. test: Use correct type to avoid copy
    std::pair<A, B> is different from std::map<A, B>::value_type (aka
    std::pair<const A, B>). So fix the type to also avoid a copy.
    7568af2c41
  58. WIP fc7ce2b464
  59. maflcko force-pushed on Jan 21, 2023
  60. DrahtBot removed the label Needs rebase on Jan 21, 2023
  61. DrahtBot cross-referenced this on Jan 30, 2023 from issue cli: rework -addrinfo cli to use addresses which arenโ€™t filtered for quality/recency by stratospher
  62. DrahtBot added the label Needs rebase on Jan 30, 2023
  63. DrahtBot commented at 10:29 AM on January 30, 2023: contributor

    <!--cf906140f33d8803c4a75a2196329ecb-->

    ๐Ÿ™ This pull request conflicts with the target branch and needs rebase.

  64. maflcko closed this on Feb 23, 2023

  65. maflcko deleted the branch on Feb 23, 2023
  66. bitcoin locked this on Feb 23, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-20 06:53 UTC