Make SER_GETHASH implicit for CHashWriter and SerializeHash #13462

pull Empact wants to merge 4 commits into bitcoin:master from Empact:serialize-hash-type changing 12 files +37 −34
  1. Empact commented at 8:42 PM on June 13, 2018: member

    Most invocations of CHashWriter use SER_GETHASH and version 0, this allows for eliding those values, ~and removes SER_GETHASH as a type, because it functions simply as "has no external destination" in practice.~

    ~SerializeHash basically existed due to the overhead of CHashWriter construction, now that that is minimized, it's unnecessary.~

  2. Empact force-pushed on Jun 13, 2018
  3. Empact force-pushed on Jun 13, 2018
  4. Empact renamed this:
    scripted-diff: Simplify common case of CHashWriter and drop SER_GETHASH
    scripted-diff: Simplify common case of CHashWriter and drop SER_GETHASH & SerializeHash
    on Jun 13, 2018
  5. in src/primitives/block.cpp:15 in d697362d10 outdated
      11 | @@ -12,7 +12,7 @@
      12 |  
      13 |  uint256 CBlockHeader::GetHash() const
      14 |  {
      15 | -    return SerializeHash(*this);
      16 | +    return (CHashWriter(PROTOCOL_VERSION) << *this).GetHash();
    


    Empact commented at 8:55 PM on June 13, 2018:

    Note this was the only call that relied on the SerializeHash nVersion default value.

  6. meshcollider added the label Refactoring on Jun 13, 2018
  7. DrahtBot commented at 8:10 PM on June 14, 2018: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #24410 ([kernel 2a/n] Split hashing/index GetUTXOStats codepaths, decouple from coinstatsindex by dongcarl)
    • #24058 (BIP-322 basic support by kallewoof)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  8. Empact commented at 9:06 PM on June 14, 2018: member

    Withdrawing this pending #10785. Will re-open after.

  9. Empact closed this on Jun 14, 2018

  10. sipa commented at 9:35 PM on June 14, 2018: member

    @Empact #10785 is low priority, and will probably take a while. Don't let it stop you.

  11. Empact reopened this on Jun 14, 2018

  12. Empact commented at 9:48 PM on June 14, 2018: member

    Good to know, thanks.

  13. Empact closed this on Jun 14, 2018

  14. Empact reopened this on Jun 14, 2018

  15. sipa commented at 10:15 PM on June 14, 2018: member

    I don't understand "scripted-diff: Drop SER_GETHASH"; it changes !(s.GetType() & SER_GETHASH) to s.GetType(). That means that what used to be the branch for DISK/NETWORK will end up being run just for DISK, and what used to be the branch for GETHASH will end up being run for NETWORK and GETHASH.

  16. Empact commented at 11:23 PM on June 14, 2018: member

    @sipa SER_NETWORK is (1 << 0) i.e. 0x01, SER_DISK is (1 << 1) i.e. 0x02, so both will evaluate to true, while with 0 used where SER_GETHASH once was, it will evaluate to false. Note also while they're defined as flags all uses of SER_GETHASH are singular. https://github.com/bitcoin/bitcoin/pull/13462/files#diff-1fc2d3d7edc00ab8ea29eb1ca30cdcbbR163

  17. sipa commented at 11:27 PM on June 14, 2018: member

    Ah of course; I missed the (1 << ...) around it.

    This isn't very readable code though. Would you mind keeping SER_GETHASH as a constant (perhaps just defined as 0), and explicit comparisons with it (rather than bit masking). That should still be a nice simplification.

  18. Empact force-pushed on Jun 14, 2018
  19. Empact force-pushed on Jun 14, 2018
  20. Empact force-pushed on Jun 15, 2018
  21. Empact renamed this:
    scripted-diff: Simplify common case of CHashWriter and drop SER_GETHASH & SerializeHash
    scripted-diff: Simplify common case of CHashWriter and drop SerializeHash
    on Jun 15, 2018
  22. Empact commented at 12:13 AM on June 15, 2018: member

    @sipa Fair enough, I dropped that commit. Def more straight-forward now.

  23. sipa commented at 12:24 AM on June 15, 2018: member

    Thinking a bit more about this, I think you can actually go further. I believe none of the serializers which have conditionals that mention SER_GETHASH (CDiskBlockIndex, CBlockLocator, CAddress, CKeyPool, CWalletKey, CAccountingEntry, CAccount) are ever actually invoked with SER_GETHASH as nType, so we could literally delete SER_GETHASH entire and all conditions that test for it. SerializeHash could then just use SER_NETWORK afaict.

  24. Empact commented at 12:25 AM on June 15, 2018: member

    @sipa Nice I'll look into that.

  25. Empact force-pushed on Jun 17, 2018
  26. in src/hash.h:126 in f141484483 outdated
     121 | @@ -122,8 +122,8 @@ class CHashWriter
     122 |      const int nVersion;
     123 |  public:
     124 |  
     125 | -    CHashWriter() : CHashWriter(SER_GETHASH, 0) {}
     126 | -    CHashWriter(int nVersionIn) : CHashWriter(SER_GETHASH, nVersionIn) {}
     127 | +    CHashWriter() : CHashWriter(0, 0) {}
    


    sipa commented at 11:37 PM on June 26, 2018:

    Can you use SER_NETWORK instead here? 0 is not a valid serialization type, even if it's not actually used anywhere.


    Empact commented at 5:03 AM on June 28, 2018:

    Reviewing calls to GetType after these changes, it seems like there are only 2 calls that affect the output, both testing GetType for SER_DISK in CAddress::SerializationOp. Is it possible to remove those? If so we could get right of GetType entirely, which seems like a big win.

    If not, it seems like SER_NETWORK would not cause negative effects, but it would be a bit misleading for those cases where the serialization is not propagated to the network.


    Empact commented at 5:10 AM on June 28, 2018:

    Relatedly: #13558 - note that CAddress refers to a node network address, not a wallet address, etc.


    Empact commented at 6:58 AM on June 28, 2018:

    As of #13560, I'm pretty confident that GetType can be removed across the board. Spoke too soon.


    Empact commented at 10:49 AM on December 2, 2018:

    Switched to SER_NETWORK - it's equivalent to 0 now as only SER_DISK has conditional behavior associated with it.

  27. Empact cross-referenced this on Jun 28, 2018 from issue Drop unused GetType() from CSizeComputer by Empact
  28. Empact renamed this:
    scripted-diff: Simplify common case of CHashWriter and drop SerializeHash
    Simplify common case of CHashWriter and drop SerializeHash
    on Jun 28, 2018
  29. DrahtBot cross-referenced this on Jul 31, 2018 from issue [wallet] Kill accounts by jnewbery
  30. DrahtBot closed this on Aug 10, 2018

  31. DrahtBot reopened this on Aug 10, 2018

  32. DrahtBot cross-referenced this on Aug 23, 2018 from issue p2p: Drop CADDR_TIME_VERSION checks now that MIN_PEER_PROTO_VERSION is greater by Empact
  33. DrahtBot added the label Needs rebase on Aug 30, 2018
  34. Empact force-pushed on Sep 1, 2018
  35. DrahtBot removed the label Needs rebase on Sep 1, 2018
  36. DrahtBot cross-referenced this on Sep 7, 2018 from issue Define SIGHASH_MASK in validation and determine the use of SIGHASH_SINGLE in signing by jl2012
  37. DrahtBot cross-referenced this on Sep 7, 2018 from issue uint256: Remove unnecessary crypto/common.h dependency by kallewoof
  38. laanwj referenced this in commit bcffd8743e on Sep 11, 2018
  39. DrahtBot cross-referenced this on Sep 27, 2018 from issue [refactoring] Create common interface for CMutableTransaction and CTransaction with static polymorphism by lucash-dev
  40. in src/hash.h:126 in 5775e8becd outdated
     121 | @@ -122,6 +122,8 @@ class CHashWriter
     122 |      const int nVersion;
     123 |  public:
     124 |  
     125 | +    CHashWriter() : CHashWriter(0, 0) {}
     126 | +    CHashWriter(int nVersionIn) : CHashWriter(0, nVersionIn) {}
    


    practicalswift commented at 8:13 PM on October 23, 2018:

    This should be explicit? :-)

  41. DrahtBot cross-referenced this on Oct 31, 2018 from issue Some simple improvements to the RNG code by sipa
  42. DrahtBot added the label Needs rebase on Dec 1, 2018
  43. Empact force-pushed on Dec 2, 2018
  44. Empact force-pushed on Dec 2, 2018
  45. DrahtBot removed the label Needs rebase on Dec 3, 2018
  46. DrahtBot added the label Needs rebase on Dec 13, 2018
  47. Empact force-pushed on Dec 14, 2018
  48. Empact renamed this:
    Simplify common case of CHashWriter and drop SerializeHash
    Simplify common case of CHashWriter
    on Dec 14, 2018
  49. Empact force-pushed on Dec 14, 2018
  50. Empact renamed this:
    Simplify common case of CHashWriter
    Make SER_GETHASH implicit for CHashWriter and SerializeHash
    on Dec 14, 2018
  51. Empact commented at 10:01 AM on December 14, 2018: member

    In case the SER_GETHASH removal was complicating things, I reoriented this around simply making SER_GETHASH implicit in the obviously hash-specific contexts.

  52. DrahtBot removed the label Needs rebase on Dec 14, 2018
  53. in src/hash.h:127 in d90fb85823 outdated
     122 | @@ -123,6 +123,8 @@ class CHashWriter
     123 |      const int nVersion;
     124 |  public:
     125 |  
     126 | +    CHashWriter() : CHashWriter(SER_GETHASH, 0) {}
     127 | +    CHashWriter(int nVersionIn) : CHashWriter(SER_GETHASH, nVersionIn) {}
    


    practicalswift commented at 2:41 PM on January 7, 2019:

    Should be explicit?


    Empact commented at 7:31 PM on January 7, 2019:

    Indeed, fixed.

  54. Empact force-pushed on Jan 7, 2019
  55. Empact force-pushed on Jan 8, 2019
  56. DrahtBot added the label Needs rebase on Aug 27, 2019
  57. Empact force-pushed on Feb 28, 2020
  58. DrahtBot removed the label Needs rebase on Feb 28, 2020
  59. Empact commented at 10:34 AM on February 29, 2020: member

    Rebased, refined, how about another look?

  60. DrahtBot cross-referenced this on Mar 7, 2020 from issue [WIP] Index for UTXO Set Statistics by fjahr
  61. DrahtBot cross-referenced this on May 27, 2020 from issue Add MuHash3072 implementation by fjahr
  62. DrahtBot cross-referenced this on Jun 3, 2020 from issue Add hash_type MUHASH for gettxoutsetinfo by fjahr
  63. DrahtBot cross-referenced this on Jun 19, 2020 from issue Add gettxoutsetinfo hash_type option by fjahr
  64. DrahtBot added the label Needs rebase on Jul 6, 2020
  65. Empact force-pushed on Apr 13, 2021
  66. Empact force-pushed on Apr 13, 2021
  67. Empact force-pushed on Apr 13, 2021
  68. DrahtBot removed the label Needs rebase on Apr 13, 2021
  69. Empact commented at 4:32 PM on April 13, 2021: member

    Rebased

  70. DrahtBot cross-referenced this on Apr 13, 2021 from issue Coinstats Index by fjahr
  71. laanwj commented at 12:12 PM on April 14, 2021: member

    This has been open for two (almost three) years ! we probably should get to either merging it if it is worth doing, or decide not to, but not keep @Empact rebasing it forever.

    Edit: personally I'm not 100% convinced, I mean, yes the arguments can be elided now in some cases but does this make the code clearer than being explicit?

  72. Empact force-pushed on Apr 14, 2021
  73. Empact commented at 5:39 PM on April 14, 2021: member

    Hi @laanwj - I took 2020 (and then some) off, but it's good to be back at it. As you note, this is old work and may not be worthwhile but here are my thoughts:

    In these contexts, SER_GETHASH is redundant/noise. It's a matter of interpretation whether CHashWriter(0) is more informative than CHashWriter - I tend to think it does not convey information so should be elided for readability.

    It's a bit striking when you consider that of 39 references to SER_GETHASH in master, 30 of them are content-free/noise. This removes those. The original PR was to remove SER_GETHASH entirely, but I walked that back to this more modest change.

    As the bulk of the changes are in the form of a scripted-diff covering the specific cases, I look at this as a low-risk improvement to readability - refinement/readability being important to the long-term health of any software project.

    If the answer to this is "meh," though, fair enough.

  74. practicalswift commented at 7:18 AM on April 15, 2021: contributor

    Personally I don't have any strong opinions about this PR but I do have strong opinions about another thing: very glad to have you back @Empact! Warm welcome back as a contributor! :)

    I took 2020 (and then some) off, but it's good to be back at it.

  75. laanwj commented at 8:00 AM on April 19, 2021: member

    Yes, welcome back!

    If the answer to this is "meh," though, fair enough.

    I hope we can get some people who are more familiar with this specific code to review it. I mean if @sipa is okay with it we should go ahead with it.

  76. DrahtBot cross-referenced this on Apr 19, 2021 from issue refactor: Move more stuff to blockstorage by maflcko
  77. DrahtBot added the label Needs rebase on Apr 30, 2021
  78. Empact force-pushed on Feb 28, 2022
  79. Empact force-pushed on Feb 28, 2022
  80. bitcoin deleted a comment on Feb 28, 2022
  81. bitcoin deleted a comment on Feb 28, 2022
  82. DrahtBot removed the label Needs rebase on Feb 28, 2022
  83. DrahtBot cross-referenced this on Feb 28, 2022 from issue BIP-322 basic support by kallewoof
  84. DrahtBot cross-referenced this on Mar 1, 2022 from issue net: Encapsulate asmap in NetGroupManager by jnewbery
  85. DrahtBot cross-referenced this on Mar 1, 2022 from issue addrman: treat Tor/I2P/CJDNS as a single group by vasild
  86. DrahtBot added the label Needs rebase on Apr 22, 2022
  87. Drop nType argument from SerializeHash
    All callers either relied on the default value of SER_GETHASH or provided
    SER_GETHASH.
    f387b6d512
  88. Add convenience constructors for CHashWriter f9239759bc
  89. scripted-diff: Use new CHashWriter constructors
    -BEGIN VERIFY SCRIPT-
    sed -i -E "s/CHashWriter\(SER_GETHASH, 0\)(.[^{])/CHashWriter()\1/" $(git grep -l 'CHashWriter')
    sed -i -E "s/CHashWriter (.+)\(SER_GETHASH, 0\)/CHashWriter \1/" $(git grep -l 'CHashWriter')
    sed -i -E "s/CHashWriter (.+)\(SER_GETHASH, /CHashWriter \1(/" $(git grep -l 'CHashWriter')
    -END VERIFY SCRIPT-
    7fa972bd42
  90. Include serialize.h wherever SER_GETHASH is referenced c05246bf00
  91. Empact force-pushed on Apr 22, 2022
  92. DrahtBot removed the label Needs rebase on Apr 22, 2022
  93. DrahtBot cross-referenced this on Apr 29, 2022 from issue [kernel 2a/n] Split hashing/index `GetUTXOStats` codepaths, decouple from `coinstatsindex` by dongcarl
  94. Munkybooty referenced this in commit 331ac88012 on Apr 29, 2022
  95. Munkybooty referenced this in commit ed33495fa4 on Apr 29, 2022
  96. Munkybooty referenced this in commit 7467255e4c on Apr 29, 2022
  97. Munkybooty referenced this in commit 966112ed80 on May 6, 2022
  98. Munkybooty referenced this in commit 2ab77afb79 on May 17, 2022
  99. jonatack commented at 5:44 AM on May 18, 2022: contributor

    Approach ACK on this simplification, code looks reasonable on first read.

  100. jonatack approved
  101. jonatack commented at 6:54 AM on May 18, 2022: contributor

    At nearly four years old, this may be the pull that has been open the longest so far.

    ACK c05246bf00bf8ffa5e73ed739a5001dc3639830c code review, clean rebase to master, clean debug build, ran unit tests

  102. Empact commented at 6:13 PM on May 21, 2022: member

    Incidentally, @jonatack do you happen to know the situation with Cirrus-CI? I'd like to retry this build, but I don't have the option to do so on the site.

  103. hebasto commented at 6:20 PM on May 21, 2022: member

    Incidentally, @jonatack do you happen to know the situation with Cirrus-CI? I'd like to retry this build, but I don't have the option to do so on the site.

    Restarted.

  104. DrahtBot commented at 1:41 PM on May 24, 2022: contributor

    <!--cf906140f33d8803c4a75a2196329ecb-->

    🐙 This pull request conflicts with the target branch and needs rebase.

    <sub>Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".</sub>

  105. DrahtBot added the label Needs rebase on May 24, 2022
  106. Munkybooty referenced this in commit d413109da5 on May 30, 2022
  107. maflcko commented at 1:05 PM on June 6, 2022: member

    Not sure on this. I think it might be cleaner to remove ser-type and ser-version completely. See #19477 (comment)

  108. maflcko commented at 10:04 AM on June 10, 2022: member
  109. maflcko commented at 5:27 PM on July 20, 2022: member

    Can this be closed now?

  110. Empact commented at 7:18 AM on July 22, 2022: member

    Closing in favor of #25331

    Thanks, yours is cleaner, but for those brackets. ;)

  111. Empact closed this on Jul 22, 2022

  112. Empact deleted the branch on Jul 22, 2022
  113. Empact restored the branch on Nov 18, 2022
  114. bitcoin locked this on Nov 18, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-20 06:55 UTC