Additional thread safety annotations for CNode/Peer members accessed via the message processing thread #24474

pull ajtowns wants to merge 10 commits into bitcoin:master from ajtowns:202203-msgproc-mutex changing 11 files +132 −142
  1. ajtowns commented at 7:14 PM on March 4, 2022: contributor

    Documents why the existing accesses of elements in net.h's CNode and net_processing.cpp's Peer objects is thread safe, generally by making it so that the compiler will complain if the non-safe accesses are added. Only leaves CNode's m_permissionFlags and m_prefer_evict unenforced by the compiler.

    Adds a global mutex NetEventsInterface::g_mutex_msgproc_thread which is used to document the contract between net_processing and net that ProcessMessages and SendMessages are only invoked from a single thread, and which replaces the per-peer cs_sendProcessing mutex.

  2. DrahtBot added the label P2P on Mar 4, 2022
  3. ajtowns commented at 7:39 PM on March 4, 2022: contributor

    I've pulled the m_mutex_message_handling change out of #21527 since it seems to have been the sticking point and refined it so that it is used to document all the un-annotated net/net_processing CNode/Peer variables in regards to thread-safety. The idea is this PR should just be converting the implicit assumptions we're already making to explicit ones.

  4. ajtowns cross-referenced this on Mar 4, 2022 from issue net_processing: lock clean up by ajtowns
  5. DrahtBot commented at 8:35 PM on March 4, 2022: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #25168 (refactor: Avoid passing params where not needed by MarcoFalke)
    • #24970 (net processing: Move cleanSubVer, fPreferredDownload and nLocalHostNonce to Peer by jnewbery)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  6. DrahtBot cross-referenced this on Mar 4, 2022 from issue refactor: Release cs_main before MaybeSendFeefilter by MarcoFalke
  7. DrahtBot cross-referenced this on Mar 5, 2022 from issue p2p: Replace RecursiveMutex `m_tx_inventory_mutex` with Mutex and rename it by w0xlt
  8. DrahtBot cross-referenced this on Mar 5, 2022 from issue refactor: replace RecursiveMutex `cs_vProcessMsg` with Mutex (and rename) by theStack
  9. DrahtBot cross-referenced this on Mar 5, 2022 from issue net processing: Reduce resource usage for inbound block-relay-only connections by jnewbery
  10. DrahtBot cross-referenced this on Mar 5, 2022 from issue net/net processing: Move tx inventory into net_processing by jnewbery
  11. ajtowns force-pushed on Mar 7, 2022
  12. ajtowns commented at 8:12 AM on March 7, 2022: contributor

    Rebased on top of #24427

  13. DrahtBot cross-referenced this on Mar 12, 2022 from issue BIP324: Add encrypted p2p transport {de}serializer by dhruv
  14. DrahtBot cross-referenced this on Mar 12, 2022 from issue BIP324: Enable v2 P2P encrypted transport by dhruv
  15. DrahtBot cross-referenced this on Mar 12, 2022 from issue net processing: Move remaining globals into PeerManagerImpl by dergoegge
  16. w0xlt commented at 3:46 AM on March 16, 2022: contributor

    Concept ACK.

  17. ajtowns cross-referenced this on Mar 21, 2022 from issue net-processing refactoring -- lose globals, move implementation details from .h to .cpp by ajtowns
  18. jonatack commented at 1:20 PM on March 24, 2022: contributor

    Will have a look soon.

  19. ajtowns force-pushed on Mar 25, 2022
  20. ajtowns commented at 4:52 PM on March 25, 2022: contributor

    Rebased past #21160

  21. DrahtBot cross-referenced this on Mar 29, 2022 from issue refactoring: [Net Processing] Follow-ups to #21160 by jnewbery
  22. DrahtBot cross-referenced this on Apr 5, 2022 from issue test/BIP324: functional tests for v2 P2P encryption by stratospher
  23. DrahtBot cross-referenced this on Apr 20, 2022 from issue Strengthen thread safety assertions by ajtowns
  24. DrahtBot cross-referenced this on Apr 25, 2022 from issue net processing: Move cleanSubVer, fPreferredDownload and nLocalHostNonce to Peer by jnewbery
  25. DrahtBot added the label Needs rebase on Apr 30, 2022
  26. ajtowns force-pushed on May 3, 2022
  27. DrahtBot removed the label Needs rebase on May 3, 2022
  28. DrahtBot cross-referenced this on May 12, 2022 from issue Strengthen AssertLockNotHeld assertions by ajtowns
  29. DrahtBot added the label Needs rebase on May 16, 2022
  30. ajtowns force-pushed on May 16, 2022
  31. ajtowns commented at 2:28 PM on May 16, 2022: contributor

    Rebased past #25109

  32. DrahtBot removed the label Needs rebase on May 16, 2022
  33. DrahtBot cross-referenced this on May 16, 2022 from issue refactor: Pass Peer& to Misbehaving() by MarcoFalke
  34. DrahtBot added the label Needs rebase on May 19, 2022
  35. net/net_processing: add missing thread safety annotations 70a3324d84
  36. net: mark TransportSerializer/m_serializer as const
    The (V1)TransportSerializer instance CNode::m_serializer is used from
    multiple threads via PushMessage without protection by a mutex. This
    is only thread safe because the class does not have any mutable state,
    so document that by marking the methods and the object as "const".
    fca9fadd4b
  37. net: mark CNode unique_ptr members as const
    Dereferencing a unique_ptr is not necessarily thread safe. The reason
    these are safe is because their values are set at construction and do
    not change later; so mark them as const and set them via the initializer
    list to guarantee that.
    021a96d41b
  38. net: note CNode members that are treated as const
    m_permissionFlags and m_prefer_evict are treated as const -- they're
    only set immediately after construction before any other thread has
    access to the object, and not changed again afterwards. As such they
    don't need to be marked atomic or guarded by a mutex; though it would
    probably be better to actually mark them as const...
    5df0bd1e8d
  39. net: add NetEventsInterface::g_mutex_msgproc_thread mutex
    There are many cases where we asssume message processing is
    single-threaded in order for how we access node-related memory to be
    safe. Add an explicit mutex that we can use to document this, which allows
    the compiler to catch any cases where we try to access that memory from
    other threads and break that assumption.
    c8c4faac48
  40. net: drop cs_sendProcessing
    SendMessages() is now protected g_mutex_msgproc_thread; so this additional
    per-node mutex is redundant.
    b1af785346
  41. [move-only] net: move NetEventsInterface before CNode
    This allows CNode members to be marked as guarded by the
    g_mutex_msgproc_thread mutex.
    8e75e74e6a
  42. net: add thread safety annotations for CNode/Peer members accessed only via the msgproc thread 49b72d235c
  43. net_processing: add thread safety annotations for Peer members accessed only via the msgproc thread 3ca9617d45
  44. net_processing: extra transactions data are accessed only via the msgproc thread
    Previously vExtraTxnForCompact and vExtraTxnForCompactIt were protected
    by g_cs_orphans; protect them by g_mutex_msgproc_thread instead, as they
    are only used during message processing.
    49c3b28ab4
  45. ajtowns force-pushed on May 19, 2022
  46. ajtowns commented at 8:32 PM on May 19, 2022: contributor

    Rebased past #22778

  47. DrahtBot removed the label Needs rebase on May 19, 2022
  48. ajtowns cross-referenced this on May 19, 2022 from issue net/net_processing: Add thread safety related annotations for CNode and Peer by ajtowns
  49. ajtowns commented at 3:00 AM on May 20, 2022: contributor

    See #25174 for the first few patches from this.

  50. DrahtBot cross-referenced this on May 20, 2022 from issue refactor: Avoid passing params where not needed by MarcoFalke
  51. DrahtBot added the label Needs rebase on May 20, 2022
  52. DrahtBot commented at 1:40 PM on May 20, 2022: contributor

    <!--cf906140f33d8803c4a75a2196329ecb-->

    🐙 This pull request conflicts with the target branch and needs rebase.

    <sub>Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".</sub>

  53. ajtowns renamed this:
    Additional thread safety annotations for CNode/Peer
    Additional thread safety annotations for CNode/Peer members accessed via the message processing thread
    on May 20, 2022
  54. ajtowns commented at 6:53 AM on August 29, 2022: contributor

    Closing this. If you'd like it resurrected please help #25174 make progress.

  55. ajtowns closed this on Aug 29, 2022

  56. ajtowns cross-referenced this on Sep 7, 2022 from issue net: add NetEventsInterface::g_msgproc_mutex by ajtowns
  57. ajtowns commented at 6:06 PM on September 7, 2022: contributor

    Replaced by #26036

  58. bitcoin locked this on Sep 7, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-20 06:53 UTC