Replace global trickle node with random delays #7125

1. 
   sipa commented at 2:56 PM on November 28, 2015: member

   This is based upon and partially replaces #5989, and rebased on top of #7100.

   We used to have a trickle node, a node which was chosen in each iteration of the send loop that was privileged and allowed to send out queued up non-time critical messages. Since the removal of the fixed sleeps in the network code, this resulted in fast and attackable (see #7123) treatment of such broadcasts.

   This pull request changes each of these trickle-dependant mechanisms with a per-node and per-feature random delay timer, improving privacy and performance by making queueing useful again.

   The commits are mostly squashed/rebased versions taken from #5989 by @pstratem, except:

   • No randomization of tx invs (more code changes, and may not interact well with transactions that need to be sent after their dependencies).
   • No removal of the "1/4 blast to all" behaviour, as #7123 proposes an alternative.
   • No removal of the "announce yourself to outbound peers" behaviour.
2. sipa cross-referenced this on Nov 28, 2015 from issue [WIP] Make trickle logic useful again, delay trickle when past upload limit. by gmaxwell
3. sipa force-pushed on Nov 28, 2015
4. sipa cross-referenced this on Nov 28, 2015 from issue improve addr/inv trickle logic by pstratem
5. sipa force-pushed on Nov 28, 2015
6. sipa force-pushed on Nov 28, 2015
7. sipa force-pushed on Nov 29, 2015
8. 
   sipa commented at 12:21 PM on November 29, 2015: member

   Rebased on top of #7125, and make blocks not use the vInventoryToSend system anymore (making it safer to make it filter like #7100).

9. sipa cross-referenced this on Nov 29, 2015 from issue Replace setInventoryKnown with a rolling bloom filter. by gmaxwell
10. sipa force-pushed on Nov 29, 2015
11. sipa force-pushed on Nov 30, 2015
12. 
    sipa commented at 12:37 PM on November 30, 2015: member

    Rebased on top of #7133.

13. sipa force-pushed on Nov 30, 2015
14. laanwj added the label P2P on Dec 1, 2015
15. sipa cross-referenced this on Dec 3, 2015 from issue Net: Remove "Address refresh broadcast" logic. by pstratem
16. 
    gmaxwell commented at 12:25 AM on December 5, 2015: contributor

    Could use some rebasing love, as the bloomy stuff was merged.

17. sipa force-pushed on Dec 5, 2015
18. 
    sipa commented at 6:25 AM on December 5, 2015: member

    Rebased.

19. laanwj added the label Needs backport on Dec 9, 2015
20. laanwj added this to the milestone 0.12.0 on Dec 9, 2015
21. 
    laanwj commented at 8:02 AM on December 9, 2015: member

    Some nodes connect to to as many peers as possible and flood them with pings, apparently to trigger the issue where pings interfere with trickle delays. So this is being actively exploited.

    Added 0.12 milestone.

22. 
    gmaxwell commented at 11:00 AM on December 10, 2015: contributor

    I am currently testing this on top of 0.12.

23. 
    morcos commented at 4:27 PM on December 10, 2015: member

    utACK

24. in src/main.cpp:None in 33a0ec2c6b outdated

    5390 |          //
    5391 | -        if (fSendTrickle)
    5392 | -        {
    5393 | +
    5394 | +        if (pto->nNextLocalAddrSend < nNow) {
    5395 | +            AdvertizeLocal(pto);
    

    ---

    ---

    

    sdaftuar commented at 4:11 PM on December 11, 2015:

    Should we be guarding this with a !IsInitialBlockDownload() as before?

25. sipa force-pushed on Dec 11, 2015
26. 
    sipa commented at 9:13 PM on December 11, 2015: member

    Addressed @sdaftuar's comment, rebased, and switched to Poisson-distributed sends, so nobody can ever make a guess about when the next send will happen.

27. Replace trickle nodes with per-node/message Poisson delays

    We used to have a trickle node, a node which was chosen in each iteration of
    the send loop that was privileged and allowed to send out queued up non-time
    critical messages. Since the removal of the fixed sleeps in the network code,
    this resulted in fast and attackable treatment of such broadcasts.
    
    This pull request changes the 3 remaining trickle use cases by random delays:
    * Local address broadcast (while also removing the the wiping of the seen filter)
    * Address relay
    * Inv relay (for transactions; blocks are always relayed immediately)
    
    The code is based on older commits by Patrick Strateman.

    5400ef6bcb
28. sipa force-pushed on Dec 11, 2015
29. 
    gmaxwell commented at 12:22 PM on December 14, 2015: contributor

    ACK

30. laanwj merged this on Dec 14, 2015
31. laanwj closed this on Dec 14, 2015

    ---

32. laanwj referenced this in commit 7a5040155e on Dec 14, 2015
33. sipa referenced this in commit 10b88be798 on Dec 14, 2015
34. unknown cross-referenced this on Jan 17, 2016 from issue Improve 0-conf tracking by ghost
35. laanwj removed the label Needs backport on Feb 4, 2016
36. 
    laanwj commented at 8:07 AM on February 4, 2016: member

    This is cherry-picked to 0.12 as 10b88be79856ee7ee66f69705c16b335941e396e

37. rebroad cross-referenced this on Aug 26, 2016 from issue Provide bloom services to whitelisted nodes. by rebroad
38. dgenr8 cross-referenced this on May 18, 2017 from issue Networking improvements by dgenr8
39. str4d cross-referenced this on Jul 14, 2017 from issue Bitcoin 0.12 P2P/Net PRs 1 by str4d
40. 
    dgenr8 commented at 12:06 AM on March 17, 2018: contributor

    Where was the discussion of the choice of 5 seconds as the mean delay between inventory broadcasts? This was 7-10x the median transaction propagation time of the entire network at that time.

41. Fuzzbawls cross-referenced this on Jun 23, 2020 from issue [Net] Pre-requirements for network encapsulation by Fuzzbawls
42. random-zebra referenced this in commit 8bbc0650e6 on Jul 1, 2020
43. sdaftuar cross-referenced this on Dec 3, 2020 from issue p2p: periodically clear m_addr_known by sdaftuar
44. laanwj referenced this in commit 1a9fa4c5ba on Dec 7, 2020
45. sidhujag referenced this in commit bf48f3125c on Dec 7, 2020
46. jnewbery cross-referenced this on Jul 27, 2021 from issue [p2p] Reduce addr blackholes by amitiuttarwar
47. str4d cross-referenced this on Aug 4, 2021 from issue ZIP 239 preparations 1 by str4d
48. zkbot referenced this in commit 65dff184f6 on Aug 4, 2021
49. zkbot referenced this in commit 6f38962577 on Aug 5, 2021
50. zkbot referenced this in commit 1aa59fd4ac on Aug 5, 2021
51. zkbot referenced this in commit 8c3d670c8a on Aug 9, 2021
52. zkbot referenced this in commit 1dc1ababa9 on Aug 9, 2021
53. zkbot referenced this in commit 18fe0a377b on Aug 9, 2021
54. zkbot referenced this in commit 1cb1ed2653 on Aug 10, 2021
55. bitcoin locked this on Feb 15, 2022

Contributors

sipagmaxwelllaanwjmorcossdaftuardgenr8

Milestone
0.12.0