build: Link time garbage collection #18605

pull fanquake wants to merge 2 commits into bitcoin:master from fanquake:link_time_garbage_collection changing 2 files +5 −5
  1. fanquake commented at 12:28 PM on April 12, 2020: member

    Closes #18579.

    Adds the linker flags required to turn on link time garbage collection to the Linux gitian descriptors and Guix build. After some experimentation I'm not convinced this is something that we should do for Windows builds at this stage.

    Size comparison for a gitian build of master (df303ceb650521dc7b1ba91e0eea383c387a5860) vs this PR (8c31ef9ec68549c4105e8115f74f435c71b3901c):

    binary master (df303ceb650521dc7b1ba91e0eea383c387a5860) PR (8c31ef9ec68549c4105e8115f74f435c71b3901c) saving
    bitcoind 10525336 9775672 7%
    bitcoin-cli 2055800 1113624 45%

    | bitcoin-qt | 31896456 | 31114024 | 2.5% | bitcoin-tx | 2376152 | 1372536 | 42% | | bitcoin-wallet | 5899168 | 3199808 | 45% | | test_bitcoin | 17063328 | 15354784 | 10% |

  2. fanquake added the label Build system on Apr 12, 2020
  3. fanquake added the label Needs gitian build on Apr 12, 2020
  4. DrahtBot commented at 1:07 PM on April 12, 2020: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    No conflicts as of last run.

  5. DrahtBot cross-referenced this on Apr 12, 2020 from issue build: Drop make dist in gitian builds by hebasto
  6. DrahtBot cross-referenced this on Apr 12, 2020 from issue build: add linker optimisation flags to gitian & guix (Linux) by fanquake
  7. DrahtBot cross-referenced this on Apr 12, 2020 from issue gitian-linux: Build binaries for 64-bit POWER by luke-jr
  8. laanwj commented at 11:14 AM on April 13, 2020: member

    Concept ACK.

  9. DrahtBot removed the label Needs gitian build on Apr 13, 2020
  10. fanquake cross-referenced this on Apr 14, 2020 from issue scripts: add PE .reloc section check to security-check.py by fanquake
  11. fanquake force-pushed on Apr 15, 2020
  12. fanquake commented at 12:27 AM on April 16, 2020: member

    Rebased for #17929 and added a commit for Guix.

  13. jonasschnelli commented at 6:42 AM on April 16, 2020: contributor

    Nice. Concept ACK. Is there a clang equivalent for --gc-sections? fdata-sections AFAIK is also supported in clang. Could the clang linker flag be -dead_strip?

  14. laanwj cross-referenced this on Apr 16, 2020 from issue Compress PNG images with `zopflipng`. by mikispag
  15. in contrib/gitian-descriptors/gitian-linux.yml:45 in 9f2e435761 outdated 
      43 | @@ -44,9 +44,9 @@ script: |
  44 |    CONFIGFLAGS="--enable-glibc-back-compat --enable-reduce-exports --disable-bench --disable-gui-tests"
  45 |    FAKETIME_HOST_PROGS="gcc g++"
  46 |    FAKETIME_PROGS="date ar ranlib nm"
  47 | -  HOST_CFLAGS="-O2 -g"
  48 | -  HOST_CXXFLAGS="-O2 -g"
  49 | -  HOST_LDFLAGS_BASE="-static-libstdc++ -Wl,-O2"
  50 | +  HOST_CFLAGS="-O2 -g -ffunction-sections -fdata-sections"
    laanwj commented at 4:24 PM on April 16, 2020:

    These flags are effective for the depends build too, I hope?

  16. fanquake commented at 9:36 AM on April 22, 2020: member

    I've been doing some experiments with Windows builds, and passing --gc-sections there currently results in an increase in binary size. I still need to test using a more recent version of binutils.

    Is there a clang equivalent for --gc-sections?

    gc-sections is a linker flag, so as long as Clang forwards it through, and whichever linker you are using understands it, it should work. Note that ld64 on macOS does not.

    fdata-sections AFAIK is also supported in clang.

    Clang supports -ffunction-sections and -fdata-sections. However only for ELF targets.

    -fdata-sections         Place each data in its own section (ELF Only)
-ffunction-sections     Place each function in its own section (ELF Only)

    Could the clang linker flag be -dead_strip?

    We do currently pass dead_strip to ld64 for our macOS builds; and see size reductions. For example, compiling master with and without:

    Binary dead_strip no dead_strip
    bitcoind 10900836 11741428
    bitcoin-qt 28593032 32178676
    bitcoin-cli 466896 1065220
    bitcoin-tx 1196536 1671900
    bitcoin-wallet 3054504 5581956
  17. DrahtBot cross-referenced this on Apr 22, 2020 from issue guix: Make source tarball using git-archive by dongcarl
  18. fanquake referenced this in commit ac21090f20 on Apr 28, 2020
  19. sidhujag referenced this in commit d861869e55 on Apr 28, 2020
  20. fanquake renamed this:
    [WIP] build: Link time garbage collection
    build: Link time garbage collection
    on May 22, 2020
  21. fanquake force-pushed on May 22, 2020
  22. fanquake added the label Needs gitian build on May 22, 2020
  23. fanquake added the label Needs Guix build on May 22, 2020
  24. fanquake marked this as ready for review on May 22, 2020
  25. fanquake force-pushed on May 22, 2020
  26. fanquake commented at 6:44 AM on May 22, 2020: member

    I've updated this to drop the changes for Windows for now. After some more experimenting it wasn't clear that it was working as we'd like and I was still seeing increased binary sizes with newer binutils.

    I have also updated the PR description with a comparison of the size differences for a gitian build of master vs this branch.

  27. fanquake force-pushed on May 22, 2020
  28. DrahtBot removed the label Needs gitian build on May 25, 2020
  29. DrahtBot removed the label Needs Guix build on Jun 18, 2020
  30. bitcoin deleted a comment on Jun 18, 2020
  31. bitcoin deleted a comment on Jun 18, 2020
  32. practicalswift commented at 5:25 AM on September 18, 2020: contributor

    Concept ACK

  33. bitcoin deleted a comment on Sep 24, 2020
  34. build: perform link time garbage collection in gitian
    Adds the linker flag required to turn on link time garbage collection
to the linux gitian descriptors. Combined with `-ffunction-sections` and
`-fdata-sections` this results in a between 5% and 45% reduction in file
size for the binaries we are shipping.
    19b20ee9d7
  35. build: perform link time garbage collection in Guix
    Adds the linker flags required to turn on link time garbage collection
to the Linux Guix build. Combined with `-ffunction-sections` and
`-fdata-sections` this results in a between ~5% and 45% reduction in file
size for the binaries we are shipping.
    d05d56047a
  36. fanquake force-pushed on Nov 22, 2020
  37. fanquake commented at 4:21 AM on November 22, 2020: member

    I rebased this and ran some gitian builds to compare master (816132e6eb23c06287c1e820cc0990d2317c2f10) and this PR (d05d56047a38686c70dc066d010ed8f06990508b) (also included 0.20.1):

    # ls -la bitcoin-0.20.1/bin/
2055800 Jul 31 12:01 bitcoin-cli
31843176 Jul 31 12:01 bitcoin-qt
2376152 Jul 31 12:01 bitcoin-tx
5952832 Jul 31 12:01 bitcoin-wallet
10312344 Jul 31 12:01 bitcoind
16702880 Jul 31 12:01 test_bitcoin

# ls -la bitcoin-816132e6eb23/bin/
4689528 Nov 21 06:47 bitcoin-cli
36844872 Nov 21 06:47 bitcoin-qt
4792088 Nov 21 06:47 bitcoin-tx
10000192 Nov 21 06:47 bitcoin-wallet
15277080 Nov 21 06:47 bitcoind
21103040 Nov 21 06:47 test_bitcoin

# ls -la bitcoin-d05d56047a38/bin
4439576 Apr 15  2020 bitcoin-cli
34407752 Apr 15  2020 bitcoin-qt
4505304 Apr 15  2020 bitcoin-tx
7214880 Apr 15  2020 bitcoin-wallet
14953496 Apr 15  2020 bitcoind
20516928 Apr 15  2020 test_bitcoin

    All of our shipped binaries have certainly grown a bit since 0.20.1. There's still a few mb to be saved in some cases.

  38. dergoegge commented at 1:51 PM on December 9, 2020: member

    I confirmed the space savings by running the gitian builds on master and this PR. I also read up on how the used options (functions-sections, data-sections, gc-sections) work and if there any unwanted side effects.

    Two questions:

    1. AFAICT the options are not active in the depends build. Why not?

    https://github.com/bitcoin/bitcoin/blob/d05d56047a38686c70dc066d010ed8f06990508b/contrib/gitian-descriptors/gitian-linux.yml#L96-L99

    1. The gcc docs warn to only use these options if there is a significant benefit from doing so because certain compiler optimizations wont work while they are enabled. Is it ok to ignore that warning? If so, why?

    From https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html:

    Only use these options when there are significant benefits from doing so. [...] They prevent optimizations by the compiler and assembler using relative locations inside a translation unit since the locations are unknown until link time. An example of such an optimization is relaxing calls to short call instructions.

  39. DrahtBot cross-referenced this on Jan 19, 2021 from issue gitian-linux: Build binaries for 64-bit POWER (continued) by laanwj
  40. fanquake closed this on Feb 17, 2021
  41. fanquake cross-referenced this on Oct 1, 2021 from issue build: add `--enable-lto` configuration option by fanquake
  42. fanquake referenced this in commit 681b25e3cd on Nov 25, 2021
  43. sidhujag referenced this in commit 67658eec7d on Nov 25, 2021
  44. PastaPastaPasta referenced this in commit 0ec2a3b640 on Jun 19, 2022
  45. PastaPastaPasta referenced this in commit f7a94b58d4 on Jun 19, 2022
  46. fanquake cross-referenced this on Aug 3, 2022 from issue Build with -ffunction-sections -fdata-sections -Wl,--gc-sections ? by sipa
  47. bitcoin locked this on Aug 16, 2022
  48. fanquake deleted the branch on Nov 9, 2022
Contributors
fanquakeDrahtBotlaanwjjonasschnellipracticalswiftdergoegge
Labels
Build system
Linked (view graph)
#17929 build: add linker optimisation flags to gitian & guix (Linux)#18579 Build with -ffunction-sections -fdata-sections -Wl,--gc-sections ?#18629 scripts: add PE .reloc section check to security-check.py#18661 Compress PNG images with `zopflipng`.#23152 build: add `--enable-lto` configuration option
github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-20 06:54 UTC