gitian: Improve error handling #15549

pull laanwj wants to merge 1 commits into bitcoin:master from laanwj:2019_03_gitian_error_handling changing 6 files +12 −3
  1. laanwj commented at 5:29 PM on March 6, 2019: member

    Improve error handling in gitian builds:

    • Set fail-on-error and pipefail flag, this causes a command to fail when either of the pipe stages fails, not only when the last of the stages fails, so this improves error detection.
    • Also use xargs instead of find -exec, because find will not propagate errors in the executed command, but xargs will.

    This will avoid some issues like #15541 where non-determinism is silently introduced due to errors caused by environment conditions (such as lack of disk space in that case).

  2. laanwj added this to the milestone 0.18.0 on Mar 6, 2019
  3. laanwj force-pushed on Mar 6, 2019
  4. laanwj cross-referenced this on Mar 6, 2019 from issue Reproducibility issue with 0.18.0rc1 by fanquake
  5. MarcoFalke added the label Needs backport on Mar 6, 2019
  6. MarcoFalke added the label Build system on Mar 6, 2019
  7. hebasto commented at 6:23 PM on March 6, 2019: member

    Concept ACK.

  8. laanwj requested review from theuni on Mar 6, 2019
  9. MarcoFalke added the label Needs gitian build on Mar 6, 2019
  10. practicalswift commented at 9:12 PM on March 6, 2019: contributor

    Concept ACK

  11. laanwj commented at 12:17 AM on March 7, 2019: member

    #15541 (comment) :

    I see the default size for the LXC disk is 10240MB maybe that's no longer enough

    Independently of this we should probably increase the default LXC base VM image size (here https://github.com/devrandom/gitian-builder/blob/master/bin/make-base-vm#L267 ), or make the descriptor clean up the intermediate result after each host build. I think I have a slight preference for the latter as it doesn't require everyone to regenerate—or resize—their base images.

  12. luke-jr commented at 7:06 AM on March 7, 2019: member

    Related: https://github.com/devrandom/gitian-builder/pull/172

  13. DrahtBot removed the label Needs gitian build on Mar 7, 2019
  14. DrahtBot commented at 12:29 PM on March 7, 2019: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #15554 (docs: binary tar improvement by cisba)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  15. cisba cross-referenced this on Mar 7, 2019 from issue docs: binary tar improvement by cisba
  16. in contrib/devtools/split-debug.sh.in:2 in 9793a5ca75 outdated 
       0 | @@ -1,5 +1,5 @@
   1 |  #!/bin/sh
   2 | -
   3 | +set -e -o pipefail
    MarcoFalke commented at 3:05 PM on March 7, 2019:

    DrahtBot fails with

    ../contrib/devtools/split-debug.sh: 2: set: Illegal option -o pipefail
    MarcoFalke commented at 3:07 PM on March 7, 2019:

    On Ubuntu the default shell is dash (aka Debian Almquist Shell), to which /bin/sh is symlink. When your shell script is run with #!/bin/sh, you are effectively trying to run it with the default shell. However, dash doesn't have the pipefail option, which is why you're getting the error.

    https://askubuntu.com/a/886540

    laanwj commented at 4:17 PM on March 7, 2019:

    Oh crap. I hate shellscript programming so much.

    laanwj commented at 4:20 PM on March 7, 2019:

    Oh wait, is this only about split-debug.sh.in, not the main gitian scripts? I don't think the pipefail matters there at all as the script doesn't use pipes.

    Edit: removed it here, please retry

  17. laanwj closed this on Mar 7, 2019
  18. laanwj reopened this on Mar 7, 2019
  19. gitian: Improve error handling 32da92bdf6
  20. laanwj force-pushed on Mar 7, 2019
  21. bitcoin deleted a comment on Mar 7, 2019
  22. MarcoFalke added the label Needs gitian build on Mar 7, 2019
  23. DrahtBot commented at 10:49 PM on March 8, 2019: contributor

    <!--a722867cd34abeea1fadc8d60700f111-->

    Gitian builds for commit d211edb34982aedbcd514ba3723396f2369ff286 (master):

    Gitian builds for commit 6eb1a44d8d316c144d96924ca5c55dac35bc0ca3 (master and this pull):

  24. DrahtBot removed the label Needs gitian build on Mar 8, 2019
  25. MarcoFalke commented at 11:18 PM on March 8, 2019: member

    utACK 32da92bdf6bb55d6d312b0f85797d439cc942db5

  26. laanwj referenced this in commit f810f14cf6 on Mar 9, 2019
  27. laanwj merged this on Mar 9, 2019
  28. laanwj closed this on Mar 9, 2019
  29. laanwj referenced this in commit 12408d33c6 on Mar 9, 2019
  30. fanquake removed the label Needs backport on Mar 9, 2019
  31. fanquake commented at 5:49 AM on March 9, 2019: member

    This has been backported to 0.18.0 in https://github.com/bitcoin/bitcoin/commit/f810f14cf61db3968b905f84c3eea3056f89c2a8.

  32. HashUnlimited referenced this in commit f22c67f72f on Mar 10, 2019
  33. hebasto cross-referenced this on Mar 10, 2019 from issue build: Default size for the LXC disk is no longer enough by hebasto
  34. hebasto commented at 5:56 PM on March 14, 2019: member

    Just noted that -e option is already set in bin/gbuild:

     script.puts "set -e"

    So set -o pipefail (without -e) could be sufficient.

  35. hebasto cross-referenced this on Mar 16, 2019 from issue Increase the LXC disk size from 10241 MiB to 12 GiB by hebasto
  36. laanwj cross-referenced this on Jul 8, 2019 from issue contrib: Enable building in Guix containers by dongcarl
  37. PastaPastaPasta referenced this in commit 7e64f9a6fe on Jun 27, 2021
  38. PastaPastaPasta referenced this in commit f8134bc157 on Jun 28, 2021
  39. PastaPastaPasta referenced this in commit 6ae817a5b7 on Jun 29, 2021
  40. PastaPastaPasta referenced this in commit 009543c188 on Jul 1, 2021
  41. PastaPastaPasta referenced this in commit 9a144a5e55 on Jul 1, 2021
  42. PastaPastaPasta referenced this in commit 2a9cc9afdc on Jul 8, 2021
  43. PastaPastaPasta referenced this in commit a33756c72e on Jul 10, 2021
  44. bitcoin locked this on Dec 16, 2021
Contributors
laanwjhebastopracticalswiftluke-jrDrahtBotMarcoFalkefanquake


theuni

Labels
Build system

Milestone
0.18.0

Linked (view graph)
#15277 contrib: Enable building in Guix containers#15541 Reproducibility issue with 0.18.0rc1#15554 docs: binary tar improvement#15574 build: Default size for the LXC disk is no longer enough
github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-19 06:54 UTC