ci: Upgrade most ci configs to focal #19267

pull MarcoFalke wants to merge 4 commits into bitcoin:master from MarcoFalke:2006-ciFocal changing 9 files +47 −30
  1. MarcoFalke commented at 1:12 PM on June 13, 2020: member

    Generally developers compile with recent compilers, so bumping the ci configs to a recent OS should be uncontroversial. Older OSes (especially with compiler sanitizers) need workarounds that can be dropped by running on a more recent OS.

    This pull changes the asan sanitizer and the experimental multiprocess build to use focal. Also, it runs the no_wallet config on xenial to test against python 3.5, according to doc/dependencies.md.

    Finally, all configs that mimic gitian (win and mac) will stay at bionic.

  2. MarcoFalke added the label Tests on Jun 13, 2020
  3. MarcoFalke force-pushed on Jun 13, 2020
  4. in .travis.yml:130 in fa062788c3 outdated 
     114 | @@ -129,13 +115,12 @@ jobs:
 115 |          FILE_ENV="./ci/test/00_setup_env_native_fuzz.sh"
 116 |  
 117 |      - stage: test
 118 | -      name: 'x86_64 Linux [GOAL: install]  [bionic]  [multiprocess]'
 119 | -      if: type != pull_request OR commit_message =~ /depends:|multiprocess:/ # Skip on non-depends, non-multiprocess PRs
    hebasto commented at 1:42 PM on June 13, 2020:

    Why this if: removed?

    MarcoFalke commented at 1:45 PM on June 13, 2020:

    I think it is not worth the risk of having the build go broken unnoticed as opposed to the cost of needing one more vm. The travis contract is expiring this month and we will either renew it or buy a contract at a different provider.

  5. MarcoFalke commented at 2:21 PM on June 13, 2020: member

    For some reason this doesn't link:

    /usr/bin/ld: qt/libbitcoinqt.a(libbitcoinqt_a-bitcoin.o): in function `GuiMain(int, char**)':

/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/qt/bitcoin.cpp:444: undefined reference to `qInitResources_bitcoin()'

/usr/bin/ld: /home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/qt/bitcoin.cpp:445: undefined reference to `qInitResources_bitcoin_locale()'

clang: error: linker command failed with exit code 1 (use -v to see invocation)

    locally everything works fine, but on travis not: https://travis-ci.org/github/bitcoin/bitcoin/jobs/697962112#L4091

  6. hebasto commented at 2:42 PM on June 13, 2020: member

    For some reason this doesn't link:

    /usr/bin/ld: qt/libbitcoinqt.a(libbitcoinqt_a-bitcoin.o): in function `GuiMain(int, char**)':

/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/qt/bitcoin.cpp:444: undefined reference to `qInitResources_bitcoin()'

/usr/bin/ld: /home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/qt/bitcoin.cpp:445: undefined reference to `qInitResources_bitcoin_locale()'

clang: error: linker command failed with exit code 1 (use -v to see invocation)

    locally everything works fine, but on travis not: https://travis-ci.org/github/bitcoin/bitcoin/jobs/697962112#L4091

    Could you try to add #include <QDir> into bitcoin.cpp?

  7. DrahtBot commented at 2:48 PM on June 13, 2020: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #19321 (ci: Run asan ci config on cirrus by MarcoFalke)
    • #19179 ([WIP RFC DONOTMERGE] ci: Run ci configs on cirrus by MarcoFalke)
    • #19077 (wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets by achow101)
    • #18912 (ci: Run fuzz testing test cases (bitcoin-core/qa-assets) under valgrind to catch memory errors by practicalswift)
    • #18870 (build: Allow BDB between 4.8 and 5.3 without --with-incompatible-bdb by achow101)
    • #18077 (net: Add NAT-PMP port forwarding support by hebasto)
    • #16546 (External signer support - Wallet Box edition by Sjors)
    • #15382 (util: add runCommandParseJSON by Sjors)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  8. kiminuo referenced this in commit 7d70344729 on Jun 13, 2020
  9. kiminuo commented at 3:52 PM on June 13, 2020: contributor

    (@hebasto Different PR but similar error. However, your suggestion did not help me: https://travis-ci.com/github/kiminuo/bitcoin/jobs/348750106 https://travis-ci.com/github/kiminuo/bitcoin/jobs/348750106#L4065)

  10. DrahtBot cross-referenced this on Jun 13, 2020 from issue ci: Run ci configs on cirrus by MarcoFalke
  11. MarcoFalke force-pushed on Jun 13, 2020
  13. MarcoFalke commented at 4:59 PM on June 13, 2020: member

    Nice catch. Though,

    • Why does this error not abort the build immediately?
    • Why does it not reproduce locally?
  14. hebasto commented at 5:51 PM on June 13, 2020: member

    Travis is happy with 17dc996ec5c7a53627396b0dbd6a2b61804b9ba4 :)

  15. kiminuo referenced this in commit 0ef40ea4f8 on Jun 13, 2020
  16. DrahtBot cross-referenced this on Jun 13, 2020 from issue ci: Run fuzz testing test cases (bitcoin-core/qa-assets) under valgrind to catch memory errors by practicalswift
  17. DrahtBot cross-referenced this on Jun 13, 2020 from issue build: Allow BDB between 4.8 and 5.3 without --with-incompatible-bdb by achow101
  18. DrahtBot cross-referenced this on Jun 13, 2020 from issue net: Add NAT-PMP port forwarding support by hebasto
  19. fanquake commented at 1:16 AM on June 14, 2020: member

    Concept ACK.

    Generally developers compile with recent compilers, so bumping the ci configs to a recent OS should be uncontroversial.

    If we're going to do this I think we should make a decision in #18921. Otherwise the way we compile and test in our CI (as well as the way some developers are compiling locally) is starting to diverge from the way we actually build releases.

    For 0.21 this will be Ubuntu Bionic with python3.6.

    Is the minimum supported Python being bumped to 3.6 for 0.21, or are we just not going to test Python 3.5 compat any more? You said here:

    https://packages.ubuntu.com/xenial/python3 is still supported and not EOL until April next year, so we can't bump to python 3.6 yet.

  20. MarcoFalke force-pushed on Jun 14, 2020
  21. MarcoFalke referenced this in commit fa654ee1ff on Jun 14, 2020
  22. MarcoFalke force-pushed on Jun 14, 2020
  23. MarcoFalke referenced this in commit fa71186ccd on Jun 14, 2020
  24. MarcoFalke force-pushed on Jun 14, 2020
  25. MarcoFalke commented at 12:14 PM on June 14, 2020: member

    starting to diverge from the way we actually build releases.

    I don't mind bumping gitian to focal, but I think that discussion should happen elsewhere. I kept the three gitian-mimic configs at bionic (win64, mac, and the C++17 linux build). Though, sanitizers are not supported by gitian, so I think requiring them to use the older LTS and patch them with workarounds might not be the best way forward.

    Is the minimum supported Python being bumped to 3.6 for 0.21, or are we just not going to test Python 3.5 compat any more?

    I am testing python 3.5 compat, but most of my testing has moved out of this repo. Though, I've switched the existing no_wallet config to xenial in the last commit to make it a bit more explicit that python3.5 compat is not changed by this pull.

  26. MarcoFalke referenced this in commit d8e83f8cc7 on Jun 14, 2020
  27. MarcoFalke force-pushed on Jun 14, 2020
  28. practicalswift commented at 4:57 PM on June 14, 2020: contributor

    Concept ACK

  29. MarcoFalke referenced this in commit fa650d96a8 on Jun 15, 2020
  30. MarcoFalke force-pushed on Jun 15, 2020
  31. MarcoFalke commented at 11:56 AM on June 15, 2020: member

    Force pushed to get rid of the already merged commit

  32. DrahtBot cross-referenced this on Jun 17, 2020 from issue wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets by achow101
  33. DrahtBot cross-referenced this on Jun 17, 2020 from issue UI external signer support (e.g. hardware wallet) by Sjors
  34. DrahtBot cross-referenced this on Jun 17, 2020 from issue External signer support - Wallet Box edition by Sjors
  35. DrahtBot cross-referenced this on Jun 17, 2020 from issue util: add RunCommandParseJSON by Sjors
  36. laanwj commented at 2:03 PM on June 17, 2020: member

    Concept ACK. I think focal is a better reflection of the compiler and binutils most people are using than bionic, at this point.

    Though I agree with @fanquake's point that we're still building releases using bionic, so we need to test that.

    However, there should be at least one config to test compile and run-time compatibility on the oldest supported OS. For 0.21 this will be Ubuntu Bionic with python3.6.

    This is somewhat confusing to me—compile-time and run-time compatibility was never the same. Run-time compatibility (at least for pre-built releases) historically at least tends to back farther than we support building on. But I suppose backward-testing gitian releases isn't part of the CI so it doesn't matter here.

  37. DrahtBot cross-referenced this on Jun 19, 2020 from issue ci: Run asan ci config on cirrus by MarcoFalke
  38. MarcoFalke cross-referenced this on Jun 19, 2020 from issue [WIP DONOTMERGE] Replace boost with C++17 (std::shared_mutex) by MarcoFalke
  39. travis: Always run multiprocess build fa6ddb2fa1
  40. MarcoFalke referenced this in commit faafabb4a9 on Jun 19, 2020
  41. MarcoFalke force-pushed on Jun 19, 2020
  42. DrahtBot added the label Needs rebase on Jun 19, 2020
  43. MarcoFalke commented at 2:34 PM on June 19, 2020: member

    doc/dependencies.md says it is python 3.5, so I went ahead and updated OP

  44. DrahtBot removed the label Needs rebase on Jun 19, 2020
  45. ci: Have one config run in xenial to test against python3.5
    Also, bump the travis env to bionic. This shouldn't matter at all
because all ci configs run inside a docker, but it does seem to fix a
bug. See
https://github.com/bitcoin/bitcoin/pull/19267#issuecomment-643630309
    fa880773b4
  46. doc: move doc to ci readme fad6720891
  47. ci: Upgrade most ci configs to focal fa05f44893
  48. MarcoFalke force-pushed on Jun 19, 2020
  49. MarcoFalke commented at 2:45 PM on June 19, 2020: member

    re-arranged commits to minimize diff

  50. Sjors commented at 3:07 PM on June 19, 2020: member

    ACK fa05f44893d228f672f39436d0cb6b3376f81ac2, assuming Travis passes (it hasn't shown up in the checks yet)

  51. MarcoFalke commented at 4:15 PM on June 19, 2020: member

    (it hasn't shown up in the checks yet)

    It shows for me:

    Screenshot_2020-06-19 ci Upgrade most ci configs to focal by MarcoFalke · Pull Request #19267 · bitcoin bitcoin

  52. in .travis.yml:120 in fa05f44893 
     118 |        env: >-
 119 |          FILE_ENV="./ci/test/00_setup_env_native_multiprocess.sh"
 120 |  
 121 |      - stage: test
 122 | -      name: 'x86_64 Linux  [GOAL: install]  [bionic]  [no wallet]'
 123 | +      name: 'x86_64 Linux  [GOAL: install]  [xenial]  [no wallet]'
    hebasto commented at 4:21 PM on June 19, 2020:

    Why xenial?

    MarcoFalke commented at 4:25 PM on June 19, 2020:

    Please refer to the commit message and pull request description for the motivation behind changes

    hebasto commented at 4:32 PM on June 19, 2020:

    I see.

  53. hebasto approved
  54. hebasto commented at 4:33 PM on June 19, 2020: member

    ACK fa05f44893d228f672f39436d0cb6b3376f81ac2

  55. MarcoFalke merged this on Jun 19, 2020
  56. MarcoFalke closed this on Jun 19, 2020
  57. MarcoFalke deleted the branch on Jun 19, 2020
  58. Sjors commented at 4:57 PM on June 19, 2020: member

    I show up for me now as well.

  59. stackman27 referenced this in commit a50fe718f0 on Jun 26, 2020
  60. sidhujag referenced this in commit 558a1b0739 on Jul 7, 2020
  61. janus referenced this in commit 7c67a54475 on Nov 22, 2020
  62. bitcoin locked this on Feb 15, 2022
Contributors
MarcoFalkehebastoDrahtBotkiminuofanquakepracticalswiftlaanwjSjors
Labels
Tests
Linked (view graph)
#18210 test: type hints in Python tests#18921 build: add stack-clash and control-flow protection options to hardening flags#19183 [WIP DONOTMERGE] Replace boost with C++17 (std::shared_mutex)
github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-20 06:54 UTC