build: use `-isysroot` over `--sysroot` on macOS #21793

fanquake commented at 9:30 AM on April 28, 2021: member

Not only does this seem to be the more correct behaviour when targeting Darwin, but if you use -isysroot, Clangs Darwin driver will infer the deployment target from the SDK and use other SDK info when parsing arguments to the linker. In the case of -platform_version, which is added if the linker is new enough, the version tuple is constructed from the SDKInfo, and SDKInfo, as far as I can tell, only exists when -isysroot has been passed, see parseSDKSettings

As a result, the SDK version field in the LC_BUILD_VERSION command is filled out. i.e when building master:

      cmd LC_BUILD_VERSION
  cmdsize 32
 platform 1
    minos 10.14
      sdk n/a
   ntools 1
     tool 3
  version 650.9

vs this PR:

      cmd LC_BUILD_VERSION
  cmdsize 32
 platform 1
    minos 10.14
      sdk 11.3
   ntools 1
     tool 3
  version 650.9

This, from what I understand, will fix the issue we are having with Qt deciding wether or not to enable features like "Dark mode" on macOS, see #21771, however I have not tested that. Thus this is an alternative to #21782.

Our usage of --sysroot was added in #17118.

-isysroot <dir>         Set the system root directory (usually /)

fanquake added the label macOS on Apr 28, 2021

fanquake added the label Build system on Apr 28, 2021

fanquake cross-referenced this on Apr 28, 2021 from issue build: Fix static Darwin builds by hebasto

MarcoFalke added the label Needs gitian build on Apr 28, 2021

MarcoFalke added the label Needs Guix build on Apr 28, 2021

promag commented at 10:02 AM on April 28, 2021: member

Tested ACK 6777543c67dee43d1b92ba1ea8cbf9492e509620 on macOS 11.2.3 (Big Sur) arm-apple-darwin20.3.0.

fanquake force-pushed on Apr 28, 2021

hebasto commented at 3:21 PM on April 28, 2021: member

Approach ACK 1f3932e0dc085ef641e03bd08d4a7d7f22794c4d.

Tested for builds with depends:

macOS Big Sur 11.3 (20E232)

% otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION                                                           
Load command 9
      cmd LC_BUILD_VERSION
  cmdsize 32
 platform 1
    minos 10.14
      sdk 11.3
   ntools 1
     tool 3
  version 650.9

macOS Mojave 10.14.6 (18G9028)

$ otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION
Load command 9
       cmd LC_BUILD_VERSION
   cmdsize 32
  platform macos
       sdk 10.14
     minos 10.14
    ntools 1
      tool ld
   version 450.3

The dark mode works in both cases above.

nit: This change could be a scripted-diff.

Our docs should be updated as well.

DrahtBot commented at 4:06 PM on April 28, 2021: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#21778 (POC: LLVM 13 & LLD based macOS toolchain by fanquake)
#19817 (build: macOS toolchain bump by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot cross-referenced this on Apr 28, 2021 from issue build: LLVM 16 & LLD based macOS toolchain by fanquake

hebasto approved

hebasto commented at 5:10 PM on April 28, 2021: member

ACK 1f3932e0dc085ef641e03bd08d4a7d7f22794c4d

nit: Why not being consistent about absence/presence a space after -isysroot? Clang docs suggests the former.

Additionally to my previous tests, tested cross-compiling on Linux Mint 20.1 (x86_64):

this PR

$ depends/x86_64-apple-darwin18/native/bin/x86_64-apple-darwin18-otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION
Load command 9
      cmd LC_BUILD_VERSION
  cmdsize 32
 platform 1
    minos 10.14
      sdk 505.0
   ntools 1
     tool 3
  version 530.0

#19817 rebased on top of this PR

$ depends/x86_64-apple-darwin18/native/bin/x86_64-apple-darwin18-otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION
Load command 9
      cmd LC_BUILD_VERSION
  cmdsize 32
 platform 1
    minos 10.14
      sdk 10.15.6
   ntools 1
     tool 3
  version 609.0

The dark mode works in both cases above.

Clangs Darwin driver will infer the deployment target from the SDK and use other SDK info when parsing arguments to the linker.

It seems works in that way since llvm clang v8.0.0.

hebasto cross-referenced this on Apr 28, 2021 from issue build: macOS toolchain bump by fanquake

jarolrod commented at 8:53 PM on April 28, 2021: member

ACK 1f3932e

Tested this PR by performing the following for each supported macOS version (11, 10.15, 10.14) and checking SDK values and if dark mode works:

Native Depends Build
Native Depends Build with Toolchain Bump
Cross Compile from Linux
Cross Compile with Toolchain Bump

I have the branch with this PR + Toolchain Bump here: https://github.com/jarolrod/bitcoin/tree/21793_and_19817

build: use -isysroot over --sysroot on macOS cf971c9ea0

fanquake force-pushed on Apr 29, 2021

fanquake commented at 9:44 AM on April 29, 2021: member

Why not being consistent about absence/presence a space after -isysroot? Clang docs suggests the former. Our docs should be updated as well.

Addressed both.

hebasto approved

hebasto commented at 10:12 AM on April 29, 2021: member

re-ACK cf971c9ea0e8637d36a7ab8b7b8804d9b89f1d9a, only rebased and addressed comments since my previous review.

Guix build:

$ find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
fcb1b98f431d6467ea2fd77ba7d05b24012d84959845945fa2bed4aaeab92fad  guix-build-cf971c9ea0e8/output/dist-archive/bitcoin-cf971c9ea0e8.tar.gz
0982cc7b542cf0e6e03a48d30008945ffb2083625d72a907dd02a675c9cb1309  guix-build-cf971c9ea0e8/output/x86_64-apple-darwin18/bitcoin-cf971c9ea0e8-osx-unsigned.dmg
5d3087fb46f458eb34f28e07aa39db67c8b48420de319dc6aaea7e7bade78247  guix-build-cf971c9ea0e8/output/x86_64-apple-darwin18/bitcoin-cf971c9ea0e8-osx-unsigned.tar.gz
a4bdee56b3bf2bff2ae3e4061a173e5e9b2f4d264161dceb4ef6496c87ed4f89  guix-build-cf971c9ea0e8/output/x86_64-apple-darwin18/bitcoin-cf971c9ea0e8-osx64.tar.gz

Gitian build:

Generating report
cfc2a1ac9187bf7438ec1e45e057ffdf6811b9b0503107a2d3e8727f81083001  bitcoin-cf971c9ea0e8-osx-unsigned.dmg
b01df9e37555c31608f2e6df6bccdceb53064dd067733d31e1edcbdacdaa57dd  bitcoin-cf971c9ea0e8-osx-unsigned.tar.gz
f3b3a9cf79e8b6671a88e97de2b886fb853445919cd333ba9e4fd0ebf193ee99  bitcoin-cf971c9ea0e8-osx64.tar.gz
fcb1b98f431d6467ea2fd77ba7d05b24012d84959845945fa2bed4aaeab92fad  src/bitcoin-cf971c9ea0e8.tar.gz
8c436ea239b03f75aad8f5062dd751c72acbe03ec48bf4a5503d49ffd69b0edb  bitcoin-core-osx-22-res.yml
Done.

UPDATE: another gitian build (with guaranteed clear cache):

Generating report
cfc2a1ac9187bf7438ec1e45e057ffdf6811b9b0503107a2d3e8727f81083001  bitcoin-cf971c9ea0e8-osx-unsigned.dmg
b01df9e37555c31608f2e6df6bccdceb53064dd067733d31e1edcbdacdaa57dd  bitcoin-cf971c9ea0e8-osx-unsigned.tar.gz
f3b3a9cf79e8b6671a88e97de2b886fb853445919cd333ba9e4fd0ebf193ee99  bitcoin-cf971c9ea0e8-osx64.tar.gz
fcb1b98f431d6467ea2fd77ba7d05b24012d84959845945fa2bed4aaeab92fad  src/bitcoin-cf971c9ea0e8.tar.gz
2cd1011b1c37a6fd403884e66175376313285a3bd9119995fe04df975080b98a  bitcoin-core-osx-22-res.yml
Done.

fanquake commented at 10:49 AM on April 29, 2021: member

Guix builds:

bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
fcb1b98f431d6467ea2fd77ba7d05b24012d84959845945fa2bed4aaeab92fad  guix-build-cf971c9ea0e8/output/dist-archive/bitcoin-cf971c9ea0e8.tar.gz
0982cc7b542cf0e6e03a48d30008945ffb2083625d72a907dd02a675c9cb1309  guix-build-cf971c9ea0e8/output/x86_64-apple-darwin18/bitcoin-cf971c9ea0e8-osx-unsigned.dmg
5d3087fb46f458eb34f28e07aa39db67c8b48420de319dc6aaea7e7bade78247  guix-build-cf971c9ea0e8/output/x86_64-apple-darwin18/bitcoin-cf971c9ea0e8-osx-unsigned.tar.gz
a4bdee56b3bf2bff2ae3e4061a173e5e9b2f4d264161dceb4ef6496c87ed4f89  guix-build-cf971c9ea0e8/output/x86_64-apple-darwin18/bitcoin-cf971c9ea0e8-osx64.tar.gz

Gitian builds (Done twice, second with a cleared cache):

# macOS:
71fd96f0d87488a473c577bac1248085c48c4d925a6c3be09a5ea0adce6b54ae  bitcoin-cf971c9ea0e8-osx-unsigned.dmg
994a5175ce8dcba226976a708dd43cc459f891e11857c783f54b3e8c9f1570c4  bitcoin-cf971c9ea0e8-osx-unsigned.tar.gz
ae732d49acc56a44ddee1aa228d6c177df1473a824f65b780d15fd8613cc9801  bitcoin-cf971c9ea0e8-osx64.tar.gz
fcb1b98f431d6467ea2fd77ba7d05b24012d84959845945fa2bed4aaeab92fad  src/bitcoin-cf971c9ea0e8.tar.gz
13c6f297c299f45a9a7ea6f7f5284afdd155d58c6e9cbc12a47064e1e4d22b2a  bitcoin-core-osx-22-res.yml

Sjors commented at 10:50 AM on April 29, 2021: member

tACK cf971c9ea0e8637d36a7ab8b7b8804d9b89f1d9a

I built depends on macOS 11.3 Big Sur (XCode 12.5, Intel CPU) with the whole kitchen-sink make MULTIPROCESS=1 DEBUG=1 and then built and ran QT. It seems to work, including switching between dark and light mode.

otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION finds sdk 11.3

Also cross-compiled on Linux. otool finds sdk 505.0. There's some UI glitches when toggling between dark and light mode, but I guess that's what https://github.com/bitcoin-core/gui/pull/275 is trying to fix.

(I used the previous push 1f3932e0dc085ef641e03bd08d4a7d7f22794c4d on macOS, and cf971c9 on Linux)

fanquake merged this on May 1, 2021

fanquake closed this on May 1, 2021

fanquake cross-referenced this on May 1, 2021 from issue build: Static build behavior depends on the used compiler by hebasto

MarcoFalke removed the label Needs Guix build on May 1, 2021

MarcoFalke removed the label Needs gitian build on May 1, 2021

sidhujag referenced this in commit 788d32deca on May 1, 2021

hebasto cross-referenced this on May 1, 2021 from issue build, qt: Fix wrong cross-compiling detection on macOS by hebasto

hebasto cross-referenced this on May 1, 2021 from issue Support runtime appearance adjustment on macOS by hebasto

DrahtBot commented at 8:20 PM on May 4, 2021: contributor

Gitian builds

File	commit 13f24d135b280a9ab947f7948f6d86f00104cde1<br>(master)	commit 21f6843eef4693c02119e6ecc7aeddd0bc40b84f<br>(master and this pull)
*-aarch64-linux-gnu-debug.tar.gz	`2ae1d164869a2b1b...`	`d5af25134e83242e...`
*-aarch64-linux-gnu.tar.gz	`a88ff95f65f98d9a...`	`c468e25bf44e2c1b...`
*-arm-linux-gnueabihf-debug.tar.gz	`a5064041a36a4d6a...`	`c5fae3b0c49d4111...`
*-arm-linux-gnueabihf.tar.gz	`b7e2e194e4888f81...`	`5a6974747364fa0d...`
*-osx-unsigned.dmg	`554bd917c8c1f194...`	`14459cab24244cfa...`
*-osx64.tar.gz	`81c038d85238bcee...`	`576a5064d2d85522...`
*-powerpc64-linux-gnu-debug.tar.gz	`c1ddccbadb3c407a...`	`bfd60aebcfdaf9ef...`
*-powerpc64-linux-gnu.tar.gz	`ec3d9de57d82293f...`	`c240773578db4d43...`
*-powerpc64le-linux-gnu-debug.tar.gz	`a288fdab7434581f...`	`c6921805e9c8fd7b...`
*-powerpc64le-linux-gnu.tar.gz	`8e470ee15e20108f...`	`29a2c39643d1a9ce...`
*-riscv64-linux-gnu-debug.tar.gz	`564faccb866388e7...`	`5df0c320aca49add...`
*-riscv64-linux-gnu.tar.gz	`6a4efd483783486a...`	`dfd6ed19902b3dde...`
*-win64-debug.zip	`e1f8a015afa3d74d...`	`452248f82af41366...`
*-win64-setup-unsigned.exe	`225a8cf86123a054...`	`91d883f3299e6815...`
*-win64.zip	`9336468dd991303d...`	`0a3e1ad12f8a869d...`
*-x86_64-linux-gnu-debug.tar.gz	`eb4abea6ee4328ce...`	`6e43982133599f5e...`
*-x86_64-linux-gnu.tar.gz	`cffd18cbbdddb2c0...`	`b8452d92766a19d5...`
*.tar.gz	`8a08cdc873a22249...`	`72a93eae0a7145d1...`
bitcoin-core-linux-22-res.yml	`ac3298355af64e47...`	`dcfa8114be2483a8...`
bitcoin-core-osx-22-res.yml	`82cf158d3424910b...`	`a220ab06de00957b...`
bitcoin-core-win-22-res.yml	`b9daa890a73d2f54...`	`fec02b1a7bc34b49...`
linux-build.log	`5ce895ebc01e0a95...`	`44c785113babbfb2...`
osx-build.log	`232cb244fdb91c65...`	`38b78bcc1009ebfa...`
win-build.log	`0b3af0d4e3e8fb27...`	`f4fe251cc0bb1033...`
bitcoin-core-linux-22-res.yml.diff		`e2c40ed4a3d0abb0...`
bitcoin-core-osx-22-res.yml.diff		`7ccbb11414286382...`
bitcoin-core-win-22-res.yml.diff		`e78f0a8cfe632226...`
linux-build.log.diff		`9242c3e0fcfd8f00...`
osx-build.log.diff		`0ada5b6473f1be6a...`
win-build.log.diff		`2b2f17e09e5d7524...`

fanquake referenced this in commit 4bce17bf48 on May 5, 2021

fanquake deleted the branch on May 6, 2021

furszy cross-referenced this on Jun 30, 2021 from issue [Build] fix macOS depends deployment by furszy

random-zebra referenced this in commit 64da8d9167 on Jul 2, 2021

barton2526 referenced this in commit 43e7989b03 on Jul 12, 2021

barton2526 cross-referenced this on Jul 12, 2021 from issue build: use -isysroot over --sysroot on macOS by barton2526

kwvg referenced this in commit 24262c9895 on Jul 15, 2021

kwvg referenced this in commit e36f94e2ff on Jul 15, 2021

kwvg referenced this in commit a7062707c4 on Jul 16, 2021

kwvg referenced this in commit f5598c3040 on Jul 20, 2021

kwvg referenced this in commit 782853cac2 on Jul 20, 2021

kwvg referenced this in commit 9f710e4025 on Jul 20, 2021

kwvg referenced this in commit 5afb3659c1 on Jul 20, 2021

kwvg referenced this in commit 7b57818cac on Aug 1, 2021

kwvg referenced this in commit fa3d21d488 on Aug 24, 2021

kwvg referenced this in commit f94634e2d1 on Aug 24, 2021

kwvg referenced this in commit 421d728bb1 on Aug 24, 2021

kwvg referenced this in commit a1452d6a6d on Aug 24, 2021

kwvg referenced this in commit 2bc3d63dc8 on Aug 25, 2021

kwvg referenced this in commit afe13466b9 on Aug 25, 2021

kwvg referenced this in commit 5886d63c3e on Aug 26, 2021

kwvg referenced this in commit 5adacde953 on Aug 26, 2021

kwvg referenced this in commit c960494f14 on Aug 26, 2021

kwvg referenced this in commit 5ce8187981 on Aug 27, 2021

kwvg referenced this in commit 551d8f90fc on Aug 30, 2021

kwvg referenced this in commit d3a3fd793c on Sep 1, 2021

kwvg referenced this in commit 0a62d8fcb0 on Sep 1, 2021

kwvg referenced this in commit c084adcfe0 on Sep 2, 2021

kwvg referenced this in commit f6c9aaab02 on Sep 3, 2021

kwvg referenced this in commit d87e0831dc on Sep 3, 2021

kwvg referenced this in commit 8eca7b4fd6 on Sep 3, 2021

kwvg referenced this in commit 96f968c250 on Sep 4, 2021

DeckerSU referenced this in commit 813245cf7d on Nov 16, 2021

TheComputerGenie referenced this in commit c16477f24c on Jan 5, 2022

str4d cross-referenced this on Apr 15, 2022 from issue Backport upstream macOS build changes from 2021 by str4d

Jouzo cross-referenced this on Apr 27, 2022 from issue Add -isysroot to macOS native compile flags by Jouzo

gades referenced this in commit 09e5f43feb on May 4, 2022

gwillen referenced this in commit 608e6123b8 on Jun 1, 2022

bitcoin locked this on Aug 16, 2022