ci: Built depends cache misses OS toolchain updates #35648

On the master branch @ 4a007126fbe3254d4025da5c623f378db44a1125, the built depends cache is keyed by the hash of a configuration script and the depends subdirectory:https://github.com/bitcoin/bitcoin/blob/0a1bbec688b8c075ba45b6028b6f4fe3ef636be9/.github/actions/configure-environment/action.yml#L20

Unfortunately, the current approach fails to invalidate the cache when toolchain packages shipped by the running OS are updated. Moreover, in such a scenario, the depends build subsystem rebuilds packages from scratch but they won't be cached by the CI framework.

As a solution, the final_build_id variable can be used:

gmake -C depends --no-print-directory print-final_build_id

This approach has another advantage: it does not require git and maybe be exploited in environments where the source code is fetched as a tarball rather than as a repository.

However, this approach is currently broken on systems (macOS, *BSD, etc). For further details, please see #35618.

My understanding here is that, if we merge #35618, then we still want to change our Ci caching, right?

It seems to me like it would make most sense to have a DEPENDS_SOURCES_HASH (the current caclulation) and a new DEPENDS_BUILD_HASH which is computed form gmake -C depends --no-print-directory print-final_build_id. Then we should avoid invalidating sources unnecessarily, while still getting updates from toolchains correctly.

Is this roughly what you have in mind?

My understanding here is that, if we merge #35618, then we still want to change our Ci caching, right?

It seems to me like it would make most sense to have a DEPENDS_SOURCES_HASH (the current caclulation) and a new DEPENDS_BUILD_HASH which is computed form gmake -C depends --no-print-directory print-final_build_id. Then we should avoid invalidating sources unnecessarily, while still getting updates from toolchains correctly.

Is this roughly what you have in mind?

You read my mind.

Here's the code I wrote for https://github.com/hebasto/bitcoin-core-nightly/pull/290:

    - name: Determine the depends `final_build_id`
      id: build_id
      shell: bash
      run: |
        build_id="$(ssh ${{ inputs.vm }} "gmake --no-print-directory -C ${{ inputs.source-dir }}/depends ${{ inputs.host }} ${{ inputs.options }} print-final_build_id" | sed -n 's/^final_build_id=//p')"
        echo "build_id=${build_id}" >> "$GITHUB_OUTPUT"