This PR fixes the interface_ipc*.py tests, which have been failing on NetBSD since #35037.
As file descriptor budget regressions of this kind recur (see #31537 and #36078), the macOS CI job now models NetBSD's limit so they are caught before merging rather than by nightly NetBSD CI afterwards.
See the commit messages for details.
Easy to review using the following:
```
git diff --color-moved=dimmed-zebra --color-moved-ws=allow-indentation-change
```
Since `-ipcbind` reserves one listening socket plus
`ipc::DEFAULT_MAX_CONNECTIONS` (16) client slots in `min_required_fds`,
the framework's `-maxconnections=76` no longer fits the 256 file
descriptor budget on NetBSD, and every `interface_ipc*.py` node starts
with "Warning: Reducing -maxconnections ...", which the framework treats
as a failure.
Lower `-maxconnections` by that reservation on IPC-enabled nodes only,
so the rest of the suite keeps its P2P connection slots.
On select()-based platforms such as NetBSD, `bitcoind` can use at most
`FD_SETSIZE` (256) file descriptors, so the functional test framework
pins `-maxconnections` to that budget. Any change that grows
`min_required_fds` then makes every node print "Warning: Reducing
-maxconnections ...", which the framework treats as a failure. Such
regressions are currently caught only by nightly NetBSD CI (see #31537,
Add a `CI_LIMIT_FD_COUNT` knob that runs functional test node processes
through a wrapper capping `RLIMIT_NOFILE`, and enable it for the macOS
native job with NetBSD's value of 256.
<!--e57a25ab6845829454e8d69fc972939a-->
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
<!--006a51241073e994b41acfe9ec718e94-->
<!--021abf342d371248e50ceaed478a90ca-->
See the guideline and AI policy for information on the review process.
| Type | Reviewers |
|---|---|
| Concept ACK | Sjors |
If your review is incorrectly listed, please copy-paste <code><!--meta-tag:bot-skip--></code> into the comment that the bot should ignore.
<!--5faf32d7da4f0f540f40219e4f7537a3-->
NetBSD test logs are available here: https://github.com/hebasto/bitcoin-core-nightly/actions/runs/39563835829.
Concept ACK
But wrap-fd-limit.sh in 311bc7c3bb274c7065f871198fa0910c205b8a6a loses our coverage for "infinite" file descriptors introduced by #34937.
We could run feature_init.py both capped and uncapped:
diff --git a/ci/test/03_test_script.sh b/ci/test/03_test_script.sh
index d5981147f4..1468638690 100755
--- a/ci/test/03_test_script.sh
+++ b/ci/test/03_test_script.sh
@@ -161,5 +161,6 @@ if [ -n "${CI_LIMIT_FD_COUNT}" ]; then
# Only the node processes are capped (via BITCOIN_CMD), not the Python
# test runner, which needs more descriptors when running jobs in parallel.
- export BITCOIN_CMD="${BASE_ROOT_DIR}/ci/test/wrap-fd-limit.sh ${BITCOIN_CMD:-bitcoin}"
+ BITCOIN_CMD_UNCAPPED="${BITCOIN_CMD:-bitcoin}"
+ export BITCOIN_CMD="${BASE_ROOT_DIR}/ci/test/wrap-fd-limit.sh ${BITCOIN_CMD_UNCAPPED}"
fi
@@ -208,4 +209,20 @@ if [ "$RUN_FUNCTIONAL_TESTS" = "true" ]; then
--quiet \
--failfast
+
+ if [ -n "${CI_LIMIT_FD_COUNT}" ]; then
+ # The capped suite also runs feature_init.py, but the wrapper overrides the
+ # RLIM_INFINITY and above-INT_MAX limits it tests. Repeat it without the cap.
+ BITCOIN_CMD="${BITCOIN_CMD_UNCAPPED}" \
+ LD_LIBRARY_PATH="${DEPENDS_DIR}/${HOST}/lib" \
+ "${BASE_BUILD_DIR}/test/functional/test_runner.py" \
+ "${MAKEJOBS}" \
+ --tmpdirprefix "${BASE_SCRATCH_DIR}/test_runner_uncapped/" \
+ --ansi \
+ --combinedlogslen=99999999 \
+ --timeout-factor="${TEST_RUNNER_TIMEOUT_FACTOR}" \
+ --quiet \
+ --failfast \
+ feature_init.py
+ fi
fi
Alternatively we could drop that test.
12 | +# platform, so an fd-budget regression (e.g. a new "Reducing -maxconnections ..." 13 | +# warning on stderr, which the functional test framework treats as a failure) is 14 | +# caught by this job instead of only on NetBSD. 15 | + 16 | +export LC_ALL=C.UTF-8 17 | +set -o errexit
missing pipefail and nounset? I guess the script may fail anyway when nounset is hit, but can't hurt to be explicit here