node: smooth oversized `dbcache` warnings #35200

l0rinc commented at 12:56 PM on May 3, 2026: contributor

Problem: The oversized -dbcache warning threshold has a sharp formula cliff when detected RAM crosses the cutoff used by the warning logic.

This was reported during review of #34641, where an earlier version could jump from the auto default at 4095 MiB RAM to 75% of RAM at 4096 MiB. That made 1 MiB of extra detected RAM raise the warning threshold from about 511 MiB to 3072 MiB.

The surviving warning-only code has the same shape at a different boundary. Below 2 GiB RAM the cap is DEFAULT_DB_CACHE (450 MiB), but at 2 GiB it switches to 75% of total RAM, so a tiny increase in detected RAM can suddenly raise the warning threshold from 450 MiB to about 1536 MiB.

Fix: Base the warning on a reserved non-dbcache memory budget instead:

\text{warn if } \mathit{dbcache} > \max\left(\mathit{DEFAULT\_DB\_CACHE}, 0.75 \cdot \max(\text{total RAM} - \mathit{DBCACHE\_WARNING\_RESERVED\_RAM}, 0)\right)

DBCACHE_WARNING_RESERVED_RAM is 2 GiB, so the fixed DEFAULT_DB_CACHE cap remains the floor below that reserve and the warning threshold grows monotonically above it.

This keeps the warning conservative around low-memory boundaries and avoids treating a boundary-crossing 1 MiB RAM difference as a reason to allow a much larger explicit -dbcache.

Quick reference:

System RAM	Previous warning cap	New warning cap
1 GiB	450 MiB	450 MiB
2 GiB	1536 MiB	450 MiB
3 GiB	2304 MiB	768 MiB
4 GiB	3072 MiB	1536 MiB
8 GiB	6144 MiB	4608 MiB
16 GiB	12288 MiB	10752 MiB
32 GiB	24576 MiB	23040 MiB

On 32-bit builds, effective -dbcache values are still capped to 1024 MiB before the warning check, so thresholds above that cap are not reachable there.

DrahtBot commented at 12:56 PM on May 3, 2026: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35200.

Reviews

See the guideline and AI policy for information on the review process.

Type	Reviewers
ACK	optout21, sedited, w0xlt

If your review is incorrectly listed, please copy-paste <code></code> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#35616 (refactor: Use u64 over size_t for all cache sizes to fix a 32-bit overflow by maflcko)
#35205 (kernel,node: clean up dbcache helpers and add kernel API by l0rinc)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot added the label CI failed on May 3, 2026

DrahtBot commented at 1:45 PM on May 3, 2026: contributor

🚧 At least one of the CI tasks failed. <sub>Task i686, no IPC: https://github.com/bitcoin/bitcoin/actions/runs/25279789848/job/74115183233</sub> <sub>LLM reason (✨ experimental): CI failed because the CTest “Bitcoin Core Test Suite” had failing tests, specifically sock_tests (1 test failed).</sub>

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

</details>

l0rinc closed this on May 5, 2026

l0rinc reopened this on May 5, 2026

DrahtBot removed the label CI failed on May 5, 2026

node: smooth oversized `-dbcache` warnings

The oversized dbcache warning still switches from a fixed `450 MiB` cap below `2 GiB` RAM to `75%` of total RAM at `2 GiB`.
That creates a discontinuity where adding a small amount of RAM can raise the warning threshold from `450 MiB` to `1536 MiB`.

Use a single monotonic rule based on a `2 GiB` reserved-memory budget instead.
Above `DBCACHE_WARNING_RESERVED_RAM`, the threshold grows linearly at `0.75 MiB` per MiB of additional RAM.
This keeps a `25%` margin above the reserve for OS and mempool variance while avoiding the cliff.

Co-authored-by: Bortlesboat <Bortlesboat@users.noreply.github.com>

b4f7fc3d92

l0rinc force-pushed on May 5, 2026

sedited commented at 2:34 PM on June 7, 2026: contributor

Is this really problem given that RAM size more or less follows a discrete distribution across deployments? I'm guessing the next common RAM configuration is probably 3GiB, which seems a more justifiable jump.

optout21 commented at 8:10 AM on June 18, 2026: contributor

ACK b4f7fc3d92d21e3a58ccad67de2af8c56ca772f3

This change streamlines the formula used for warning about too large DB cache value set. The old formula with a discontinuity is changed to a linear formula, which is more intuitive, and has less risk of unintended consequences.

l0rinc commented at 12:11 PM on June 18, 2026: contributor

Is this really problem given that RAM size more or less follows a discrete distribution across deployments

On k8s clusters, people can set any weird memory value, so it kind of makes sense to make it more predictable. Also, we've encountered a few surprising cases with swapping when dbcache was too high (e.g. https://x.com/caesrcd/status/2062766249038250356). As it turns out, after a while, IBD just gets slower and slower with bigger dbcache values - many people reported that the ideal was 2-3 GB, after which validation just got slower. Also, running reindex-chainstate with 2 GB of total memory still works with roughly -dbcache=100 (the previous warning was at 1536 MiB; the new 450 MiB is closer).

in src/node/caches.h:38 in b4f7fc3d92

  34 | @@ -31,7 +35,8 @@ struct CacheSizes {
  35 |  CacheSizes CalculateCacheSizes(const ArgsManager& args, size_t n_indexes = 0);
  36 |  constexpr bool ShouldWarnOversizedDbCache(uint64_t dbcache, uint64_t total_ram) noexcept
  37 |  {
  38 | -    const uint64_t cap{(total_ram < 2_GiB) ? DEFAULT_DB_CACHE : (total_ram / 100) * 75};
  39 | +    const uint64_t headroom{total_ram > DBCACHE_WARNING_RESERVED_RAM ? total_ram - DBCACHE_WARNING_RESERVED_RAM : 0};

sedited commented at 1:25 PM on June 18, 2026:

Nit: headroom would imply this is the space left free, but it's the opposite, right? If so, how about calling this available or something along the lines?

w0xlt commented at 11:53 PM on July 8, 2026:

Yes, available_ram, available_for_dbcache, or ram_above_reserve would be clearer.

sedited approved

sedited commented at 1:42 PM on June 18, 2026: contributor

ACK b4f7fc3d92d21e3a58ccad67de2af8c56ca772f3

w0xlt commented at 12:00 AM on July 9, 2026: contributor

ACK b4f7fc3d92d21e3a58ccad67de2af8c56ca772f3

The old rule:

if RAM < 2 GiB: warn above 450 MiB if RAM >= 2 GiB: warn above 75% of RAM

That means crossing from just under 2 GiB to exactly 2 GiB raises the warning threshold from 450 MiB to 1536 MiB. One extra MiB of detected RAM should not make a 1.5 GiB dbcache suddenly look safe.

The new model is more coherent:

warn if dbcache > max(DEFAULT_DB_CACHE, 75% * (total_ram - 2 GiB))

nit: headroom can be renamed.

l0rinc commented at 12:23 AM on July 9, 2026: contributor

rfm?