slow GUI with large wallets #15015

issue HashUnlimited opened this issue on December 21, 2018

HashUnlimited commented at 12:51 PM on December 21, 2018: contributor

Loading a large wallet (around 23 mio transactions here on test) is resulting in an unusable GUI. Spinning wheel of death on every mouse click for 5 seconds at least.

branch: master OS: macOS 10.14.2 CPU: Intel i9 (8th Gen) RAM: 32 GB DDR4

Note: this behaviour is not limited to the Overview page or CoinControl but in general as long as the wallet is loaded. A headless setup performs nicely though.
fanquake added the label GUI on Dec 21, 2018
benthecarman cross-referenced this on Jan 12, 2019 from issue large wallet: Bitcoind freezes by RomanBelkov
RealBokito cross-referenced this on Jan 15, 2019 from issue RPC calls resulting in unresponsive node by RealBokito
cryptozeny commented at 12:21 PM on February 7, 2019: none

same here. on custom testnet, empty wallet works like charm, but large wallet tooooooooo slow... one click for 5 seconds even no RPC calls!

debug is OK, but only GUI freeze

OS: ubuntu 16.04.5-64bit CPU: i5 skylake RAM: 24GB

v0.16.3
promag cross-referenced this on Feb 17, 2020 from issue gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged by promag
promag commented at 10:09 AM on February 17, 2020: member

@HashUnlimited are you able reproduce with #18160? ty
decryp2kanon cross-referenced this on Mar 29, 2020 from issue large wallet freezes GUI wallet by decryp2kanon
jonatack commented at 9:53 AM on March 30, 2020: contributor
I was not able to reproduce this issue, though I initially had a different one (see the bottom).

Yesterday I tested the GUI on current master 5f9cd62f33f, and with #18160 rebased onto it, both on Debian and MacOS.
- Debian 4.19.37amd64 (2019-08-08) x86_64 - Intel Core i7-6500U 2.50GHz × 4 - 32GB
- MacOS 10.14.6 - MacBook Pro Retina 2012 - Intel Core i7 2.7GHz - 16GB 1600 DDR3
I did not see a real difference in performance between the four configurations. EDIT: While scanning, or in sometimes in general, performance does seem better with #18160. The results are variable and it's hard for me to have confidence in them. When all is snappy and well, I am not seeing a difference. When the GUI is laggy/scanning, the PR does seem better, but it's hard to say. I built and compared both several times, back and forth.

The test was to load several wallets, 2-3 small wallets and then one large 177MB wallet, and run various console commands like getbalances, getwalletinfo, listwallets, etc. The large wallet is a regtest one generated with @achow101's make-big-wallet.py script.

On both Debian and MacOS it takes the GUI 5-6 minutes to load the big wallet.

Once loaded, the GUI ran fine in my tests, with an additional reaction latency of ~400-500 ms with console commands like getbalances or getwalletinfo when the commands were run on big wallet. The other commands, and clicking on and opening transactions, ran basically as fast as they do with tiny wallets.

With one build on Debian, I did see the GUI halt, and then crash with segmentation faults, when loading any wallet at all. However, rebuilding after make clean solved that, so I suspect it was the build.

If anyone has suggestions on how to best instrument this or for further tests to run with the big wallet, let me know.
achow101 commented at 4:15 PM on March 30, 2020: member

@jonatack The big wallet script does ~100,000 txs. It's possible that this is not big enough as it seems that OP has a wallet with several million txs. You should try increasing the iteration count to make a wallet that has a few million txs.
promag commented at 4:19 PM on March 30, 2020: member

According to OP

Note: this behaviour is not limited to the Overview page or CoinControl but in general as long as the wallet is loaded. A headless setup performs nicely though. @achow101 see my last post at #18160, I think it's safe to say that #18160 will improve in this case.
jonatack commented at 4:30 PM on March 30, 2020: contributor

@achow101 @promag thanks. Both trying with a larger wallet and instrumenting as per #18160 (comment) to reproduce @promag's result (and/or with a flamegraph) seem worthwhile.
HashUnlimited commented at 10:56 AM on April 1, 2020: contributor

Sorry for the late reply. Just tested with the same wallet - performance is WAY better now.
HashUnlimited closed this on Apr 1, 2020
promag commented at 10:58 AM on April 1, 2020: member

performance is WAY better now. @HashUnlimited I guess you tested with #18160, and if so happy to know that.
HashUnlimited commented at 11:01 AM on April 1, 2020: contributor

Yes, tested twice. Once with master and once with 0.18 and implemented #18160 just to make sure this is the performance booster.
promag commented at 11:17 AM on April 1, 2020: member

Great, thanks for reporting the result.
promag cross-referenced this on Apr 10, 2020 from issue gui: Avoid wallet tryGetBalances calls in WalletModel::pollBalanceChanged by ryanofsky
promag cross-referenced this on Apr 10, 2020 from issue 0.19: gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged by promag
decryp2kanon cross-referenced this on Jun 9, 2020 from issue large size wallet.dat too slow by decryp2kanon
bitcoin locked this on Feb 15, 2022

Contributors

Labels

Linked (view graph)

#15148 large wallet: Bitcoind freezes #18160 gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged #18542 0.19: gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged #18587 gui: Avoid wallet tryGetBalances calls in WalletModel::pollBalanceChanged