User-facing, this is a refactor. Internally, the scheduler does not have to call a mostly empty function every half a second.
wallet: Never schedule MaybeCompactWalletDB when -flushwallet is off #18923
pull MarcoFalke wants to merge 5 commits into bitcoin:master from MarcoFalke:2005-walletNoFlush changing 13 files +45 −27-
MarcoFalke commented at 1:12 PM on May 9, 2020: member
-
MarcoFalke commented at 1:13 PM on May 9, 2020: member
Can be tested by applying the following diff and observing a failing test:
diff --git a/test/functional/wallet_dump.py b/test/functional/wallet_dump.py index 6bfb468823..f30589761a 100755 --- a/test/functional/wallet_dump.py +++ b/test/functional/wallet_dump.py @@ -190,7 +190,7 @@ class WalletDumpTest(BitcoinTestFramework): assert_raises_rpc_error(-8, "already exists", lambda: self.nodes[0].dumpwallet(wallet_enc_dump)) # Restart node with new wallet, and test importwallet - self.restart_node(0, ['-wallet=w2']) + self.restart_node(0, ['-wallet=w2', '-flushwallet=0']) # Make sure the address is not IsMine before import result = self.nodes[0].getaddressinfo(multisig_addr) - MarcoFalke added the label Refactoring on May 9, 2020
- MarcoFalke added the label Wallet on May 9, 2020
- MarcoFalke force-pushed on May 9, 2020
- MarcoFalke force-pushed on May 9, 2020
- MarcoFalke force-pushed on May 9, 2020
- MarcoFalke force-pushed on May 9, 2020
-
DrahtBot commented at 3:16 AM on May 10, 2020: contributor
<!--e57a25ab6845829454e8d69fc972939a-->
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
<!--174a7506f384e20aa4161008e828411d-->
Conflicts
Reviewers, this pull request conflicts with the following ones:
- #19461 (multiprocess: Add bitcoin-gui -ipcconnect option by ryanofsky)
- #19460 (multiprocess: Add bitcoin-wallet -ipcconnect option by ryanofsky)
- #19335 (wallet: Cleanup and separate BerkeleyDatabase and BerkeleyBatch by achow101)
- #19101 (refactor: remove ::vpwallets and related global variables by ryanofsky)
- #19099 (refactor: Move wallet methods out of chain.h and node.h by ryanofsky)
- #19098 (test: Remove duplicate NodeContext hacks by ryanofsky)
- #18904 (Don't call lsn_reset in periodic flush by bvbfan)
- #18354 (Use shared pointers only in validation interface by bvbfan)
- #15937 (Add loadwallet and createwallet load_on_startup options by ryanofsky)
- #10102 ([experimental] Multiprocess bitcoin by ryanofsky)
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
- DrahtBot cross-referenced this on May 10, 2020 from issue wallet: Keep inactive seeds after sethdseed and derive keys from them as needed by achow101
- MarcoFalke force-pushed on May 11, 2020
- DrahtBot added the label Needs rebase on May 22, 2020
- MarcoFalke force-pushed on May 22, 2020
- MarcoFalke force-pushed on May 22, 2020
- DrahtBot removed the label Needs rebase on May 22, 2020
- DrahtBot cross-referenced this on May 28, 2020 from issue Remove g_rpc_chain global by ryanofsky
- MarcoFalke cross-referenced this on May 28, 2020 from issue test: Remove duplicate NodeContext hacks by ryanofsky
- DrahtBot cross-referenced this on May 29, 2020 from issue refactor: remove ::vpwallets and related global variables by ryanofsky
- DrahtBot cross-referenced this on May 29, 2020 from issue refactor: Move wallet methods out of chain.h and node.h by ryanofsky
-
in src/interfaces/wallet.cpp:494 in faccdaba58 outdated
490 | @@ -491,7 +491,7 @@ class WalletClientImpl : public ChainClient 491 | } 492 | bool verify() override { return VerifyWallets(m_chain, m_wallet_filenames); } 493 | bool load() override { return LoadWallets(m_chain, m_wallet_filenames); } 494 | - void start(CScheduler& scheduler) override { return StartWallets(scheduler); } 495 | + void start(CScheduler& scheduler, const ArgsManager& argsman) override { return StartWallets(scheduler, argsman); }
ryanofsky commented at 11:49 AM on May 29, 2020:In commit "wallet: Pass unused argsman to StartWallets()" (faccdaba583288273399481b1f7892a910080fda)
Not a big deal and I can fix it later, but this is a bit broken with #10102 because it is trying to pass a whole argsman object from the node to wallet process just to pass a single bool. (See "Interface method parameter and return types should either be serializable or be other interface classes" examples in https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#internal-interface-guidelines)
Recommended fix for this would be to pass
-flushwalletbool setting to StartWallets the same way the-walletlist setting gets passed to LoadWallets above: first read inWalletInit::Constructand passed as an argument toMakeWalletClient, stored in the WalletClientImpl instance, and used as needed
MarcoFalke commented at 12:49 PM on May 29, 2020:Thanks, tried to implement your suggestion in branch 9ddddd2d927b3a752e4698c4771fc68cdc2d9698
ryanofsky commented at 1:33 PM on May 29, 2020:Thanks, tried to implement your suggestion in branch 9ddddd2
Branch looks good. Current version is fine too in case you prefer that or the suggestion doesn't work.
I was thinking maybe the best another approach after #19096 would be to add an ArgsManager pointer to the WalletContext struct, and initializing it from a new
MakeWalletClientArgsManager&argument so the wallet filenames and periodic flush arguments and temporary storage could be dropped
MarcoFalke commented at 5:21 PM on June 5, 2020:add an ArgsManager pointer to the WalletContext struct
thx, done
ryanofsky approvedryanofsky commented at 11:57 AM on May 29, 2020: contributorCode review ACK fa6195467a3533fcdcc001d559555a86717cd5d3
MarcoFalke force-pushed on May 29, 2020ryanofsky approvedryanofsky commented at 1:40 PM on May 29, 2020: contributorCode review ACK 9ddddd2d927b3a752e4698c4771fc68cdc2d9698. SInce last review was changed to pass periodic flush option from WalletInit::Construct
DrahtBot cross-referenced this on May 29, 2020 from issue Use shared pointers only in validation interface by bvbfanDrahtBot cross-referenced this on Jun 2, 2020 from issue Multiprocess bitcoin by ryanofskyDrahtBot added the label Needs rebase on Jun 5, 2020MarcoFalke force-pushed on Jun 5, 2020DrahtBot removed the label Needs rebase on Jun 5, 2020MarcoFalke force-pushed on Jun 5, 2020in src/wallet/test/init_test_fixture.h:21 in fa00c37cba outdated
17 | @@ -18,7 +18,6 @@ struct InitWalletDirTestingSetup: public BasicTestingSetup { 18 | fs::path m_datadir; 19 | fs::path m_cwd; 20 | std::map<std::string, fs::path> m_walletdir_path_cases; 21 | - NodeContext m_node;
ryanofsky commented at 6:27 PM on June 5, 2020:In commit "test: Avoid overwriting the NodeContext member of the testing setup" (fa00c37cba98794c1c12c4b77e885d831d9bf183)
Wow, good catches. I wonder if there was a compiler warning we could enable that would have caught these.
in src/util/check.h:47 in fa38755d2f outdated
41 | @@ -42,4 +42,12 @@ class NonFatalCheckError : public std::runtime_error 42 | } \ 43 | } while (false) 44 | 45 | +/** Assert that the passed pointer is not nullptr and return a reference to the pointed object */ 46 | +template <typename PtrT> 47 | +typename std::remove_reference<decltype(*PtrT{})>::type& Ensure(const PtrT& ptr)
ryanofsky commented at 6:46 PM on June 5, 2020:In commit "wallet: Pass unused args to WalletClientImpl" (fa38755d2fc444589ae506e4edd8597ab25e55e9)
Slight preference for calling this
AssertNotNullinstead ofEnsurejust because it could wind up being used widely, and it'd be nice for it to be obvious what it does.Also for code review purposes, C++ programmers are already pretty trained to know that
*ptris dangerous and can kill the program, whileEnsure(ptr)might look more innocuous if it's part of a larger expression. And since the name doesn't contain "Assert" or "Abort", it might actually be clearer if this returned a pointer so the caller is forced to write*. Could also go more generic and providetemplate<typename T> T&& Assert(T&& value) { assert(value); return std::forward<T>(value); }ryanofsky approvedryanofsky commented at 6:51 PM on June 5, 2020: contributorCode review ACK 00000d48588689f7576a8f4cbd9eb8edee3b53b0. No important suggestions. First two commits are the same since last review, last commit is basically the same, middle commits switch back to passing ArgsManager instead of the bool, but going through WalletContext struct from #19096
fanquake commented at 6:06 AM on June 6, 2020: memberConcept ACK
MarcoFalke force-pushed on Jun 8, 2020MarcoFalke commented at 12:42 PM on June 8, 2020: membercommit id 00000d4 was lost in a rebase
DrahtBot cross-referenced this on Jun 9, 2020 from issue Overhaul transaction request logic by siparyanofsky approvedryanofsky commented at 4:45 PM on June 9, 2020: contributorCode review ACK faf18d6c86cbb3e90070a971ee341cc76fb1a8f5. Changes since previous review were replacing Ensure with Assert, and adding new checks to ensure the NDEBUG is disabled and that assert isn't called from RPC code. First two commits are new, previous shadow fix commit became #19188, and the other commits are basically unchanged except being divided differently and replacing assert with ensure. Long list. Nice cleanups, though
in src/util/check.h:51 in fa54cf1073 outdated
45 | @@ -46,4 +46,12 @@ class NonFatalCheckError : public std::runtime_error 46 | #error "Cannot compile without assertions!" 47 | #endif 48 | 49 | +/** Identity function. Abort if the value compares equal to zero */ 50 | +template <typename T> 51 | +T&& Assert(T&& value)
ryanofsky commented at 5:15 PM on June 9, 2020:In commit "util: Add Assert identity function" (fa54cf107319609ff44de960b987bd09b46945d0)
Thinking about this, I think it would be better to change signature to
T Assert(T&& value)to avoid a possibility of returning a reference to a temporary. Contrived example but I thinkpwould be an invalid reference here returningT&&instead ofT:std::unique_ptr<int>&& p = Assert(MakeUnique<int>(5));No other change should be needed to fix this other than removing && from return type.
MarcoFalke force-pushed on Jun 9, 2020MarcoFalke commented at 6:50 PM on June 14, 2020: memberI've split out the first commits into #19277
DrahtBot cross-referenced this on Jun 15, 2020 from issue util: Add Assert identity function by MarcoFalkeDrahtBot cross-referenced this on Jun 20, 2020 from issue build: Do not include server symbols in wallet by MarcoFalkeryanofsky approvedDrahtBot cross-referenced this on Jun 25, 2020 from issue Remove the automatic creation and loading of the default wallet by achow101DrahtBot cross-referenced this on Jun 25, 2020 from issue refactor: Drop g_orphan_list global by hebastoDrahtBot cross-referenced this on Jun 26, 2020 from issue Add loadwallet and createwallet load_on_startup options by ryanofskyDrahtBot added the label Needs rebase on Jul 1, 2020MarcoFalke force-pushed on Jul 4, 2020DrahtBot removed the label Needs rebase on Jul 4, 2020MarcoFalke commented at 6:01 PM on July 4, 2020: memberRebased
DrahtBot cross-referenced this on Jul 7, 2020 from issue multiprocess: Add bitcoin-gui -ipcconnect option by ryanofskyDrahtBot cross-referenced this on Jul 7, 2020 from issue multiprocess: Add bitcoin-wallet -ipcconnect option by ryanofskytest: Add smoke test to check that wallets are flushed by default fa28a61897fa6c186436gui tests: Limit life-time of dummy testing setup
Its only purpose is to create a directory. So instead of keeping it alive, use it only to get the path of the directory and then create it explicitly.
faf8401c19wallet: Pass unused args to StartWallets
This refactor does not change behavior
wallet: Never schedule MaybeCompactWalletDB when -flushwallet is off fa7b164d62refactor: Use C++11 range-based for loop fa73493930MarcoFalke cross-referenced this on Jul 9, 2020 from issue Refactor: clean up PeriodicFlush() by jnewberyMarcoFalke force-pushed on Jul 9, 2020in src/wallet/bdb.cpp:627 in fa73493930
622 | @@ -623,8 +623,8 @@ bool BerkeleyDatabase::PeriodicFlush() 623 | if (!lockDb) return false; 624 | 625 | // Don't flush if any databases are in use 626 | - for (auto it = env->mapFileUseCount.begin() ; it != env->mapFileUseCount.end(); it++) { 627 | - if ((*it).second > 0) return false; 628 | + for (const auto& use_count : env->mapFileUseCount) { 629 | + if (use_count.second > 0) return false;
promag commented at 11:14 AM on July 9, 2020:fa73493930e35850e877725167dc9d42e47015c8
nit, if you feel like cleaning it a bit more maybe do this below:
if (env->mapFileUseCount.erase(strFile) == 0) return falseAnd drop
.erase(it).
MarcoFalke commented at 11:21 AM on July 9, 2020:Thanks, but I'll leave that for a follow-up
in test/functional/wallet_dump.py:206 in fa73493930
201 | @@ -202,5 +202,10 @@ def run_test(self): 202 | result = self.nodes[0].getaddressinfo(multisig_addr) 203 | assert result['ismine'] 204 | 205 | + self.log.info('Check that wallet is flushed') 206 | + with self.nodes[0].assert_debug_log(['Flushing wallet.dat'], timeout=20):
jnewbery commented at 11:45 AM on July 9, 2020:I know you'll disagree, but I think it's always better to test the result of an action rather than whether it was logged. In this case, I think you could test the file modification time before and after.
MarcoFalke commented at 12:16 PM on July 9, 2020:I think both can be done at the same time, but asserting the modification time increases requires a
time.sleep(1), sinceos.path.getmtimereturns seconds. I don't like sleeps longer than 200 ms in tests, so I'll leave that for a follow-up ;)
jnewbery commented at 12:45 PM on July 9, 2020:No need for a follow-up. Fast tests are nice too!
jnewbery commented at 12:02 PM on July 9, 2020: memberutACK fa73493930e35850e877725167dc9d42e47015c8
One comment on the new test. Feel free to ignore.
DrahtBot cross-referenced this on Jul 9, 2020 from issue Don't call lsn_reset in periodic flush by bvbfanDrahtBot cross-referenced this on Jul 10, 2020 from issue wallet: Cleanup and separate BerkeleyDatabase and BerkeleyBatch by achow101meshcollider commented at 10:51 AM on July 11, 2020: contributorutACK fa73493930e35850e877725167dc9d42e47015c8
ryanofsky approvedryanofsky commented at 11:08 AM on July 11, 2020: contributorCode review ACK fa73493930e35850e877725167dc9d42e47015c8. Just rebased since last review
meshcollider merged this on Jul 11, 2020meshcollider closed this on Jul 11, 2020MarcoFalke deleted the branch on Jul 11, 2020sidhujag referenced this in commit c96248b149 on Jul 11, 2020hebasto cross-referenced this on Jul 24, 2020 from issue util: Make default arg values more specific by hebastoFabcien referenced this in commit 912bd27d4b on Dec 9, 2020Brightside56 cross-referenced this on Apr 22, 2021 from issue Add -flushwalletinterval command line arg by Brightside56bitcoin locked this on Feb 15, 2022Labels
github-metadata-mirror
This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-20 06:54 UTC