Can't upgrade wallet with 0.17.0's new -upgradewallet feature #14422

issue redblade7 opened this issue on October 7, 2018

redblade7 commented at 5:25 AM on October 7, 2018: none
When using the new Bitcoin Core v0.17.0, and trying to pass -upgradewallet to upgrade to HD, results in an error.
- Bitcoin-Qt shows a popup box entitled "Runaway exception" with "A fatal error occurred. Bitcoin can no longer continue safely and will quit." Terminal output shows:
EXCEPTION: St13runtime_error
DeriveNewSeed: AddKeyPubKey failed
bitcoin in Runaway exception

bitcoin-qt: /usr/include/boost/thread/pthread/condition_variable_fwd.hpp:116: boost::condition_variable::~condition_variable(): Assertion `!ret' failed. Aborted

When attempting to do so via bitcoind -upgradewallet, the following is displayed before a shutdown:

2018-10-07T05:24:05Z [default wallet] nFileVersion = 170000 2018-10-07T05:24:05Z [default wallet] Keys: 0 plaintext, 1158 encrypted, 1158 w/ metadata, 1158 total. Unknown wallet records: 2 2018-10-07T05:24:06Z [default wallet] Performing wallet upgrade to 169900 2018-10-07T05:24:06Z [default wallet] Upgrading wallet to HD 2018-10-07T05:24:06Z [default wallet] Releasing wallet 2018-10-07T05:24:06Z

EXCEPTION: St13runtime_error
DeriveNewSeed: AddKeyPubKey failed
bitcoin in AppInit()

EXCEPTION: St13runtime_error
DeriveNewSeed: AddKeyPubKey failed
bitcoin in AppInit()

2018-10-07T05:24:06Z Shutdown: In progress... 2018-10-07T05:24:06Z scheduler thread interrupt 2018-10-07T05:24:06Z Shutdown: done

The wallet is shown correctly, and with the correct balance, if I run it without -upgradewallet
ryanofsky commented at 4:17 AM on October 8, 2018: contributor

Is the wallet encrypted? Looking at wallet code, it seems like AddKeyPubKey fails when wallet is encrypted and locked, but generally succeeds otherwise.
achow101 commented at 4:23 AM on October 8, 2018: member

The wallet looks like it is encrypted.

2018-10-07T05:24:05Z [default wallet] Keys: 0 plaintext, 1158 encrypted, 1158 w/ metadata, 1158 total. Unknown wallet records: 2

I guess upgradewallet isn't going to work when the wallet is encrypted. I guess we will need to have some RPC method for upgrading encrypted wallets.
redblade7 commented at 5:12 AM on October 8, 2018: none

Yes, the wallet is encrypted.
meshcollider added the label Wallet on Oct 15, 2018
ryanofsky cross-referenced this on Oct 15, 2018 from issue Build previous releases and run functional tests by Sjors
dna-johnny commented at 5:23 PM on October 26, 2018: none

FYI: I have the same behaviour as @redblade7
talisein commented at 11:13 PM on May 5, 2019: none

0.18.0 still crashing
redblade7 cross-referenced this on Jul 8, 2019 from issue "Can't generate a change-address key. No keys in the internal keypool and can't generate any keys." by redblade7
codewiz commented at 6:12 AM on July 9, 2019: none

Are there any known workarounds for this?

I thought of decrypting the wallet temporarily, but both the GUI and the console disallow empty passwords.
mattpopovich commented at 5:06 AM on August 28, 2019: none

v0.18.1, still crashing. Wallet created around v0.11.1 and encrypted some time after that, ~v0.15.0 or so.

Let me know if you need anything from me to help troubleshoot/reproduce. I'd be more than happy to help!
ppppso commented at 7:34 PM on March 6, 2020: none

Still crashing on 0.19.0.1
MarcoFalke commented at 4:14 PM on April 24, 2020: member

Is this still an issue after #15761 ?
codewiz commented at 3:07 PM on June 14, 2020: none

It seems #15761 was not merged in time for v0.20 :-( When is the 0.21 release planned for?
MarcoFalke commented at 3:11 PM on June 14, 2020: member

#18947
MarcoFalke added this to the milestone 0.21.0 on Jun 14, 2020
codewiz commented at 8:05 PM on June 14, 2020: none

I built bitcoin-qt from source (master branch), and I was able to update a pre-0.17 encrypted wallet. Thank you for doing this work, @achow101 .
ryanofsky commented at 5:22 PM on June 24, 2020: contributor

Not sure if we ever found out what underlying bug was, but if #14422 (comment) is an indication, it seems like it may be fixed.

Will close this bug report, but if anyone is still having an issue upgrading, feel to reopen and post here so we can try to debug. Maybe we can post a gitian build to make the new upgrade RPC easier to test.
ryanofsky closed this on Jun 24, 2020
MatthewLM cross-referenced this on Jul 6, 2020 from issue Bech32 address not provided when creating a new receiving address by MatthewLM
mattpopovich commented at 6:52 AM on July 23, 2020: none

All, excuse my ignorance as I've only used the GUI client previously. @codewiz, what was the process you used to upgrade your pre-0.17 encrypted wallet? Did #15761 get included in v0.20.1rc1? Or do I need to build from master? If I do build from master, it seems as if -upgradewallet startup option has been removed. So now would I use upgradewallet via bitcoin-cli?

I just built v0.20.1rc1 from source and am still receiving Error: DeriveNewSeed: AddKeyPubKey failed when I try to use the -upgradewallet startup option. I get a "Method not found" when I try to run bitcoin-cli upgradewallet.

I will try to build from master in the next day or two and report back.
mattpopovich commented at 10:59 PM on July 26, 2020: none

All, excuse my ignorance as I've only used the GUI client previously. @codewiz, what was the process you used to upgrade your pre-0.17 encrypted wallet? Did #15761 get included in v0.20.1rc1? Or do I need to build from master? If I do build from master, it seems as if -upgradewallet startup option has been removed. So now would I use upgradewallet via bitcoin-cli?

I just built v0.20.1rc1 from source and am still receiving Error: DeriveNewSeed: AddKeyPubKey failed when I try to use the -upgradewallet startup option. I get a "Method not found" when I try to run bitcoin-cli upgradewallet.

I will try to build from master in the next day or two and report back.

I have rebuilt from master (v0.20.99.0- ccef10261) and can confirm that the upgradewallet RPC works successfully on my wallet. Thanks for the development @achow101. FWIW: My wallet was created around v0.11.1 and encrypted some time after that, ~v0.15.0 or so.
mderasse cross-referenced this on Sep 16, 2020 from issue Using -upgradewallet flag crashes wallet by MakisChristou
MarcoFalke commented at 8:06 AM on November 27, 2020: member
fyi, rc2 is now available (if you test, please test with caution, as this is not a final release)
- https://lists.linuxfoundation.org/pipermail/bitcoin-core-dev/2020-November/000094.html
bitcoin locked this on Feb 15, 2022