partial spend avoidance makes partial spends and getbalances doesn't notice #17603

In v0.19.0.1, I'm running regtest, creating a wallet with avoid_reuse=true, generating 31 blocks in that wallet, then generating 100 more blocks to a throwaway address to have the rewards mature.

Then I send 1 BTC to a throwaway address, expecting to see all 31 block rewards spent at once, but the resulting transaction only has 1 input. The other 30 rewards show as "reused":true when I listunspent, but when I run getbalances I see them showing us as trusted, not as used like I would expect.

Here's a log:

$ junk=mmmmmmmmmmmmmmmmmmmmmmmmmmmmq4S8nX
$ alias bcr-'bitcoin-cli -regtest'
$ alias bcrt='bcr -rpcwallet=test'
$ bcr -named createwallet wallet_name=test avoid_reuse=true
{
  "name": "test",
  "warning": ""
}
$ addr=$(bcrt getnewaddress)
$ bcr generatetoaddress 31 $addr | wc -l # generate 31 rewards
33
$ bcr generatetoaddress 100 $junk | wc -l # let them mature
102
$ bcrt getbalances
{
  "mine": {
    "trusted": 1550.00000000,
    "untrusted_pending": 0.00000000,
    "immature": 0.00000000,
    "used": 0.00000000
  }
}
$ txid=$(bcrt sendtoaddress $junk 1)
# list the #confs for each input of the spend tx - note only one utxo was spent
$ bcrt getrawtransaction $txid 1 | jq -r '.vin[] | .txid' | while read txid; do bcrt gettransaction $txid | jq -r .confirmations; done
107
# list the #confs for the unspent rewards - note the 107 conf utxo is missing
$ echo $(bcrt listunspent | jq -cM '.[] | .confirmations' | sort)
101 102 103 104 105 106 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131
$ bcrt getbalances # but the 30 unspent rewards are still "trusted" not "used"
{
  "mine": {
    "trusted": 1548.99996640,
    "untrusted_pending": 0.00000000,
    "immature": 0.00000000,
    "used": 0.00000000
  }
}
# although they show as "reused" in listunspent:
$ bcrt listunspent 0 | jq -cM '.[] | {reused,amount,confirmations,spendable,address}' | sort | head -n 10
{"reused":false,"amount":48.9999664,"confirmations":0,"spendable":true,"address":"2NAnHxL7tve4xnom7Y3f5747qzyFDEpP1Li"}
{"reused":true,"amount":50,"confirmations":101,"spendable":true,"address":"2Mww9FTTSMjc9HGsrc6swRVVzkNaus6SEMw"}
{"reused":true,"amount":50,"confirmations":102,"spendable":true,"address":"2Mww9FTTSMjc9HGsrc6swRVVzkNaus6SEMw"}
{"reused":true,"amount":50,"confirmations":103,"spendable":true,"address":"2Mww9FTTSMjc9HGsrc6swRVVzkNaus6SEMw"}
{"reused":true,"amount":50,"confirmations":104,"spendable":true,"address":"2Mww9FTTSMjc9HGsrc6swRVVzkNaus6SEMw"}
{"reused":true,"amount":50,"confirmations":105,"spendable":true,"address":"2Mww9FTTSMjc9HGsrc6swRVVzkNaus6SEMw"}
{"reused":true,"amount":50,"confirmations":106,"spendable":true,"address":"2Mww9FTTSMjc9HGsrc6swRVVzkNaus6SEMw"}
{"reused":true,"amount":50,"confirmations":108,"spendable":true,"address":"2Mww9FTTSMjc9HGsrc6swRVVzkNaus6SEMw"}
{"reused":true,"amount":50,"confirmations":109,"spendable":true,"address":"2Mww9FTTSMjc9HGsrc6swRVVzkNaus6SEMw"}
{"reused":true,"amount":50,"confirmations":110,"spendable":true,"address":"2Mww9FTTSMjc9HGsrc6swRVVzkNaus6SEMw"}
$ 

Thank you for reporting this! I have suggested a fix for the issue related to coin selection and I will submit an improvement for getbalances as well.

The reason why you are only seeing one single input is because you are creating exactly 31 outputs. The code will create up to 10 size groups for a single destination, to avoid inadvertently creating massive transactions.

In your case, you will receive groups of 10, 10, 10, and 1 UTXO. The last one will obviously result in the most efficient transaction as it is 10x smaller than the others.

Unfortunately, this results in 1499 btc being marked as dirty in this particular case, but I don't believe this is particularly common in the wild. It is not ideal for a regtest with avoidreuse enabled, though, that's for sure.

To see what I mean, change the 31 to 30 in your initial generate to $addr, then txid should give a tx with 10 inputs, not 1.

It has been a while since I reported this, but wasn't the point to defeat chain analysis companies that spam our wallets with dust? If so, those companies can spam us with 10 pieces of dust per address to trigger this issue.

I used regtest because it's quicker and cheaper than doing the same thing on mainnet but I think the result would be the same whichever network I used wouldn't it?

Either way, having the remaining balance show up as "trusted" when it should show as "used" seems wrong to me.

It has been a while since I reported this, but wasn't the point to defeat chain analysis companies that spam our wallets with dust? If so, those companies can spam us with 10 pieces of dust per address to trigger this issue.

It is not that easy. It works in your example only because the amount you send in the transaction is smaller than each one of the outputs in the destination (1 BTC < 50 BTC block reward). If you keep everything else the same and send 51 BTC instead of 1 BTC the transaction should have 10 inputs as intended. So the attack is still possible but it would be much more expensive because dust outputs would not be enough to be selected as the input in most normal transactions and, as you said, they would have to send at least 10 of them if the address had not been reused before. Essentially the chain analysis company would have to spam a used destination with 10 outputs and then hope that the wallet user sends a transaction that is smaller than each one of the 10 outputs and that then one of the 10 outputs is selected as the input.

The other outputs are not showing up as "used" because cached values are used when that should not be the case. I am working on improving this.

What is left to do here?

Oh, #17824 is not yet merged