Backport chainstate obfuscation to 0.11 #6919

pull laanwj wants to merge 6 commits into bitcoin:0.11 from laanwj:2015_10_backport_chainstate_obfuscation changing 9 files +565 −65

laanwj commented at 8:07 AM on October 31, 2015: member

This is a direct backport of #6650 and its follow-up, #6777 and #6865

Add chainstate obfuscation to avoid spurious antivirus detection

Adds an `obfuscate` parameter to `CLevelDBWrapper` and makes use of it
for all new chainstate stores built via `CCoinsViewDB`. Also adds an
`Xor` method to `CDataStream`.

Thanks to @sipa @laanwj @pstratem @dexX7 @KyrosKrane @gmaxwell.

Conflicts:
	src/Makefile.test.include

Github-Pull: #6650
Rebased-From: 42cb388167ef78f47a3a440eb651b6938c10f508

11d88c2ca2

Encapsulate CLevelDB iterators cleanly

Conflicts:
	src/leveldb.cpp
	src/leveldb.h
	src/txdb.cpp

Github-Pull: #6777
Rebased-From: 3499ce1e1ad87a86598d00b7124072c91ddad833

a8d781f863

Handle obfuscation in CLevelDBIterator

Github-Pull: #6777
Rebased-From: 0fdf8c80ee322ab747321d61faf9c72af4a51445

ba03d14234

laanwj force-pushed on Oct 31, 2015
laanwj commented at 9:42 AM on October 31, 2015: member
I like this backported becase:
- It it pretty well self-contained
- Will make it possible to downgrade from databases started with 0.12/master to 0.11.x
- Can be combined with #6917 to mitigate all known windows corruption issues
MarcoFalke commented at 9:48 AM on October 31, 2015: member

Concept ACK
dexX7 commented at 12:53 PM on October 31, 2015: contributor

Please note that #6865 should be included, too.
laanwj commented at 4:20 PM on October 31, 2015: member

Please note that #6865 should be included, too.

Good catch! included.

Add tests for gettxoutsetinfo, CLevelDBBatch, CLevelDBIterator

Thanks @dexX7.

Conflicts:
	qa/pull-tester/rpc-tests.py

Github-Pull: #6777
Rebased-From: 14885068726a8e0dc73ffa127225ab80be3e3612

775d01d4e1

Refer to obfuscate_key via pointer in peripheral CLevelDB classes

cc @sipa

Github-Pull: #6777
Rebased-From: dcd8e27c65de0d2cb972588a6e811ca7ccd1b3bd

8b7354bf2c

Fix chainstate serialized_size computation

Github-Pull: #6865
Rebased-From: 298e040bca0aad62b616a1d3e66e1e44d2b9bd3a

8d043126ca

laanwj force-pushed on Oct 31, 2015
luke-jr commented at 6:06 PM on October 31, 2015: member

Suggest disabling (using null) obfuscation by default in 0.11 so it remains compatible to 0.10.
gmaxwell commented at 7:02 PM on October 31, 2015: contributor

@luke-jr ugh, that would defeat much of the purpose because the people who need this won't know to turn it on. If you already have data and don't reindex, it stays null however.
luke-jr commented at 7:35 PM on October 31, 2015: member

As I see it, the primary purpose is of the backport is to be compatible with 0.12. If we don't want to wait on new features like this, we should just go ahead and release 0.12 early.
gmaxwell commented at 7:42 PM on October 31, 2015: contributor

@luke-jr I don't think that is an interesting reason to backport. Combined with the levedb fix this will end the constant corruption issues that are making the software unusable to many windows users.
laanwj commented at 9:00 AM on November 1, 2015: member

Suggest disabling (using null) obfuscation by default in 0.11 so it remains compatible to 0.10.

IMO due to the way obfuscation is used, only for new databases, that's not an issue. The people that want to go back to 0.10 will (usually) have started syncing on 0.10. And in the rare case it's not, they can just reindex.
laanwj commented at 8:17 AM on November 5, 2015: member

Closing this - even though this was a zero-hassle backport I'm not convinced that this is correct and I don't expect to find enough testing.
laanwj closed this on Nov 5, 2015
laanwj cross-referenced this on Nov 9, 2015 from issue [Docs] First-draft release notes for 0.11.2RC1 by harding
MarcoFalke cross-referenced this on Dec 26, 2015 from issue Assertion `hashPrevBlock == view.GetBestBlock()' failed. Aborted (core dumped) by MarcoFalke
MarcoFalke cross-referenced this on Dec 26, 2015 from issue [0.11] dbwrapper: Detect obfuscation by MarcoFalke
dgenr8 cross-referenced this on Jan 26, 2016 from issue Detect obfuscation by dgenr8
daira cross-referenced this on Jan 26, 2017 from issue Educate Windows users to configure anti-virus software to exclude the Zcash data folder by bitcartel
bitcoin locked this on Sep 8, 2021

Contributors

Linked (view graph)

#6650 Obfuscate chainstate #6777 Unobfuscate chainstate data in CCoinsViewDB::GetStats #6865 Fix chainstate serialized_size computation #6917 leveldb: Win32WritableFile without memory mapping #6968 [Docs] First-draft release notes for 0.11.2RC1 #7258 Assertion `hashPrevBlock == view.GetBestBlock()' failed. Aborted (core dumped)#7259 [0.11] dbwrapper: Detect obfuscation