[tests] [utils] test bitcoin-cli #10798

1. 
   jnewbery commented at 10:36 PM on July 11, 2017: member

   We don't test bitcoin-cli at all. That means that we can miss inconsistencies between the bitcoin-cli client and the RPC interface, such as #10698 and #10747. It also means that the various bitcoin-cli options and features are untested and regressions could be silently introduced.

   Let's fix that.

   This PR adds bitcoin-cli testing in the python functional test_framework:

   1. Add a bitcoin_cli.py test script that tests bitcoin-cli. At the moment it only tests that the result of getinfo is the same if you run it as an RPC or through bitcoin-cli, but can easily be extended to test additional bitcoin-cli features

   EDIT: --usecli option is moved to a separate PR. This PR now only covers the bitcoin_cli.py test.

   2. ~Add a --usecli option to the test framework. This changes the test to use bitcoin-cli for all RPC calls instead of using direct HTTP requests. This is somewhat experimental. It works for most tests, but there are some cases where it can't work transparently because:~

   • ~the testcase is asserting on a specific error code, and bitcoin-cli returns a different error code from the direct RPC~
   • ~we're sending a very large RPC request (eg submitblock) and it can't be serialized into a shell bitcoin-cli call.~

   ~I think that even though --usecli doesn't work on all tests, it's still a useful experimental feature. Future potential enhancements:~

   • ~enhance the framework to automatically skip tests that are known to fail with bitcoin-cli if the --usecli option is used.~
   • ~run a subset of tests in Travis with -usecli~

   This builds on and requires the TestNode PR #10711 . As an aside, this is a good demonstration of how tidy it is to add additional features/interfaces now that test node logic/state is encapsulated in a TestNode class.

   Addresses #10791

2. jnewbery renamed this:
   test bitcoin-cli
   [tests] [utils] test bitcoin-cli
   on Jul 11, 2017
3. fanquake added the label Tests on Jul 11, 2017
4. jnewbery force-pushed on Jul 12, 2017
5. jnewbery force-pushed on Jul 13, 2017
6. 
   jnewbery commented at 11:29 AM on July 13, 2017: member

   Changed subprocess.run() to subprocess.check_output() to be compatible with pre3.5 python

7. laanwj cross-referenced this on Jul 20, 2017 from issue Handle getinfo in bitcoin-cli w/ -getinfo (revival of #8843) by achow101
8. jnewbery cross-referenced this on Jul 30, 2017 from issue [tests] Fix RPC failure testing (again) by jnewbery
9. [tests] add TestNodeCLI class for calling bitcoin-cli for a node b23549f6e6
10. [tests] Add bitcoin_cli.py test script c6ec4358a7
11. jnewbery force-pushed on Aug 15, 2017
12. 
    jnewbery commented at 10:01 PM on August 15, 2017: member

    Rebased now that #10711 is merged

13. jnewbery cross-referenced this on Aug 24, 2017 from issue Add bitcoin-cli -stdin and -stdinrpcpass functional tests by promag
14. 
    MarcoFalke commented at 7:59 PM on August 24, 2017: member

    Looks really straightforward.

    utACK c6ec4358a797b7a11283238a0cf0b4531def9e92

15. MarcoFalke merged this on Aug 24, 2017
16. MarcoFalke closed this on Aug 24, 2017

    ---

17. MarcoFalke referenced this in commit 4ae6d0fbef on Aug 24, 2017
18. jnewbery deleted the branch on Aug 24, 2017
19. laanwj cross-referenced this on Sep 6, 2017 from issue Need tests for bitcoin-cli by laanwj
20. laanwj added the label Needs backport on Sep 6, 2017
21. laanwj added this to the milestone 0.15.1 on Sep 6, 2017
22. jnewbery cross-referenced this on Sep 19, 2017 from issue [Test] Tests for zmqpubrawtx and zmqpubrawblock by achow101
23. MarcoFalke referenced this in commit 016b9ada21 on Oct 3, 2017
24. MarcoFalke referenced this in commit 716066dbea on Oct 3, 2017
25. MarcoFalke removed the label Needs backport on Oct 4, 2017
26. PastaPastaPasta referenced this in commit f524b78783 on Sep 19, 2019
27. PastaPastaPasta referenced this in commit 78c04e8126 on Sep 23, 2019
28. PastaPastaPasta referenced this in commit 095eb5d1cf on Sep 24, 2019
29. codablock referenced this in commit 1adc2001a8 on Sep 24, 2019
30. barrystyle referenced this in commit e8bcd62ce1 on Jan 22, 2020
31. bitcoin locked this on Sep 8, 2021

Contributors

jnewberyMarcoFalke

Milestone
0.15.2