Skip to content

README.md

Latest commit

 

History

History
231 lines (165 loc) · 11.9 KB

README.md

File metadata and controls

231 lines (165 loc) · 11.9 KB

Concord-BFT: a Distributed Trust Infrastructure

License clang-tidy Build Status Build Status Build Status Build Status codecoverage

Overview

Concord-bft is a generic state machine replication library that can handle malicious (byzantine) replicas.

Its implementation is based on the algorithm described in the paper SBFT: a Scalable Decentralized Trust Infrastructure for Blockchains.

It is designed to be used as a core building block for replicated distributed data stores, and is especially suited to serve as the basis of permissioned Blockchain systems.

For a real-life integration example, please take a look at Project Concord, a highly scalable and energy-efficient distributed trust infrastructure for consensus and smart contract execution.

Releases

We abide by semantic versioning. Public APIs can change at any time until we reach version 1.0. We will, however, bump the minor version when a backwards incompatible change is made.

v0.5

Install and Build (Ubuntu Linux 18.04)

Concord-BFT supports two kinds of builds are,

  • Docker
  • Native

The docker build is strongly recommended

Docker

cd concord-bft
make
make test

Run make help to see more commands.

Note:

  • If you face any format related issues than run make format before running make command to build concord-bft.
  • The output binaries are stored in the host's concord-bft/build.
  • Default compiler is clang and clang++.
  • Makefile is configurable. If you want to use another compiler you may pass it to the make. For example,
make CONCORD_BFT_CONTAINER_CXX=g++ \
    CONCORD_BFT_CONTAINER_CC=gcc \
    build

Other build options, including passthrough options for CMake, are defined in the Makefile and prefixed with CONCORD_BFT_. Variables that are capable of being overridden on the commandline are set with the Make conditional operator ?= and are at the beginning of Makefile. Please check that file for options.

Native

git clone https://github.com/vmware/concord-bft
cd concord-bft
sudo ./install_deps.sh    # Install all dependencies and 3rd parties
mkdir build
cd build
cmake ..
make
sudo make test

In order to turn on or off various options, you need to change your cmake configuration. This is done by passing arguments to cmake with a -D prefix: e.g. cmake -DBUILD_TESTING=OFF. Note that make must be run afterwards to build according to the configuration. Please see CMakeLists.txt for configurable options.

Select comm module

One option that is worth calling out explicitly is the communication (transport) library. Transport defaults to TLS and can be configured explicitly by setting the CONCORD_BFT_CMAKE_TRANSPORT flag. The flag defaults to TLS, but also supports UDP and TCP. In order to enable TCP or UDP communication, build with below flags,

make build CONCORD_BFT_CMAKE_TRANSPORT=TCP    # To enable TCP communication
make build CONCORD_BFT_CMAKE_TRANSPORT=UDP    # To enable UDP communication

These can be useful because the use of pinned certificates for TLS requires an out of band setup.

See create_tls_certs.sh for an example. This script is used in apollo tests. For production usage, an out of band deployment for each replica must be used to avoid revealing private keys to each replica.

C++ Linter

The C++ code is statically checked by clang-tidy as part of the CI.
To check code before submitting PR, please run make tidy-check.

Detailed information about clang-tidy checks.

(Optional) Python client

The python client is required for running tests. If you do not want to install python, you can configure the build of concord-bft by running cmake -DBUILD_TESTING=OFF .. from the build directory for native builds, and CONCORD_BFT_CMAKE_BUILD_TESTING=TRUE make for docker builds.

The python client requires python3(>= 3.5) and trio, which is installed via pip.

python3 -m pip install --upgrade trio

Adding a new dependency or tool

The CI builds and runs tests in a docker container. To add a new dependency or tool, follow the steps below:

  • Rebase against master
  • In order to add/remove dependencies update the file install_deps.sh
  • Build a new image: make build-docker-image
  • Check image current version in the Makefile
  • Tag the new image: docker tag concord-bft:latest concordbft/concord-bft:<version>,
    where version is current version + 1.
  • Update the version in the Makefile
  • Make sure that Concord-BFT is built and tests pass with the new image: make stop-c build test
  • Ask one of the maintainers for a temporary write permission to Docker Hub repository(you need to have a Docker ID)
  • Push the image: docker push concordbft/concord-bft:<version>
  • Create a PR for the update:
    • The PR must contain only changes related to the updates in the image
    • PR's summary has to be similar to Docker update to version <new version>
    • PR's message has to list the changes made in the image content and preferably the reason
    • Submit the PR

Important notes:

  1. Adding dependencies or tools directly to the Dockerfile is strongly not recommended because it breaks the native build support.
  2. If any tools are installed during the build but not needed for the actual compilation/debugging/test execution(for example, git), please remove them(Dockerfile is an example). The reason is that the image is supposed to be as tiny as possible.

Apollo testing framework

The Apollo framework provides utilities and advanced testing scenarios for validating Concord BFT's correctness properties, regardless of the running application/execution engine. For the purposes of system testing, we have implemented a "Simple Key-Value Blockchain" (SKVBC) test application which runs on top of the Concord BFT consensus engine.

Apollo enables running all test suites (without modification) against any supported BFT network configuration (in terms of n, f, c and other parameters).

Various crash or byzantine failure scenarios are also covered (including faulty replicas and/or network partitioning).

Apollo test suites run regularly as part of Concord BFT's continuous integration pipeline.

Please find more details about the Apollo framework here

Run examples

Simple Test application

Simple Test application is used to demonstrate how to use concord and enables you to perform READ and WRITE requests from client to replicas on a single-register state machine and then the consensus algorithm is run using this setup. Simple Test is compiled into the build directory and can be run from anywhere as long as it isn't moved. Simple Test can be run using 2 methods:

1) Shell script

./build/tests/simpleTest/scripts/testReplicasAndClient.sh You can use this shell script to run Simple Test using default configuration i.e. 4 replicas and 1 client.

2) Python script

./build/tests/simpleTest/scripts/simpleTest.py You can use the simpleTest.py script to run various configurations via a simple command line interface. Please find more information on Simple Test here

Directory Structure

  • bftengine: Library containing State transfer, SBFT consensus, and Pre execution.
  • ccron: Distributed Job Scheduler. Find more information here
  • client: Contains the implementation of BFT Client and Thin Replica Client.
  • communication: Contains the abstractions which is used by all the Replicas to send and receive messages.
  • diagnostics: This library is used for building interactive diagnostics for applications built on top of concord-bft.
  • docs: Contains documentation of design decisions for various components of concord. This is a place where we will opensource our design and provide tutorials to onboard new developers from the community.
  • kvbc: This contains the code for Key-Value Blockchain (Merkle tree-based). Find more information here
  • logging: Log4cpp abstraction for concord open source.
  • messages: Concord Message format library which does code generation for serialization and deserialization of messages. Find more information here
  • performance: Components provided for making better performance measurements and bottleneck analysis.
  • reconfiguration: This is the reconfiguration handler library. But major code of reconfiguration is in closed source.
  • scripts: Some miscellaneous scripts.
  • secretsmanager: Abstraction for encryption and decryption.
  • storage: Library abstracting the storage layer. It provides functions to access RocksDB and MemoryDB.
  • tests: Contains a Tester Replica, Test client, and Apollo Testing Framework.
  • thin-replicap-server: Implementation of Thin Replica Server.
  • thirdparty: Some third party cmake files for external dependencies.
  • threshsign: crypto library that supports digital threshold signatures.
  • tools: This is a tools folder for any tools that we create, like tools for generating all the keys required by the replicas in a Concord cluster.
  • util: This contains the utility libraries, like thread pools, metrics, asserting, crypto utilities, etc.

Contributing

The concord-bft project team welcomes contributions from the community. If you wish to contribute code and you have not signed our contributor license agreement (CLA), our bot will update the issue when you open a Pull Request. For any questions about the CLA process, please refer to our FAQ. For more detailed information, refer to CONTRIBUTING.md.

Community

Concord-BFT Slack.

Request a Slack invitation via concordbft@gmail.com.

License

concord-bft is available under the Apache 2 license.
⚡