From b02c004321d29d3f9c752353f97b0ba9f5eebd9c Mon Sep 17 00:00:00 2001 From: apostolof Date: Tue, 23 Feb 2021 20:52:11 +0200 Subject: [PATCH] refactor: init separate readme files --- README.md | 31 +--- docker/README.md | 136 +++++++++++++----- docker/env/concordia.env | 2 +- packages/concordia-app/CONCORDIA.md | 36 +++++ .../CONTRACTS-PROVIDER.md | 22 +++ packages/concordia-contracts/CONTRACTS.md | 107 ++++++++++++++ packages/concordia-contracts/README.md | 68 --------- packages/concordia-pinner/PINNER.md | 29 ++++ 8 files changed, 298 insertions(+), 133 deletions(-) create mode 100644 packages/concordia-app/CONCORDIA.md create mode 100644 packages/concordia-contracts-provider/CONTRACTS-PROVIDER.md create mode 100644 packages/concordia-contracts/CONTRACTS.md delete mode 100644 packages/concordia-contracts/README.md create mode 100644 packages/concordia-pinner/PINNER.md diff --git a/README.md b/README.md index 5ede2d4..f80fded 100644 --- a/README.md +++ b/README.md @@ -3,35 +3,16 @@ ## Setup -```shell script -cd concordia -yarn -``` +You can find detailed instructions on how to get started with each service in the respective package directories. The +overall setup pipeline **for development** is this though: -## Compile contracts - -```shell script -cd packages/concordia-contracts -yarn compile -``` - -## Run app - -```shell script -cd packages/concordia-app -yarn start -``` - -## Build app - -```shell script -cd packages/concordia-app -yarn build -``` +- start rendezvous server and ganache +- migrate contracts +- start application ## Using Docker images -This project provides docker images for a number of services required to setup Concordia, as well as for Concordia +This project provides docker images for a number of services required to set up Concordia, as well as for Concordia itself. Check out the README.md in the `./docker` directory diff --git a/docker/README.md b/docker/README.md index b069dd7..ff8b8ea 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1,6 +1,6 @@ # Concordia Dockerized -This page provides information about the provided docker images, their configuration and supported deployment +This document gives information about the provided docker images, their configuration and supported deployment strategies. TLDR: head down to [Putting it all together/Scripts](#piat-mkfile-targets) for a quick setup. @@ -24,6 +24,16 @@ Note that any other Ethereum compliant blockchain can be used. Concordia uses a distributed database to store forum data. A rendezvous server is needed in order for users to discover peers in the network and get access to the data. +### Contracts Provider + +Contracts provider is a **very** simple resource provider server that handles saving the contract artifacts produced +during contracts migration and serving them to the users (and pinner). + +### Pinner + +Pinner is a bot node of the network that pins all IPFS content, so the forum data may be available even if the users +that create it are not. + ### Application The Concordia application is a React app that handles interactions with the contracts and the distributed database used. @@ -40,33 +50,51 @@ below. | Environment variable | Default value | Usage | | --- | --- | --- | -| ACCOUNTS_NUMBER | 10 | Set the number of accounts generated | -| ACCOUNTS_ETHER | 100 | Set the amount of ETH assigned to each account | -| MNEMONIC | NaN | The mnemonic phrase sued as a seed for deterministic account generation | -| HOST | 0.0.0.0 | The hostname to listen on | -| PORT | 8545 | The port to listen on | -| NETWORK_ID | 5778 | The network id used to identify ganache | +| ACCOUNTS_NUMBER | `10` | Set the number of accounts generated | +| ACCOUNTS_ETHER | `100` | Set the amount of ETH assigned to each account | +| MNEMONIC | `NaN` | The mnemonic phrase sued as a seed for deterministic account generation | +| HOST | `0.0.0.0` | The hostname to listen on | +| PORT | `8545` | The port to listen on | +| NETWORK_ID | `5778` | The network id used to identify ganache | Note that the Ganache instance running inside the container will save the generated blockchain keys in the path -`/home/ganache_keys/keys.json`. If you need to access the keys (eg for getting a private key and importing in Metamask) -you can mount a volume to this path to have easier access. +`/mnt/concordia/ganache_keys/keys.json`. If you need to access the keys (e.g. for getting a private key and importing in +Metamask) you can mount a volume to this path to have easier access. -Also, the database used by Ganache for storing blockchain information is placed in the path `/home/ganache_db/`. You can -maintain the blockchain state between runs by mounting a volume to the database path. To do that, add the docker flag -`-v host/absolute/path/to/ganache_db:/home/ganache_db`. +Also, the database used by Ganache for storing blockchain information is placed in the path +`/mnt/concordia/ganache_db/`. You can maintain the blockchain state between runs by mounting a volume to the database +path. To do that, add the docker flag `-v host/absolute/path/to/ganache_db:/mnt/concordia/ganache_db`. ### Rendezvous The rendezvous server used here is `js-libp2p-webrtc-star`. The server listens on port 9090. More information can be found on the github page of the project [here](https://github.com/libp2p/js-libp2p-webrtc-star). +### Contracts provider + +A Dockerfile for the contracts provider service is provided in the path `./concordia-contracts-provider`. The Dockerfile +contains only one stage, that is the runtime. + +The image makes use of the environment variables described below. + +| Environment variable | Default value | Usage | +| --- | --- | --- | +| CONTRACTS_PROVIDER_PORT | `8400` | Set the port of the contracts provider application | +| UPLOAD_CONTRACTS_DIRECTORY | `concordia/packages/concordia-contracts-provider/contracts-uploads` | Set the directory where the uploaded contracts are saved | +| LOGS_PATH | `concordia/packages/concordia-contracts-provider/logs` | Set the directory where the application logs are saved | +| CORS_ALLOWED_ORIGINS | `http://127.0.0.1:7000`, `http://localhost:7000`, `https://127.0.0.1:7000`, `https://localhost:7000`, `http://127.0.0.1:4444`, `http://localhost:4444`, `https://127.0.0.1:4444`, `https://localhost:4444` | Set the list of addresses allowed by CORS* | + ### Contracts -This is a provision system that compiles and deploys the contracts to any Ethereum blockchain. +This is a provision system that compiles and deploys the contracts to any Ethereum blockchain. It also uploads the +deployed contract artifacts to the contracts-provider service. + +**Attention**: the contracts-provider instance targeted by the environment variables MUST be up and running before +attempting to migrate the contracts. -A Dockerfile is provided in the path `./concordia-contracts` that will build the contracts used by Concordia and -handle their deployment to any Ethereum network defined using env-vars upon container run. Dockerfile contains three -useful stages, described in the table below. +A Dockerfile is provided in the path `./concordia-contracts` that will build the contracts used by Concordia during +image build and handle their deployment to any Ethereum network defined using env-vars upon container run. Dockerfile +contains three useful stages, described in the table below. | Stage name | Entrypoint | Usage | | --- | --- | --- | @@ -78,41 +106,71 @@ The image makes use of the environment variables described below. | Environment variable | Default value | Usage | | --- | --- | --- | -| MIGRATE_NETWORK | develop | Set the network where the contracts will be deployed/tested (set this to "env" unless you know what you're doing) | -| DEPLOY_CHAIN_HOST | NaN | Set the hostname of the blockchain network that will be used for deployment (requires network to be "env") | -| DEPLOY_CHAIN_PORT | NaN | Set the port of the blockchain network that will be used for deployment (requires network to be "env") | -| TEST_CHAIN_HOST | NaN | Set the hostname of the blockchain network that will be used for testing (requires network to be "env") | -| TEST_CHAIN_PORT | NaN | Set the port of the blockchain network that will be used for testing (requires network to be "env") | +| MIGRATE_NETWORK | `develop` | Set the network where the contracts will be deployed/tested (set this to "env" unless you know what you're doing) | +| WEB3_HOST | `127.0.0.1` | Set the hostname of the blockchain network that will be used for deployment (requires network to be "env") | +| WEB3_PORT | `8545` | `NaN` | Set the port of the blockchain network that will be used for deployment (requires network to be "env") | +| CONTRACTS_PROVIDER_HOST | `http://127.0.0.1` | Set the hostname of the contracts provider | +| CONTRACTS_PROVIDER_PORT | `8400` | Set the port of the contracts provider | +| CONTRACTS_VERSION_TAG | `NaN` | Set the tag of the contracts uploaded to provided | You can find the contract artifacts in the directory `/usr/src/concordia/packages/concordia-contracts/build/` inside the image. **Attention**: make sure the targeted blockchain is up and running before trying to migrate the contracts. +### Pinner + +A Dockerfile for the pinner service is provided in the path `./concordia-pinner`. The Dockerfile contains only one +stage, that is the runtime. + +The image makes use of the environment variables described below. + +| Environment variable | Default value | Usage | +| --- | --- | --- | +| PINNER_API_HOST | `127.0.0.1` | Set the hostname of the pinner application | +| PINNER_API_PORT | `4444` | Set the port of the pinner application | +| USE_EXTERNAL_CONTRACTS_PROVIDER | `false` | Enable/Disable use of external contracts provider | +| IPFS_DIRECTORY | `concordia/packages/concordia-pinner/ipfs` | Set the directory where the ipfs blocks are saved | +| ORBIT_DIRECTORY | `concordia/packages/concordia-pinner/orbitdb` | Set the directory where the orbitdb data are saved | +| LOGS_PATH | `concordia/packages/concordia-pinner/logs` | Set the directory where the application logs are saved | +| WEB3_HOST | `127.0.0.1` | Set the hostname of the blockchain | +| WEB3_PORT | `8545` | Set the port of the blockchain | +| RENDEZVOUS_HOST | `/ip4/127.0.0.1` | Set the hostname of the rendezvous server | +| RENDEZVOUS_PORT | `9090` | Set the port of the rendezvous server | +| CONTRACTS_PROVIDER_HOST | `http://127.0.0.1` | Set the hostname of the contracts provider service | +| CONTRACTS_PROVIDER_PORT | `8400` | Set the port of the contracts provider service | +| CONTRACTS_VERSION_HASH | `latest` | Set the contracts tag that will be pulled | + ### Application -The Dockerfile provided in the path `./concordia-application` builds the application for production and serves -the resulting build using an nginx server. Dockerfile contains two useful stages, described in the table below. +The Dockerfile provided in the path `./concordia-application` either builds the application for production and serves +the resulting build using an nginx server or simply runs the node development server. Dockerfile contains four stages, +described in the table below. | Stage name | Entrypoint | Usage | | --- | --- | --- | | test | Runs tests | Fetches npm packages and runs tests | -| runtime | Serves application | Builds for production and serves it through nginx | - +| staging | Serves application for development | Starts the node development server | +| runtime | Serves application for production | Builds for production and serves it through nginx | The image makes use of the environment variables described below. | Environment variable | Default value | Usage | | --- | --- | --- | -| REACT_APP_RENDEZVOUS_HOST | 127.0.0.1 | Set the hostname of the rendezvous server | -| REACT_APP_RENDEZVOUS_PORT | 9090 | Set the port of the rendezvous server | +| REACT_APP_CONCORDIA_HOST | `127.0.0.1` | Set the hostname of the concordia application | +| REACT_APP_CONCORDIA_PORT | `7000` | Set the port of the concordia application | +| REACT_APP_RENDEZVOUS_HOST | `/ip4/127.0.0.1` | Set the hostname of the rendezvous server | +| REACT_APP_RENDEZVOUS_PORT | `9090` | Set the port of the rendezvous server | +| REACT_APP_USE_EXTERNAL_CONTRACTS_PROVIDER | `false` | Enable/Disable use of external contracts provider | +| REACT_APP_CONTRACTS_PROVIDER_HOST | `http://127.0.0.1` | Set the hostname of the contracts provider service | +| REACT_APP_CONTRACTS_PROVIDER_PORT | `8400` | Set the port of the contracts provider service | +| REACT_APP_CONTRACTS_VERSION_HASH | `latest` | Set the contracts tag that will be pulled | -**Attention**: this image will copy the contract artifacts from the directory `/packages/concordia-contracts/build`. -The image is bound the these artifacts after build. If the contracts change or get re-deployed the image must be -re-built to use the new artifacts. +**Attention**: this image will copy the contract artifacts from the directory `/packages/concordia-contracts/build` if +available. The image can then use these artifacts after build or pull new artifacts from a contracts provider. -**Attention**: make sure the contracts have been deployed before **building** this image. Also, make sure the rendezvous -server is up and running. +**Attention**: if you plan to use the imported contract artifacts instead of a provider, make sure the contracts have +been deployed before **building** this image. Also, make sure the rendezvous server is up and running. ## Docker Compose @@ -194,11 +252,11 @@ the hood is that: ### Env Files -Targets in the Makefile make use of env files suffixed by `.docker` located in the directory `./env`. Using this -environment variables, you can change various configuration options of the testing/production deploys. +Targets in the Makefile make use of env files located in the directory `./env`. Using this environment variables, you +can change various configuration options of the testing/production deploys. -Targets suffixed with `host-chain` will try to use a blockchain and rendezvous server running in the host machine. They -use the `--net=host` docker option and get the required environment variables from different env files, -`./env/contracts.env` and `./env/concordia.env` (notice these env files don't include the `.docker`). These env files do -not exist by default. The values set will largely depend on how you choose to run services in your system (which ports -you use etc.), so be sure to create them before running any `host-chain` target. Luckily example files are provided. +Makefile targets suffixed with `host-chain` will try to use a blockchain and rendezvous server running in the host +machine. They use the `--net=host` docker option. In order to work with these targets you need to create and use your +own env files (or modify the existing ones). The environment variables values will largely depend on how you choose to +run the services in your system (which ports you use etc.), so be sure to create them before running any `host-chain` +target. diff --git a/docker/env/concordia.env b/docker/env/concordia.env index 4f270ad..2a5b143 100755 --- a/docker/env/concordia.env +++ b/docker/env/concordia.env @@ -1,4 +1,4 @@ # Variables needed in runtime (in browser) REACT_APP_RENDEZVOUS_HOST=/ip4/127.0.0.1 REACT_APP_RENDEZVOUS_PORT=9090 -USE_EXTERNAL_CONTRACTS_PROVIDER=true +REACT_APP_USE_EXTERNAL_CONTRACTS_PROVIDER=true diff --git a/packages/concordia-app/CONCORDIA.md b/packages/concordia-app/CONCORDIA.md new file mode 100644 index 0000000..59e0c48 --- /dev/null +++ b/packages/concordia-app/CONCORDIA.md @@ -0,0 +1,36 @@ +# Concordia + +The Concordia application is a React app that handles interactions with the contracts and the distributed database used. + +This document provides information about the Concordia React Application and how to get it running for development. + +## Prerequisites + +Concordia requires at the minimum two services to work, a blockchain and a rendezvous server. + +Furthermore, the deployed contract artifacts must be made available. This can be done in two ways, migrate the contracts +and make sure the artifacts are present in the `concordia/packages/concordia-contracts/build` directory, or migrate the +contracts and upload the artifacts to a contracts-provider instance. + +## Running Concordia + +To start the application in development mode simply execute the `start` script: +```shell +yarn start +``` + +The application makes use of the environment variables described below. + +| Environment variable | Default value | Usage | +| --- | --- | --- | +| REACT_APP_CONCORDIA_HOST | `127.0.0.1` | Set the hostname of the concordia application | +| REACT_APP_CONCORDIA_PORT | `7000` | Set the port of the concordia application | +| REACT_APP_RENDEZVOUS_HOST | `/ip4/127.0.0.1` | Set the hostname of the rendezvous server | +| REACT_APP_RENDEZVOUS_PORT | `9090` | Set the port of the rendezvous server | +| REACT_APP_USE_EXTERNAL_CONTRACTS_PROVIDER | `false` | Enable/Disable use of external contracts provider | +| REACT_APP_CONTRACTS_PROVIDER_HOST | `http://127.0.0.1` | Set the hostname of the contracts provider service | +| REACT_APP_CONTRACTS_PROVIDER_PORT | `8400` | Set the port of the contracts provider service | +| REACT_APP_CONTRACTS_VERSION_HASH | `latest` | Set the contracts tag that will be pulled | + +**Attention**: if using a contracts provider, make sure that the provider is set to allow CORS from the host-port combo +defined by `REACT_APP_CONCORDIA_HOST` and `REACT_APP_CONCORDIA_PORT`. diff --git a/packages/concordia-contracts-provider/CONTRACTS-PROVIDER.md b/packages/concordia-contracts-provider/CONTRACTS-PROVIDER.md new file mode 100644 index 0000000..db0f7b0 --- /dev/null +++ b/packages/concordia-contracts-provider/CONTRACTS-PROVIDER.md @@ -0,0 +1,22 @@ +# Concordia Contracts Provider + +Contracts provider is a **very** simple resource provider server that handles saving the contract artifacts produced +during contracts migration and serving them to the users (and pinner). + +## Running Concordia + +To start the application in development mode simply execute the `start` script: +```shell +yarn start +``` + +The application makes use of the environment variables described below. + +| Environment variable | Default value | Usage | +| --- | --- | --- | +| CONTRACTS_PROVIDER_PORT | `8400` | Set the port of the contracts provider application | +| UPLOAD_CONTRACTS_DIRECTORY | `concordia/packages/concordia-contracts-provider/contracts-uploads` | Set the directory where the uploaded contracts are saved | +| LOGS_PATH | `concordia/packages/concordia-contracts-provider/logs` | Set the directory where the application logs are saved | +| CORS_ALLOWED_ORIGINS | `http://127.0.0.1:7000`, `http://localhost:7000`, `https://127.0.0.1:7000`, `https://localhost:7000`, `http://127.0.0.1:4444`, `http://localhost:4444`, `https://127.0.0.1:4444`, `https://localhost:4444` | Set the list of addresses allowed by CORS* | + +* the `CORS_ALLOWED_ORIGINS` env var should be a semicolon-separated string, e.g. `127.0.0.1:7000;localhost:7000`. diff --git a/packages/concordia-contracts/CONTRACTS.md b/packages/concordia-contracts/CONTRACTS.md new file mode 100644 index 0000000..560a4fe --- /dev/null +++ b/packages/concordia-contracts/CONTRACTS.md @@ -0,0 +1,107 @@ +# Concordia Contracts + +In this package live the contracts that power Concordia. + +This document provides information about how to build and migrate the contracts. + +## Prerequisites + +To migrate the contracts, the targeted blockchain must be up and running. Furthermore, if uploading to a contracts +provider, the provider must be available. + +## Compiling + +To compile the contracts execute the `compile` script: + +```shell +yarn compile +``` + +## Linting + +To lint the contracts execute the `lint` script: + +```shell +yarn lint +``` + +## Testing + +Default host and port values of the blockchain used for testing are: + +| host | port | +|---|---| +| 127.0.0.1 | 8546 | + + +Test using the default network: + +```shell script +yarn test +``` + +### Using different host and port values for testing blockchain + +Define the host and port of the blockchain using environment variables (see below). Test using the `env` network: + +```shell script +yarn test --network env +``` + +## Migrating + +Default host and port values of the blockchain are: + +| host | port | +|---|---| +| 127.0.0.1 | 8545 | + +Migrate (using the development network by default): +```shell script +yarn migrate +``` + +### Setting different host and port values + +Define the host and port of the blockchain using environment variables (see below). Migrate using the `env` network: + +```shell script +yarn _migrate --network env +``` + +**Notice the underscore `_` prefix in the script name. This is not a mistake.** + +## Uploading + +Default hash and tag values used during contracts upload: + +| hash | tag | +|---|---| +| -dev | latest | + +To upload the contracts to a contracts provider instance execute the `upload` script: + +```shell +yarn upload +``` + +### Setting different hash and tag values + +Pass the hash and tag of the contracts during script call: + +```shell script +yarn _upload +``` + +**Notice the underscore `_` prefix in the script name. This is not a mistake.** + +## Environment Variables + +The environment variables used are described below. + +| Environment variable | Default value | Usage | +| --- | --- | --- | +| WEB3_HOST | `127.0.0.1` | Set the hostname of the blockchain | +| WEB3_PORT | `8545` | Set the port of the blockchain | +| CONTRACTS_PROVIDER_HOST | `http://127.0.0.1` | Set the hostname of the contracts provider | +| CONTRACTS_PROVIDER_PORT | `8400` | Set the port of the contracts provider | diff --git a/packages/concordia-contracts/README.md b/packages/concordia-contracts/README.md deleted file mode 100644 index 3dffe02..0000000 --- a/packages/concordia-contracts/README.md +++ /dev/null @@ -1,68 +0,0 @@ -# Concordia Contracts Package - -This is the package where the contracts that power Concordia live. - -## Compile contracts - -```shell script -yarn compile -``` - -## Lint contracts (and tests) -```shell script -yarn lint -``` - -## Migrate contracts -Default host and port values of the blockchain are: - -| host | port | -|---|---| -| 127.0.0.1 | 8545 | - -Migrate (using the development network by default): -```shell script -yarn migrate -``` - -### Setting different host and port values -Define the host and port of the blockchain in use. - -Linux: -```shell script -export CHAIN_HOST="127.0.0.1" -export CHAIN_PORT="7545" -``` - -Windows: -```shell script -SET CHAIN_HOST="127.0.0.1" -SET CHAIN_PORT="7545" -``` - -Migrate using the `env` network : -```shell script -yarn _migrate --network env -``` -**Notice the underscore `_` suffix in the script name. This is not a mistake.** - -## Test contracts -Default host and port values of the blockchain are: - -| host | port | -|---|---| -| 127.0.0.1 | 8546 | - - -Test: -```shell script -yarn test -``` - -### Setting different host and port values -Define the host and port of the blockchain in use like above. - -Test: -```shell script -yarn test --network env -``` diff --git a/packages/concordia-pinner/PINNER.md b/packages/concordia-pinner/PINNER.md new file mode 100644 index 0000000..a499d59 --- /dev/null +++ b/packages/concordia-pinner/PINNER.md @@ -0,0 +1,29 @@ +# Concordia Pinner + +Pinner is a bot node of the network that pins all IPFS content, so the forum data may be available even if the users +that create it are not. + +## Running Pinner + +To start the application in development mode simply execute the `start` script: +```shell +yarn start +``` + +The application makes use of the environment variables described below. + +| Environment variable | Default value | Usage | +| --- | --- | --- | +| PINNER_API_HOST | `127.0.0.1` | Set the hostname of the pinner application | +| PINNER_API_PORT | `4444` | Set the port of the pinner application | +| USE_EXTERNAL_CONTRACTS_PROVIDER | `false` | Enable/Disable use of external contracts provider | +| IPFS_DIRECTORY | `concordia/packages/concordia-pinner/ipfs` | Set the directory where the ipfs blocks are saved | +| ORBIT_DIRECTORY | `concordia/packages/concordia-pinner/orbitdb` | Set the directory where the orbitdb data are saved | +| LOGS_PATH | `concordia/packages/concordia-pinner/logs` | Set the directory where the application logs are saved | +| WEB3_HOST | `127.0.0.1` | Set the hostname of the blockchain | +| WEB3_PORT | `8545` | Set the port of the blockchain | +| RENDEZVOUS_HOST | `/ip4/127.0.0.1` | Set the hostname of the rendezvous server | +| RENDEZVOUS_PORT | `9090` | Set the port of the rendezvous server | +| CONTRACTS_PROVIDER_HOST | `http://127.0.0.1` | Set the hostname of the contracts provider service | +| CONTRACTS_PROVIDER_PORT | `8400` | Set the port of the contracts provider service | +| CONTRACTS_VERSION_HASH | `latest` | Set the contracts tag that will be pulled |