Skip to main content

Setting Up a Flow Access Node

This guide is for running a permissonless Access node on Flow. If you are planning to run a different type of staked node then see node bootstrap.

Permissionless Access nodes allow any operator to run a Flow Access node. Unlike the other staked nodes, a permissionless access node does not have to be approved by the service account before it can join the network, hence the term "permissionless". The goal is to make all node types permissionless and this is the first step towards achieving that goal.

Who should run a Permissionless Access node?

dApp developers can choose to run their own private permissionless access node and move away from using the community access nodes. This will also allow them to not be subjected to the API rate limits of the public access nodes.

Node operators can also run their own permissionless access node and provide access to that node as a service.

Chain analytics, audit and exploration applications can run such an access node and do not have to rely on third parties for the state of the network.

Timing

New nodes are able to join the network each time a new epoch begins. An epoch is a period of time (approximately one week) when the node operators in the network are constant. At epoch boundaries, newly staked node operators are able to join the network and existing node operators which have unstaked may exit the network. You can read more about epochs here.

In order to join the network at epoch N+1, the access node must be registered with at least 100 FLOW staked prior to the end of epoch N's Staking Auction Phase.

Currently on mainnet, the staking auction starts every Wednesday at around 20:00 UTC and ends on the next Wednesday at around 12:00 UTC. Since this deadline may shift slightly from epoch to epoch, we recommend the node be staked by Wednesday, 8:00 UTC to be able to join the network in the next epoch.

Confirmation of a new node's inclusion in epoch N+1 is included in the EpochSetup event.

Flow Epoch Schedule

Limitations

This is a preliminary implementation of the full permissionless node operation feature and there will be exactly five slots in total that will be opened for access nodes. If there are more than five staked candidate access nodes at the end of the staking phase of the current epoch which ends on Wednesday, 15th Feb, five of those will be selected by an on-chain random selection process and no other permissionless ANs can join until others unstake in a future epoch.

More slots will be opened in the future.

Support for unlimited access nodes requires staking auctions to be implemented, which is still an upgrade planned for the future.

To summarize,

DateTimeEpochEpoch Phase
02/08/202320:00 UTC63Staking auction startsStake your access nodes
02/15/202308:00 UTC63Staking auction endsNo more access nodes can be staked for epoch 64
02/15/202320:00 UTC64Epoch n+1 starts5 randomly chosen access nodes can join the network

Please ensure that you have staked the access node by 15th Feb, 2023 Wednesday 08:00 UTC if you wish to run the node in the next epoch.

How to run a Permissionless Access node?

Note: To run an access node you will need to provision a machine or virtual machine to run your node software. Please follow the node-provisioning guide for it. You can provision the machine before or after your node has been chosen.

At a high level, to run a permissionless Access node, you will have to do the following steps:

  1. Generate the node identity (private and public keys, node ID etc.).
  2. Stake the node with 100 FLOW by the end of the staking phase of the current epoch (see timing) by providing the node information generated in step 1.
  3. You can verify if your node ID was selected by the on-chain random selection process on Wednesday at around 20:00 UTC when the next epoch starts.
  4. If your node ID was selected, you can provision and start running the node. If your node wasn't selected, your tokens will have been refunded to your unstaked bucket in the staking smart contract. When the next epoch begins, you can try committing tokens again in a future epoch to get a new spot.

Following is a detail explanation of these four steps. If you want to run multiple access nodes, you will have to run through these steps for each node.

Step 1 - Generate node information

Download the Bootstrapping Kit


_10
curl -sL -O storage.googleapis.com/flow-genesis-bootstrap/boot-tools.tar
_10
tar -xvf boot-tools.tar

CheckSHA256

_10
sha256sum ./boot-tools/bootstrap
_10
0d2cb2d0edb659880ae729e9d849a98535414af33690dad545d9892268167bcf ./boot-tools/bootstrap

If you have downloaded the bootstrapping kit previously, ensure the SHA256 hash for it still matches. If not, re-download to ensure you are using the most up-to-date version.

Generate Your Node Identity


_10
#########################################################
_10
# Generate Keys
_10
$ mkdir ./bootstrap
_10
# YOUR_NODE_ADDRESS: FQDN associated to your instance
_10
$ ./boot-tools/bootstrap key --address "<YOUR_NODE_ADDRESS_GOES_HERE>:3569" --role access -o ./bootstrap

Example

_25
$./boot-tools/bootstrap key --address "flowaccess.mycompany.com:3569" --role access -o ./bootstrap
_25
<nil> DBG will generate networking key
_25
<nil> INF generated networking key
_25
<nil> DBG will generate staking key
_25
<nil> INF generated staking key
_25
<nil> DBG will generate db encryption key
_25
<nil> INF generated db encryption key
_25
<nil> DBG assembling node information address=flowaccess.mycompany.com:3569
_25
<nil> DBG encoded public staking and network keys networkPubKey=f493a74704f6961ae7903e062ecd58d990672858eff99aece7bfbccf3aa02de8f1a624ecbf21a01e8b2f4a5854c231fbe218edd7762a34fea881f3958a215305 stakingPubKey=ae8dcf81f3a70d72036b7ba2c586ed37ed0eb82b9c0a4aab998a8420f98894f94c14f84fa716e93654d3940fc0c8ff4d19b504c90a5b4918b28f421e9d3659dc2b7e246025ebeffea0d83cceefe315d7ed346dbe412fdac51b64997d97d29f7e
_25
<nil> INF wrote file bootstrap/public-root-information/node-id
_25
<nil> INF wrote file bootstrap/private-root-information/private-node-info_e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5/node-info.priv.json
_25
<nil> INF wrote file bootstrap/private-root-information/private-node-info_e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5/secretsdb-key
_25
<nil> INF wrote file bootstrap/public-root-information/node-info.pub.e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5.json
_25
_25
$tree ./bootstrap/
_25
./bootstrap/
_25
├── private-root-information
_25
│ └── private-node-info_e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5
_25
│ ├── node-info.priv.json
_25
│ └── secretsdb-key
_25
└── public-root-information
_25
├── node-id
_25
└── node-info.pub.e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5.json
_25
_25
3 directories, 4 files

⚠️ Use a fully qualified domain name for the network address. Please also include the port number in the network address e.g. flowaccess.mycompany.com:3569

⚠️ Do not include in http:// in the network address.

If you would like to stake multiple access nodes, please ensure you generate a unique identity for each node.

Your node identity has now been generated. Your node ID can be found in the file ./bootstrap/public-root-information/node-id.

Example

_10
$cat ./bootstrap/public-root-information/node-id
_10
e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5

All your private keys should be in the bootstrap folder created earlier. Please take a back up of the entire folder.

Step 2 - Stake the node

You need to now register the node on chain by staking the node via Flow Port.

Here is a guide on how to use Flow port if you are not familiar with it. If you are staking via a custody provider or would like to directly submit a staking transaction instead follow this guide.

Fund you Flow account with at least 100.01 FLOW tokens, which covers the required stake plus the storage deposit.

On Flow port, choose Stake and Delegate -> Start Staking or Stake Again and then choose Access node as the option.

choose_access_flowport

On the next screen, provide the node details of you node.

Those node details (Node ID, Network Address, Networking Key and Staking Key) can be found in the file: ./bootstrap/public-root-information/node-info.pub.<node-id>.json.

Example

_10
$cat ./bootstrap/public-root-information/node-info.pub. e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5.json
_10
{
_10
"Role": "access",
_10
"Address": "flowaccess.mycompany.com:3569",
_10
"NodeID": "e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5",
_10
"Weight": 0,
_10
"NetworkPubKey": "f493a74704f6961ae7903e062ecd58d990672858eff99aece7bfbccf3aa02de8f1a624ecbf21a01e8b2f4a5854c231fbe218edd7762a34fea881f3958a215305",
_10
"StakingPubKey": "ae8dcf81f3a70d72036b7ba2c586ed37ed0eb82b9c0a4aab998a8420f98894f94c14f84fa716e93654d3940fc0c8ff4d19b504c90a5b4918b28f421e9d3659dc2b7e246025ebeffea0d83cceefe315d7ed346dbe412fdac51b64997d97d29f7e"
_10
}

Example

node_details_permissionless_an

On the next screen, ensure that you stake 100 FLOW token.

Example

transaction_register_node_permissionless_an

Submit the Transaction.

Step 3 - Verify that your node ID was selected

On Wednesday at around 12:00 UTC, the staking auction for the current epoch will end and five nodes from candidate list of nodes will be chosen at random by the staking contract to be part of the next epoch.

Note: If all 5 slots have been taken from the previous epoch, then no new access nodes will be chosen (see #limitations)

There are several ways to verify whether your node was chosen as explained below.

When you stake the node, the tokens will show up under the tokensCommitted bucket. After the staking auction ends, if the node is selected, the tokens remain in the tokensCommitted bucket and are moved to the tokensStaked bucket at the end of the epoch. If the node is not selected, the tokens are moved to the tokensUnstaked bucket.

Check using Flow Port

You can check these balances on Flow Port before and after the epoch transition that will occur on Wednesday (see timing).

When you stake the node, you should see the following on Flow Port under Stake & Delegate

Staked_node

After the epoch transition, if you see you token balance under the Staked Amount then your node got chosen.

Staked_node

Instead, if you see that your token balance is under the Unstaked Amount, then your node did not get chosen.

Unstaked_node

Check using FlowCli

You can also check these balance using Flow Cli. Once you have downloaded and installed Flow Cli, you can query the account balance using the command,


_10
flow accounts staking-info <your account address> -n mainnet

For Example, the following node was chosen as Tokens staked is 100.

Example

_16
$ flow accounts staking-info 0xefdfb20806315bfa -n testnet
_16
_16
Account staking info:
_16
ID: "e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5"
_16
Initial Weight: 100
_16
Networking Address: "flowaccess.mycompany.com:3569"
_16
Networking Key: "f493a74704f6961ae7903e062ecd58d990672858eff99aece7bfbccf3aa02de8f1a624ecbf21a01e8b2f4a5854c231fbe218edd7762a34fea881f3958a215305"
_16
Role: 5
_16
Staking Key: "ae8dcf81f3a70d72036b7ba2c586ed37ed0eb82b9c0a4aab998a8420f98894f94c14f84fa716e93654d3940fc0c8ff4d19b504c90a5b4918b28f421e9d3659dc2b7e246025ebeffea0d83cceefe315d7ed346dbe412fdac51b64997d97d29f7e"
_16
Tokens Committed: 0.00000000
_16
Tokens To Unstake: 100.00000000
_16
Tokens Rewarded: 0.00000000
_16
Tokens Staked: 100.00000000
_16
Tokens Unstaked: 0.00000000
_16
Tokens Unstaking: 0.00000000
_16
Node Total Stake (including delegators): 0.00000000

Epoch setup event

Alternatively, if you can monitor events, look for the epoch setup event that gets emitted by the epoch contract. That event is emitted at the end of epoch N's staking auction and contains a list of node IDs that are confirmed for the next epoch.

Step 4 - Start your node

If your node was selected as part of Step 3, you can now start your node.

First you'll need to provision a machine or virtual machine to run your node software. Please see follow the node-provisioning guide for it.

The access node can be run as a docker container with the following command by replacing nodeid with your node ID and adjusting the other parameters as per your setup.


_19
docker run --rm \
_19
-v $PWD/bootstrap:/bootstrap:ro \
_19
-v $PWD/data:/data:rw \
_19
--name flow-go \
_19
--network host \
_19
gcr.io/flow-container-registry/access:v0.29.8 \
_19
--nodeid=[Insert you NODE ID here] \
_19
--bootstrapdir=/bootstrap \
_19
--datadir=/data/protocol \
_19
--secretsdir=/data/secrets \
_19
--rpc-addr=0.0.0.0:9000 \
_19
--http-addr=0.0.0.0:8000 \
_19
--rest-addr=0.0.0.0:80 \
_19
--rpc-metrics-enabled=true \
_19
--bind 0.0.0.0:3569 \
_19
--dynamic-startup-access-address=secure.mainnet.nodes.onflow.org:9001 \
_19
--dynamic-startup-access-publickey=28a0d9edd0de3f15866dfe4aea1560c4504fe313fc6ca3f63a63e4f98d0e295144692a58ebe7f7894349198613f65b2d960abf99ec2625e247b1c78ba5bf2eae \
_19
--dynamic-startup-epoch-phase=EpochPhaseStaking \
_19
--loglevel=error

Example, for the Node ID e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5, the docker command be the following,

Example

_19
docker run --rm \
_19
-v $PWD/bootstrap:/bootstrap:ro \
_19
-v $PWD/data:/data:rw \
_19
--name flow-go \
_19
--network host \
_19
gcr.io/flow-container-registry/access:v0.29.8 \
_19
--nodeid=e737ec6efbd26ef43bf676911cdc5a11ba15fc6562d05413e6589fccdd6c06d5 \
_19
--bootstrapdir=/bootstrap \
_19
--datadir=/data/protocol \
_19
--secretsdir=/data/secrets \
_19
--rpc-addr=0.0.0.0:9000 \
_19
--http-addr=0.0.0.0:8000 \
_19
--rest-addr=0.0.0.0:80 \
_19
--rpc-metrics-enabled=true \
_19
--bind 0.0.0.0:3569 \
_19
--dynamic-startup-access-address=secure.mainnet.nodes.onflow.org:9001 \
_19
--dynamic-startup-access-publickey=28a0d9edd0de3f15866dfe4aea1560c4504fe313fc6ca3f63a63e4f98d0e295144692a58ebe7f7894349198613f65b2d960abf99ec2625e247b1c78ba5bf2eae \
_19
--dynamic-startup-epoch-phase=EpochPhaseStaking \
_19
--loglevel=error

For a more mature setup, it is recommended that you run the container using systemd as described here

🚀 The access node should now be up and running, and you should be able to query the node using Flow CLI or curl,

Example

_10
flow blocks get latest --host localhost:9000

Example

_10
curl http://localhost/v1/blocks?height=sealed

Monitoring and Metrics

The node publishes several Prometheus metrics. See Monitoring Node Health to setup node monitoring.

Node Status

The metrics for the node should be able to provide a good overview of the status of the node. If we want to get a quick snapshot of the status of the node, and if it's properly participating in the network, you can check the consensus_compliance_finalized_height or consensus_compliance_sealed_height metric, and ensure that it is not zero and strictly increasing.


_10
curl localhost:8080/metrics | grep consensus_compliance_sealed_height
_10
_10
# HELP consensus_compliance_sealed_height the last sealed height
_10
# TYPE consensus_compliance_sealed_height gauge
_10
consensus_compliance_sealed_height 1.132054e+06

FAQs

Will the access node receive rewards?

No, the access nodes do not receive any rewards.

Why is there a 100 FLOW token minimum?

As mentioned in the FLIP, the minimum is required to prevent certain vulnerabilities in the smart contract that are a result of having a zero minimum stake requirement.

Can the Access node be unstaked?

Yes, like any other staked node, the Access node can be unstaked. The staked tokens will be moved to the unstaked bucket in the subsequent epoch.

How to see all the access nodes that have staked?

When the nodes are initially staked, they are all added to the candidate list of nodes before the end of the epoch staking phase. The list can be retrieved from the chain by executing the get_candidate_nodes script which returns the candidate list for the current epoch.


_10
$ flow scripts execute ./transactions/idTableStaking/scripts/get_candidate_nodes.cdc -n mainnet

How to check the availability of open access nodes slots for the next epoch?

The limits for the open slots are defined in the staking contract and can be queried from the chain by executing the get_slot_limits script.

Node types are defined here


_10
_10
$ flow scripts execute ./transactions/idTableStaking/scripts/get_slot_limits.cdc --args-json '[{ "type":"UInt8", "value":"5"}]' -n mainnet
_10
Result: 118

Currently, there are 113 access nodes already part of the network. Hence, the total number of new nodes that can join are 118 - 113 = 5.