Considerations

In the course of Avalanche L1 operation, you will inevitably need to upgrade or change some part of the software stack that is running your Avalanche L1. If nothing else, you will have to upgrade the AvalancheGo node client. Same goes for the VM plugin binary that is used to run the blockchain on your Avalanche L1, which is most likely the Subnet-EVM, the Avalanche L1 implementation of the Ethereum virtual machine.

Node and VM upgrades usually don't change the way your Avalanche L1 functions, instead they keep your Avalanche L1 in sync with the rest of the network, bringing security, performance and feature upgrades. Most upgrades are optional, but all of them are recommended, and you should make optional upgrades part of your routine Avalanche L1 maintenance. Some upgrades will be mandatory, and those will be clearly communicated as such ahead of time, you need to pay special attention to those.

Besides the upgrades due to new releases, you also may want to change the configuration of the VM, to alter the way Avalanche L1 runs, for various business or operational needs. These upgrades are solely the purview of your team, and you have complete control over the timing of their roll out. Any such change represents a network upgrade and needs to be carefully planned and executed.

This tutorial will guide you through the process of doing various Avalanche L1 upgrades and changes. We will point out things to watch out for and precautions you need to be mindful about.

General Upgrade Considerations

When operating an Avalanche L1, you should always keep in mind that Proof of Stake networks like Avalanche can only make progress if sufficient amount of validating nodes are connected and processing transactions. Each validator on an Avalanche L1 is assigned a certain weight, which is a numerical value representing the significance of the node in consensus decisions. On the Primary Network, weight is equal to the amount of AVAX staked on the node. On an L1, the validator manager contract sets each validator's weight. The P-Chain records it from the Warp message in RegisterL1ValidatorTx, and SetL1ValidatorWeightTx changes it later.

Avalanche L1s can operate normally only if validators representing 80% or more of the cumulative validator weight is connected. If the amount of connected stake falls close to or below 80%, Avalanche L1 performance (time to finality) will suffer, and ultimately the Avalanche L1 will halt (stop processing transactions).

You as an Avalanche L1 operator need to ensure that whatever you do, at least 80% of the validators' cumulative weight is connected and working at all times.

Upgrading Avalanche L1 Validator Nodes

AvalancheGo, the node client that is running the Avalanche validators is under constant and rapid development. New versions come out several times each year, bringing added capabilities, performance improvements or security fixes. Updates are usually optional, but from time to time (much less frequently than regular updates) there will be an update that includes a mandatory network upgrade. Those upgrades are MANDATORY for every node running the Avalanche L1. Any node that does not perform the update before the activation timestamp will immediately stop working when the upgrade activates.

That's why having a node upgrade strategy is absolutely vital, and you should always update to the latest AvalancheGo client immediately when it is made available.

For a general guide on upgrading AvalancheGo check out this tutorial. When upgrading Avalanche L1 nodes and keeping in mind the previous section, make sure to stagger node upgrades and start a new upgrade only once the previous node has successfully upgraded. Use the Health API to check that healthy value in the response is true on the upgraded node, and on another validator of the L1, call validators.getCurrentValidators on the L1 chain endpoint (/ext/bc/<blockchainID>/validators) and check that isConnected is true for the upgraded node's nodeID. platform.getCurrentValidators returns connected only for Primary Network validators. Once those two conditions are satisfied, node is confirmed to be online and validating the Avalanche L1 and you can start upgrading another node.

Continue the upgrade cycle until all the Avalanche L1 nodes are upgraded.

Upgrading Avalanche L1 VM Plugin Binaries

Besides the AvalancheGo client itself, new versions get released for the VM binaries that run the blockchains on the Avalanche L1. On most Avalanche L1s, that is the Subnet-EVM, so this tutorial will go through the steps for updating the subnet-evm binary. The update process will be similar for updating any VM plugin binary.

All the considerations for doing staggered node upgrades as discussed in previous section are valid for VM upgrades as well.

In the future, VM upgrades will be handled by the Avalanche-CLI tool, but for now we need to do it manually.

Go to the AvalancheGo releases page. Since v1.14.2, each AvalancheGo release includes the Subnet-EVM binaries. Use the Subnet-EVM archive from the same release as your AvalancheGo version, because the plugin version must match. Copy the link for the OS and architecture of the node machine (linux-amd64 = Linux on Intel/AMD, linux-arm64 = Linux on Arm, macos = Mac). Log into the machine where the node is running and download the archive, using wget and the link to the archive, like this:

wget https://github.com/ava-labs/avalanchego/releases/download/v1.15.1/subnet-evm-linux-amd64-v1.15.1.tar.gz

This will download the archive to the machine. Unpack it like this (use the correct filename, of course):

tar xvf subnet-evm-linux-amd64-v1.15.1.tar.gz

This unpacks the archive into the directory subnet-evm-v1.15.1. The file subnet-evm-v1.15.1/subnet-evm is the plugin binary. You need to stop the node now (if the node is running as a service, use sudo systemctl stop avalanchego command). Put that file in the node's plugin directory (--plugin-dir). The default is ~/.avalanchego/plugins, and the install script also uses ~/.avalanchego/plugins. Instead of the subnet-evm filename, VM binary needs to be named as the VM ID of the chain on the Avalanche L1. For example, for the WAGMI Avalanche L1 that VM ID is srEXiWaHuhNyGwPUi444Tu47ZEDwxTWrbQiuD7FmgSAQ6X7Dy. So, the command to copy the new plugin binary would look like:

cp subnet-evm-v1.15.1/subnet-evm ~/.avalanchego/plugins/srEXiWaHuhNyGwPUi444Tu47ZEDwxTWrbQiuD7FmgSAQ6X7Dy

After you do that, you can start the node back up (if running as service do sudo systemctl start avalanchego). You can monitor the log output on the node to check that everything is OK, or you can use the info.getNodeVersion() API to check the versions. Example output would look like:

{
  "jsonrpc": "2.0",
  "result": {
    "version": "avalanchego/1.15.1",
    "databaseVersion": "v1.4.5",
    "rpcProtocolVersion": "46",
    "gitCommit": "fdb2c1b9b02ef357b5867d5ababb6acc83837a01",
    "vmVersions": {
      "avm": "avalanchego/1.15.1",
      "evm": "v1.15.1",
      "platform": "avalanchego/1.15.1",
      "srEXiWaHuhNyGwPUi444Tu47ZEDwxTWrbQiuD7FmgSAQ6X7Dy": "v1.15.1"
    }
  },
  "id": 1
}

Note that the entry for the VM ID you upgraded says v1.15.1. Since v1.14.2, Subnet-EVM reports the AvalancheGo version that it ships with. You have successfully upgraded the VM!

Refer to the previous section on how to make sure node is healthy and connected before moving on to upgrading the next Avalanche L1 validator.

If you don't get the expected result, you can stop the AvalancheGo, examine and follow closely step-by-step of the above. You are free to remove files under ~/.avalanchego/plugins, however, you should keep in mind that removing files is to remove an existing VM binary. You must put the correct VM plugin in place before you restart AvalancheGo.

Network Upgrades

Sometimes you need to do a network upgrade to change the configured rules in the genesis under which the Chain operates. In regular EVM, network upgrades are a pretty involved process that includes deploying the new EVM binary, coordinating the timed upgrade and deploying changes to the nodes. But since Subnet-EVM v0.2.8, we introduced the long awaited feature to perform network upgrades by just using a few lines of JSON. Upgrades can consist of enabling/disabling particular precompiles, or changing their parameters. Currently available precompiles allow you to:

• Restrict Smart Contract Deployers
• Restrict Who Can Submit Transactions
• Mint Native Coins
• Configure Dynamic Fees
• Set Where Transaction Fees Go (Reward Manager)
• Send and Receive Avalanche Warp Messages (Warp Messenger)

Please refer to Customize an Avalanche L1 for a detailed discussion of possible precompile upgrade parameters.

Summary

Vital part of Avalanche L1 maintenance is performing timely upgrades at all levels of the software stack running your Avalanche L1. We hope this tutorial will give you enough information and context to allow you to do those upgrades with confidence and ease. If you have additional questions or any issues, please reach out to us on Discord.