Running a Solana Validator — Lessons & Tips
Disclaimer: We run a Solana validator ourselves. If you find this article helpful and would like to support us by staking our validator you can find the details at the bottom of this page.
Solana is a high-throughput blockchain that is relatively new on the block, it’s been around for around 18 months and has captured the imagination of blockchain enthusiasts around the world. Solana was created by Anatoly Yakovenko.
Recently the price of the chain’s native token, SOL, has spiked, leading to an increased interest in the chain and in particular how to run a validator on this Proof of Stake network.
Here’s a breakdown of some of the important points from our own experience, having run a validator on Testnet since early May 2021 and on Mainnet snice August 2021.
TL/DR: You need good hardware, probably cannot run it from home, and it costs a lot of SOL!
What is validation
Solana uses a delegated Proof of Stake protocol, this means that anyone who owns SOL can delegate some or all their SOL to a validator, whereby the validator then earns “weight” on the network which leads to them being assigned as a leader for more slots, as well as earning more rewards for voting.
When you delegate your SOL you retain full control and can undelegate and withdraw at any point, regardless of the validator’s performance or status.
Solana works in epochs, which are 2–3 days long, and every epoch consists of 420,000 blocks, with a targeted block time of 400ms. A validator must vote on every single block, so validators are incredibly busy. They earn rewards for how many of their votes are successful (since a slight delay can cause it to be invalid).
The rewards they earn are a factor of the percentage success in voting as well as the stake they have relative to the rest of the network. I.e. a validator who holds 1% of the network’s stake and lands every vote will get 1% of rewards. If they only land 80% of votes (current best is 83% I believe) they get potentially less, depending on everyone else’s percentages (each successful vote is a point, your points get tallied with all other validators then weighted by your stake). Link to Solana Docs on this topic
What hardware do you require
You will need a very good server. Solana validation is a high throughput intensive process which depends a lot on CPU and disk speed. AMD Threadripper and AMD Epyc are the most popular CPU choices.
You cannot keep up with the network on a CPU with less than 2.8GHz, ideally you’d want 3GHz+. Some of AMD’s Epycs don’t meet this requirement and people get stung trying to run those.
Additionally you’ll need 256GB of RAM as well as NVMe drives. Don’t bother with regular SSD drives or anything slower. Even with NVMe drives the TBW matters, speed is everything.
Your network should consist of at least 1gbps symmetric (up + down) and be unshaped and unmetered.
We work with our hosting partner Latitude who provide custom build servers that meet Solana’s requirements. We currently operate an AMD Epyc 7443P with 512GB RAM and 2x 3.92TB NVMes (+250GB SD for OS) and 10gbps network. We recommend choosing a reputable hosting provider that can meet those requirements, which not all can.
The Validator comprises broadly three components:
- the executable, config, keys
- the ledger, this is the recent blockchain history (not the complete blockchain, as this is too big to be stored on the drives)
- the accounts database
The accounts database is written to and from frequently and it is generally considered best practice to store it in a RAMDISK, meaning it is stored in your volatile memory not your NVMe drives.
The ledger is stored on the NVMe drives as it requires several hundred GB, best to have it on it’s own drive separate from the OS for better performance
The validator sends & receives several hundred mbps, so you need a very good internet connection, ideally 1gbps, although 500mbps+ could work if it’s reliable. In addition it consumes upwards of 50TB a month so ensure you aren’t capped and your line is unshaped. For this reason running behind a NAT or anywhere other than a professional data centre is not advised.
Resources
Solana Official Validator Requirements
Solana Official Validator Startup Guide (this is very useful and provides almost everything you need)
Unofficial guide on starting a devnet validator (very useful especially for the systemd files, but make sure to adjust to your use case)
Official Solana Discord community (this is where most validators frequently hang out and chat, as well as other parts of the community, notably developers)
Unofficial Profit Calculator (created by Shinobi Systems, they’re usually ranked no 1 on stakeview.app and very engaged in the community)
Latitude (web3 hosting provider that can provide suitable servers for Solana’s needs)
Costs
In addition to the hardware and networking costs you need to pay vote fees. These add up to around 1 SOL per day (in our observation currently they’re around 0.75 SOL per day but plan for 1 SOL per day).
Validators are currently running on v1.6.22 of the Solana program (as of 1 Sept 2021), with the plan to upgrade to 1.7 in the coming weeks. There are plans to reduce voting fees from v1.9, however this will likely not happen before early next year and we don’t know how big of a reduction it will be.
General tips & tricks
- Don’t even think about running on anything other than Ubuntu 20.04 LTS
- Run your accounts in RAMDISK
- the solana-validator.log file is your best friend in identifying early issues
- There is a system tuner program and service with Solana, not running this will cause your validator to fail and restart, run it manually first and set it up as a systemd process (see guides in resources above) that is required by the main solana process
- Monitor your disk usage
- Set up logrotate on your log file or it will consume your disk space
- Set your CPU governor to performance mode
- Publish validator info using solana validator-info publish
- You need an identity and a vote account to run a validator, use solana-keygen grind to get a custom beginning phrase on your key. Your identity key is the one that needs to have funds to pay for voting fees
- You need to open ports 8000–10000 for TCP and UDP (you can get away with a smaller range, you can use dynamic-port-range flag in the validator startup script to specify
- Monitor, monitor and monitor. Use validators.app and the integrated metrics to monitor your validator.
- Once you’re running use solana catchup validator_keypair.json to check if you’re catching up to the network. If your validator is falling behind as opposed to gaining your system is too slow.
- Startup takes 10–20 minutes before it starts catching up, for us we’re usually around 1700 slots behind by then and it takes 10–15 minutes to catch up.
- You need around 5k SOL self stakes or 50k SOL third party stake at 10% commission to break even, be ready to market yourself
I hope the above is helpful to anyone considering setting up a Solana validator. The network depends on good quality, reliable validators that are spread over a large decentralised area.
When staking a validator consider checking solanabeach.io to see the “halt line”, the validators above can collectively stop the network as they control 33% of active stake. Ideally choose a validator below this line to delegate to.
Please support our validator if you’re looking to stake somewhere
If you liked this article and would like to support our validator please consider staking with us, we’re committed to the Solana ecosystem and are trying to bring value to others through our website Stakewiz, these articles on Medium as well as active participation on Discord, Twitter and Reddit. We’re in a low-concentration data centre, have 2% commission and are one of the highest performing validators in terms of APY.
Our vote key is GE6atKoWiQ2pt3zL7N13pjNHjdLVys8LinG8qeJLcAiL or you can search for “Laine” in the validator list in your wallet, or see more info on https://laine.co.za/solana
