Key Takeaways:

• When thinking about earning ETH rewards, the focus should be on a risk-adjusted basis. Risk-adjusted staking rewards are the net amount received after taking into account for slashing, smart contract failure, validator downtime, etc.
• Metrics such as the Staking Rewards Rate (SRR), attestation effectiveness, participation, etc should not be the primary tool to choose staking service providers; these metrics are better used for sanity checks.
• Past performance represents one way to assess staking service providers, and it almost certainly does not imply future performance (compare pre- and post-Merge metrics as an example).
• When using any metric it is important to understand what it measures and exercise caution in the metric’s implication – for instance, validator effectiveness only has a loose association with a validator’s total rewards.
• Standalone metrics miss a lot of vital qualitative information; slashing protection and safeguards, as well as coverage is not something that shows up on chain
• Data can also be subject to survivorship bias – broadly, the concept that better-performing validators are captured in any analysis and poorer-performing ones are excluded from the sample (as they no longer exist). For instance, analyzing performance of all currently active validators will exclude most, if not all, slashed validators. Excluding slashed validators will tend to bias performance upwards.

Performance Misconceptions

Misconception 1: “I looked on xyz website and staking service provider A had a Rewards Rate of 7%, while staking service provider B had a Rewards Rate of 6.75%; all else equal I am going with provider A.”

This kind of thinking has an appeal but there are pitfalls. Beyond the need to control for luck, there are several other considerations with such a simplistic comparison:

• Are the comparisons being made on an equal basis? Many public sources that report on staking service provider performance are not providing a full picture – often reporting on a subset of validators (sometimes a minority of the staking service providers’ total).
• A validator that optimizes for rewards at the expense of everything else could be taking significant risks to do so. For instance, many slashing incidents that have been experienced on Ethereum are related to failover processes, whereby two instances of the same validator are online at the same time, causing equivocation (double-signing). To avoid an outcome like this, it is often better to accept some downtime. 

The points above assume that luck has been controlled, but it is worth emphasizing just how important this first step is. This is particularly the case with EL rewards, which are very volatile and introduce a second layer of luck. Validator performance will be especially high if a validator is chosen for block proposal (a random process)during a period of heightened blockspace demand, in which case EL rewards can be an order of magnitude higher than usual.

Misconception 2: “ABC’s effectiveness metric for provider A is greater than provider B; therefore, all else equal, I will choose provider A.”

Typically effectiveness ratings seek to represent a validator’s performance in a single number. This is a difficult task and has become more difficult with some of Ethereum’s upgrades, such as Altair and the Merge. For example, given EL rewards’ outsized and volatile influence on total rewards, many traditional metrics have become less relevant in predicting rewards. Not only that, but the best approach to capture these new rewards is not clear. For instance, see some of these notes (one, two, three) from Rated’s documentation.

Misconception 3: “I want a provider connected to as many relays as possible to maximize rewards.”

Relays, through MEV-Boost, allow validators to effectively sell blockspace to builders (who often include MEV bundles in blocks) [see here]. The reward maximizing behavior here is to simply connect to every relay possible.

Technically, this is not necessarily a misconception (ignoring the upward limit on relays from MEV-Boost). The misconception here is the belief that a staker should seek a provider that connects to as many relays as possible without considering the risk. Although the regulatory landscape differs for each staker and there is still a lack of clarity, haphazardly connecting to any and every relay likely heightens regulatory risk to the staker. Not to mention the fact that carelessly connecting to additional relays could lead to missed rewards should any of those relays experience a glitch.

Misconception 4: “Less than maximum rewards implies less than perfect validator performance.”

To achieve the maximum reward rate a few things need to line up – the validator needs to fulfill its duties correctly, the entire network of validators needs to be participating during that time and the validator’s duties need to occur in a timely fashion – part of this is reliant on the validator, the other part relies on other validators. An attesting validator needs to rely on a proposing validator to include their attestations in a block. Also the attesting validator’s rewards are proportional to the percentage of network participation (see here for more, specifically, see the four reasons for reward divergence below the pie chart). 

Misconception 5: “I compared one validator from operator A to one validator from operator B – operator A’s Reward Rate is higher than validator B, therefore, operator A is better.”

This is an even weaker comparison than that described in Misconception 1 – choosing one validator to represent an operator’s performance is extremely misleading and a useless exercise. Choosing two validators at random and comparing their performance all but guarantees they will differ significantly on at least a few factors, not the least of which is “luck” – that is, which duties they were selected for during the comparison period.

Some considerations:

• What events have occurred during the comparison window? Unless both validators have had nearly identical experiences in terms of being selected to propose blocks, participate in sync committee, etc – the comparison is meaningless.
• If both validators have had identical experiences in terms of duty assignments, is the difference in performance meaningful?
  • For instance, should a staker choose a validator with a Rewards Rate that is 0.25% higher if that operator does not have slashing protection in place?

Misconception 6: Use of slashing, penalties and downtime interchangeably.

On Ethereum slashing carries a very specific meaning (and very specific penalties) – equivocation, or voting for two different things at the same time; specifically:

• Either proposing or attesting to two different blocks at the same height, or
• Surround vote – effectively voting for two different canonical chains at the same time

(see here for more explanation and the slashing conditions as they exist in the specs for attestations and for proposals)

Additionally, validators are penalized for failing to fulfill specific validator duties; for instance, failing to attest correctly in a timely manner carries a penalty. In other words, the computer that is running a validator does not have to be offline or down to incur penalties.

So what is the Answer?

The most honest answer is that nothing replaces a deep understanding of how rewards and penalties work on Ethereum. Widely available data and metrics must be treated with scrutiny especially those without documentation. 

At a minimum, breaking down a validator’s performance into the most basic parts and comparing it to the network is a reasonable first step. Generally, a larger sample size and longer time periods of analysis will help to reduce the problem of randomness and luck.

Ethereum, as most networks within Web3, is a nuanced one. Having clear expectations about your goals as a staker including which risks you can tolerate and which you will not is vital.  

This last step is important as it helps you to define what your best risk-adjusted reward rate could be, which will likely differ from other stakers. We will be writing more about this in a future piece!