Performance is one piece of a larger whole. Staking service providers are not commodities – they are not the same, nor are they interchangeable. Stakers need to evaluate aspects such as security, reporting, coverage, access (to research analysts, customer service, technical support etc), data, wallet integrations, APIs, as well as performance.

In seeking to understand performance, the general idea is to control for luck and make sure that a comparison is being made on the same basis. 

At a minimum, a comparison should: 

• Include all relevant validators being run by the operator; 
• Have a reasonable comparison group (for instance all validators on the Ethereum network)
• Control for luck (even though a large sample size will help to decrease the luck factor, it will not eliminate it necessarily).

A practical way to achieve this is to get a sample of all of an operator’s validators and then examine each duty (attestations, proposals, sync committee participation, etc) separately and compare it to the average (and median) validator on the network. Here are some examples of these kinds of metrics:

• Median and average CL and EL rewards for operator when they propose vs network [both calculated as a total, i.e., total rewards = CL + EL rewards; and each piece calculated separately]
  • The median should be favored for EL rewards comparisons given their volatility and skewness
• Slashing events experienced by the operator vs network
• % of operator’s successful TIMELY_HEAD votes vs network average
• % of operator’s successful TIMELY_SOURCE votes vs network average
• % of operator’s successful TIMELY_TARGET votes vs network average
• % of missed proposals vs network
• Average rewards for sync committee performance vs network average

In addition to the questions above, any useful comparison needs to control for outlier time periods, either by removing them or ensuring similar coverage in both samples. For instance, periods of significantly higher EL rewards need to be controlled for; see November 7th, 2022 to November 13th, 2022 as an example (see first chart in ‘EL Rewards’ section below).

To properly assess the importance of the above metrics, a thorough understanding of rewards and penalties on Ethereum is necessary, which is provided in the next two sections.

Ethereum Rewards

Ethereum validators are compensated for performing consensus-related duties. These duties can be broken down into two broad categories – attesting to the state of Ethereum and proposing new blocks to be added to the chain. For performing these duties correctly and in a timely manner, Ethereum pays newly issued Ether to its validators. We refer to these rewards as Consensus Layer (CL) rewards.

Importantly, although rare, duties such as proposing a block or participating in the sync committee (a somewhat different form of attesting to Ethereum’s state [see here]) carry higher rewards, given their lower frequency, and typically make up a smaller percentage of CL rewards than attestations.

A validator can expect to attest, on average, every 6 minutes and 24 seconds; a validator is assigned to one slot per epoch to attest, there are 32 slots per epoch and each slot takes 12 seconds. So in a day a validator can expect to attest 225 times, which is the number of epochs in a day.

Validators are chosen at random to propose a block at a specific slot. There are 225 epochs and 32 slots per epoch – that is, 7200 opportunities for a validator to be chosen to propose a block in a given day. Given that there are currently 852,782 validators, the chance of an individual validator being chosen to propose is 7200/852782 ⋍ 0.84% per day. This points to the fact that attestations are much more frequent and form a larger proportion of CL rewards than proposals.

On the rare occasion when a validator is selected to propose a block, they also receive Execution Layer (EL) rewards. These represent payments from Ethereum users seeking to expedite their transactions, i.e., priority fees or tips and (potentially) MEV-related rewards. These EL rewards add greater weight to the importance of proposing blocks and add volatility to total rewards.

In other words, a validator’s performance captures not just its luck in being selected to propose, but also the luck of being chosen to propose during heightened demand for blockspace (and therefore higher EL rewards).

To make this a bit more tangible, imagine you are comparing two validators, trying to understand which has better performance. You view the last 30 days’ worth of performance and you see validator A was in the top decile, while validator B was in the bottom decile.

Naturally, you choose validator A. But the next month validator B outperforms validator A and is now in the top decile and validator A is now in the bottom decile.

This chart shows five separate real validators that went from the top decile, in terms of rewards rate, (green bar) in one month to the bottom decile (yellow bar) in the next. The vertical axis is the staking rewards rate for the month period:

The above is an extreme and exaggerated example, but meant to represent just how great the swing in rewards can be, i.e., how much luck can be involved. Over two different time periods, the same validator received a drastically different amount of rewards. 

Ethereum Rewards – One Level Deeper

The base reward is the key unit to consider when understanding rewards on Ethereum. The maximum possible rewards achievable by a validator are proportional to the base reward. There are three broad activities a validator can engage in to earn rewards: propose, attest and participate in the sync committee. Attesting can be further broken down into three pieces: TIMELY_SOURCE, TIMELY_TARGET and TIMELY_HEAD. Each of these five activities has an associated weight, or share of the base reward:

(source)

Attesting

There are 32 slots per epoch – validators are randomly assigned to one of these slots to attest. Attesting means that a validator’s consensus client (i.e., a computer program) reports its view of the state of Ethereum (i.e., it provides an attestation). This attestation contains three key pieces of information (among other data):

• the head of the chain (i.e., the block at the “top” of the chain)
• the source checkpoint; and
• the target checkpoint.

A validator needs to get the above correct and their attestation needs to be included in a timely manner. More specifically, the validator has:

1 slot to get their vote for head of the chain included (i.e., TIMELY_HEAD),

~5 slots to get source checkpoint included (i.e., TIMELY_SOURCE), and

32 slots to get target checkpoint included (i.e., TIMELY_TARGET)

(as mentioned above an epoch is divided into 32 slots, and each slot is 12 seconds).

Getting a vote included requires “effort” from two parties – the validator making the attestation and a validator proposing a block. This is the primary collaboration between attesting validators and proposing validators  – the validator attesting needs to rely not only on their own performance, but that of the validator that is proposing a block and includes their attestation in a block.

There is another aspect in which attestation rewards depend on others in the network. There is a modification made for network participation. The result is that the lower the participation rate, the lower the rewards received by the attesting validator. To see this, readers are encouraged to review get_flag_index_deltas in the Ethereum specs (Altair changes); more specifically see lines 17 and 18.

Importantly failing to correctly attest in a timely manner results in penalties for the duties TIMELY_SOURCE and TIMELY_TARGET; failing to attest to TIMELY_HEAD is excluded from penalties. This is why when looking at a validator’s performance TIMLEY_SOURCE and TIMELY_TARGET should have somewhat of a higher priority.

Sync Committee

A group of 512 validators is selected to be part of the sync committee – a 256-epoch period (~27.3 hours) during which validators in the committee are expected to sign the block header in every slot. There is always an active sync committee, but its membership changes every 256 epochs. Sync committee participation is much more active – requiring signatures every slot from a validator – compared to attesting (above), which requires a similar behavior, but only once per epoch.

The rewards (and penalties) of sync committee participation are elevated compared to attesting. This may seem counterintuitive, given that the share of rewards is much smaller (2/64 vs 54/64) for the three attestation duties), but it is important to bear in mind that this much smaller share is split amongst 512 validators vs 852,782+ validators in the case of attestation rewards.

A validator failing to participate in a particular slot receives a penalty close to what the reward would have been.

Being chosen to be part of the sync committee has a significant impact on rewards and any rewards rate calculations including them. Given that there are only 512 validators in the sync committee and that there are currently around 852,781 validators – a single validator being chosen to be part of the sync committee is expected to happen once every 5.2 years or 1895 days (roughly).

Participation in the sync committee can significantly skew validator performance numbers – see this validator’s performance over a given one-month period, for instance:

You can see the spike in daily income associated with the sync committee around May 28th. You can see the per epoch effect as well (see epochs 204,539 to 204,543 vs epochs 204,544 and 204,545):

The result is an abnormally high estimated APR for the 7-day window and to a lesser extent the 31-day window:

Finally, note the “Luck” factor (explained in greater detail below under the heading ‘EL Rewards’) in the above table – 100% is considered normal or expected.

Proposing

When a validator is chosen to propose it does two important things – it creates a block (or receives one from MEV-Boost) and proposes it and it includes attestations and signatures (from sync committee) in that block. With respect to CL rewards – a proposer’s rewards come solely from attestations and signatures they are able to successfully include in the block they are proposing.

The rewards proposers receive for including signatures and attestations are ⅛ of the reward received by the validator doing the attesting or providing the signature. Again, even though the proposer receives a smaller “slice of the pie” (see first chart in ‘Ethereum Rewards – One Level Deeper’ section above), they are sharing those rewards with fewer validators – a maximum of 32 per epoch.

Finally, unlike two of the attestation duties (TIMELY_SOURCE and TIMELY_TARGET) and sync committee participation, failing to propose does not carry an explicit penalty, though the importance of the missed opportunity should not be underestimated.

EL Rewards

The second form of rewards for proposers comes from priority fees (or tips) and (potentially) MEV-related rewards – Execution Layer (EL) rewards. These rewards are typically an order of magnitude (often 3 to 4x) greater than CL rewards for proposing. That said, there is tremendous volatility in EL rewards that tends to vary with demand for block space.

When looking at validator performance there are two ways in which a validator can be considered “lucky.” Firstly, are they being selected to propose or be part of the sync committee more often than the odds would suggest? Secondly, are they being selected to propose during particularly lucrative times (i.e., higher demand for blockspace)? For instance, notice the average EL rewards during November 2022 – around the FTX crisis (the weekly amount of 16.3K ETH paid out):

(Source)

All of these factors can come together to make some validators exceptionally lucky. Currently (as at the time of writing) this is the top performing validator:

(source)

Notice the APR for the 7- and 31-day period as well as the luck factors:

This validator deposited on the Beacon Chain (i.e., CL) a little over a month ago (as at the time of writing) – since then they were selected to be part of the sync committee twice and they were chosen to propose a block.They also received EL rewards that were over 10 times the CL rewards for proposing the block.

None of the validator activities described above are mutually exclusive. A validator that is part of the sync committee will still be chosen to attest once per epoch. Likewise, that validator could also be chosen to propose a block during the same time.

Rewards for Slashing

There is one final way in which a validator can earn rewards that is not captured above – as a proposer, a validator including evidence of a slashable offense earns rewards proportional to the slashing penalty corresponding to that offense. 

Two things are important to keep in mind about this kind of reward. First, it is an unlikely occurrence – at the time of writing there had been only 279 slashing incidents since the launch of Beacon Chain at the end of 2020. Secondly, unlike CL rewards, newly minted Ether is not issued to pay rewards for slashing – instead, they are taken from the slashed validator.

The magnitude of rewards for a proposer including evidence of a slashable offense is around 70,312,500 gwei (or 0.0703125 ETH) per validator, assuming the slashed validator has an effective balance of 32 ETH. In other words, this figure provides an approximate maximum amount for a proposer per slashed validator (a proposer can provide evidence of multiple validators committing slashable offenses).

In Conclusion

Ethereum’s intricate system of validator incentives and disincentives reveals the complexity behind staking rewards for the network. While attesting makes up the bulk of typical rewards, events like proposing blocks and capturing MEV can create unpredictable–and significant–opportunities.

In assessing and comparing validator returns, it’s critical to analyze rewards holistically over longer time horizons. As we’ve demonstrated, outsized rewards from singular lucky events can skew perceptions of consistent underlying performance. A comprehensive analysis should examine both periodic “jackpot” rewards and consistent base rewards. This full picture showcases a validator’s true capabilities and performance beyond short-term luck. 

In summary, analyzing validator performance requires examining the spectrum of rewards, duties, and penalties defining Ethereum staking incentives. While lucky wins can grab attention, sustained excellence on core responsibilities like attesting separates dependable validators from the rest. By embracing these complex dynamics, stakers can identify the best staking-as-a-service provider for generating rewards through all market cycles.