Financial markets and the Wisdom of the Crowds
Read the publiation below or click here to open the PDF
Abstract
Market expectations play a central role in financial analysis. They often serve as a useful benchmark against which economic developments are compared. They also serve as useful guides for future economic and financial developments. Yet it is important to put those market expectations in proper context. After all, market expectations are based on the market price determined by the marginal trade. However, the projections of non-price-determining market participants loom in the background, which ultimately leads to uncertainty and volatility in the market price. Thus, the ‘market expectation’ is far from perfect foresight but remains arguably one of the better (efficient) sources of information available. Given this ‘imperfection’, however, it is important to monitor not only market expectations, but also the uncertainty surrounding those expectations. This note proposes entropy as a relevant measure of uncertainty.
Financial markets as a source of information
An important prediction of standard financial theory is that prices of financial assets reflect the processing and aggregation of all (publicly) available information. Indeed, the act of trading on specific information means this information is almost automatically reflected in the price of the asset (though often partially or possibly in a distorted manner). The intensity with which trading occurs (measured e.g., via volumes) determines the extent to which such information is incorporated into the price. Very aggressive positioning will move the price more toward the information than more moderate trading. Financial markets thus act, in a sense, as an information aggregator. All information on which the market trades is included in the price. Incidentally, this does not mean that financial markets reflect information fully and/or accurately always and everywhere. Risk premiums and possible psychological biases do distort price signals from the market.
This prediction of financial theory is proving to be one of the most impactful in practice. Analysts and market participants use financial markets as a crucial source of information. The focus here is specifically on futures and forward markets because they contain (aggregate) information regarding the expectations of marginal traders. Moreover, combinations of options and/or futures contracts allow one, based on assumptions around the degree of risk aversion, to filter out risk-neutral market expectations (and probabilities) from those market prices. Those risk-neutral expectations are set - in a strong application of the so-called Wisdom of the Crowd effect - as a benchmark in financial markets.1
Futures markets as Delphi's oracle for monetary policy
One of the most important applications of these principles is the monitoring and forecasting of monetary policy. Money markets of most major economies are well developed and liquid, and the wide range of traded futures and options contracts allows for the calculation of detailed (risk-neutral) interest rate forecasts and probabilities for future central bank policy meetings.
Figure 1 illustrates this exercise for the Fed. The figure shows the risk-neutral probabilities that the markets assign (on June 10) to the various possible interest rates that the Fed could set during the next 8 policy meetings (one per row). Based on this table, one can cautiously conclude that the probability assigned by the market (i.e. the marginal trade in that futures market) to very strong interest rate cuts over the coming months is slim. The expected interest rate remains hovering around 5% while the probability of only one interest rate cut in 2024 still stands at more than 40%. This interpretation, of course, hinges on the assumption of risk neutrality and thus is not necessarily a fully accurate reflection of the true underlying market probabilities. But "the market" seems to be looking through such caveats. It is common practice to equate these risk-neutral probabilities with those of the market.2
Entropy
Nevertheless, caution should be exercised when using these market expectations. Market expectations derived from future markets may give an indication about the expected policy interest rate for the upcoming policy meetings, but they do not reflect the uncertainty surrounding those expectations. Therefore, it is important to identify the associated uncertainty in addition to those expectations.
In this analysis, we use entropy as a measure of uncertainty around average market expectations. Entropy is a measure of disorder from thermodynamics that reflects the degree of turmoil in a system. In markets, higher values of entropy reflect greater uncertainty in the (probability) distribution of possible future interest rates. Like market expectations, that entropy can be calculated from risk-neutral probabilities.3
More formally, the market expectation and the entropy are calculated as follows, with p the probability and r the possible levels of interest rates:
Figures 2 and 3 show the market expectations and entropy for the policy interest rates to be decided at the June and September FOMC policy meetings, respectively. We focus on these two dates because they were seen as possible starting dates of the Fed's easing cycle. Market expectations for those June and September levels for policy rates are available as of mid-2023. Those market expectations have evolved strongly. While markets were expecting interest rate levels of around 5.25% for June 2024 and 5% for September 2024 as of the end of September 2023, those expectations fell sharply in a few months’ time. Powell's hints in November 2023 of an end to the tightening cycle pushed market expectations sharply lower. Late January, financial markets expected a rapid and deep easing of monetary policy and those expectations for policy rates stood at 4.5% for June 2024 and below 4% for September 2024. Still, following a series of disappointing inflation news in the US, expectations were revised upward with interest rates again now around 5.25% and 5% for those respective dates.
It is important to note, however, that those interest rate expectations are accompanied by still high uncertainty, fueled in part by persistent macro uncertainty. Figure 2 shows that uncertainty regarding the June interest rate level has long remained high. Only in recent months has this uncertainty started to diminish. Moreover, longer term expectations imply more uncertainty. The entropy for the September interest rate expectations is systematically above that for the June expectations (compare Figure 2 and Figure 3). Less obvious are the reasons for the various peaks in entropy for the September expectations, seen for example in February and again in late May. These indicate that the nervousness and uncertainty of the markets regarding September expectations appears not only high but also extremely volatile. In any case, this example illustrates the importance of supplementing market expectations with additional analysis around the uncertainty of those market expectations. Supplementing market expectations with an in-depth analysis of the uncertainty surrounding those expectations using the entropy measure (Figures 2 and 3) or directly on the risk-neutral probabilities (Figure 1) therefore seems appropriate.
Conclusie
Market expectations play a central role in financial analysis, but they are Delphic rather than perfect. After all, they are based on the market price determined by the marginal trade. Surrounding this trade, however, are the forecasts of non-price-setting market participants, which will ultimately lead to uncertainty and volatility in the market price. Thus, ‘market forecasts’ are not perfect foresight. But - despite their many biases - they are still arguably one of the better (efficient) sources of information available. Given the uncertainty, however, it is important to monitor not only market expectations, but also the uncertainty surrounding those expectations. An analysis based on the entropy measure suggests that this uncertainty for expectations of central banks’ policy rates is not only large, but also volatile. Thus, market expectations should be treated with a sound dose of skepticism and certainly do not deserve the status of a perfect truth.
1 The interpretation of these risk-neutral expectations as being the true market expectations is common in practice, but theoretically not entirely correct. Price signals are distorted by changing risk premia (or behavioral biases) and thus also distort the information value of the price signals themselves. Financial prices themselves do not allow exact identification of the cause of price movements (changing expectations or adjustments to risk premia or biases).
2 Such 'approximations' are not exceptional in the financial markets, e.g. also in the Black-Scholes option price formula, which is (not quite rightly) based on the normal distribution of the disturbance term. Such an approximation must of course remain 'reasonable,' which in practice is ensured by the 'threat' of arbitrage.
3 Details of the calculation method are available upon request