clean the ETF data and understand the market instruments.
build a strategic benchmark and the covariance estimates needed for portfolio risk.
introduce the Black-Litterman prior and define views.
build the view matrix, learn confidence from realized historical payoffs, and finally convert the posterior expected returns into portfolio weights.
repeat the same idea on sector ETFs through the library implementation.
1) mathematic background: Bayes Rule
Black-Litterman is easier to understand if we start with this topic. Imagine we have an initial belief about something uncertain. Then we observe new evidence. We shouldn’t throw away the initial belief completely, and we shouldn’t ignore the new evidence either. We should update the belief.
That’s the whole idea behind Bayesian updating. The prior is what we believed before the new information. The likelihood is how compatible the new information is with possible states of the world. The posterior is what we believe after combining both.
\[
P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}
\]
where \(P(A)\) is the prior (what we believed before seeing \(B\)), \(P(B \mid A)\) is the likelihood (how compatible the evidence \(B\) is with belief \(A\)), and \(P(A \mid B)\) is the posterior (our updated belief after seeing \(B\)). Written proportionally:
Or we can write it like this: \[
\mathrm{posterior\ belief}\propto\mathrm{prior\ belief}\times\mathrm{new\ evidence}
\]
Here we’re estimating the vector of expected returns across the assets in our portfolio. Each entry is the expected return for one asset and becomes an input to the allocation decision. Expected returns are hard to estimate, so we don’t want to trust historical averages too much. We can first define a benchmark that gives us a starting point or the prior. Then we use some views about the financial markets and economic situations that will have effect on expected returns. views give us new information. Then there is confidence for each view which decides how much the views should move us away from the benchmark.
So the process is: - start with a reasonable neutral belief - observe the new evidence - update the belief considering the confidence.
We build on portfolio optimization here, so I highly recommend reading notebook 2. Portfolio optimization with Mean-Variance models first to get familiar with the mean-variance inputs and constraints that we’ll use again when we build our Black-Litterman portfolios.
For this project, we could’ve used the same nasdaq data from notebook 2, but that wouldn’t be the best choice for Black-Litterman and the economic interpretation and building views wouldn’t be as good. So we use a dataset of famous ETFs (investment funds that trade on exchanges like stocks) which is collected from yahoo-finance API. A script for reproducing the dataset is available in data folder ([from here]). The dataset contains history of these ETFs from the start of 2010, but we use the data from the start of 2013 cause we don’t want to deal with missing prices of one of the ETFs. [from here]: https://github.com/ramtin-asadi/Quantitative-Finance-Lab/tree/main/data/core_cross_asset_etfs
Show code
raw = pd.read_csv("../data/core_cross_asset_etfs.csv")raw["date"] = pd.to_datetime(raw["date"], errors="coerce")raw = raw.dropna(subset=["date"]).sort_values("date")raw = raw.drop_duplicates(subset=["date"], keep="last").set_index("date")def extract_yfinance_field(raw, field, drop_empty=True): suffix =f"__{field}" cols = [col for col in raw.columns ifstr(col).endswith(suffix)] frame = raw[cols].apply(pd.to_numeric, errors="coerce").replace([np.inf, -np.inf], np.nan) frame.columns = [col.rsplit("__", 1)[0] for col in cols]if drop_empty: frame = frame.dropna(axis=1, how="all")return frame.reindex(columns=sorted(frame.columns))close_all = extract_yfinance_field(raw, "close")volume_all = extract_yfinance_field(raw, "volume")dividends_all = extract_yfinance_field(raw, "dividends", drop_empty=False)splits_all = extract_yfinance_field(raw, "stock_splits", drop_empty=False)def action_frame(frame, field): out = frame.copy() out.columns = pd.MultiIndex.from_product([out.columns, [field]])return outactions_all = pd.concat([action_frame(dividends_all, "dividends"), action_frame(splits_all, "stock_splits")], axis=1).sort_index(axis=1)
The loaded data has \(4107\) trading days and \(63\) adjusted close columns before we select the project’s ETFs. We’re not going to use all \(63\) available ETFs and most of them are for later projects and secondary data for this project like sector ETFs.
2.1 Choosing the ETFs
We now choose the ETFs we are going to analyze for this notebook. The tradable assets are:
Ticker
Role
What it represents
SPY
U.S. equity core
Large-cap US equities, commonly used as the broad US stock-market proxy.
QQQ
U.S. growth leadership
Nasdaq-100, usually more growth and technology heavy than SPY.
IWM
U.S. small caps
Russell 2000 small-cap US equities, more cyclical and domestically sensitive.
EFA
Developed international equity
Developed international equities outside North America.
EEM
Emerging markets equity
Higher-risk Emerging-market equities, usually more sensitive to global growth, dollar strength, and commodity cycles.
TLT
Long duration Treasuries
Long-duration US Treasuries (20+ years), very sensitive to changes in long-term interest rates.
IEF
Intermediate Treasuries
Intermediate Treasury (7-10 year), less duration risk than TLT.
SHY
Short Treasuries
Short-duration Treasury (1-3 year) and a defensive cash-like sleeve. Usually low volatility and low return.
AGG
Core bonds
Broad US aggregate bond, a core bond allocation.
LQD
Investment-grade credit
corporate bonds, combining rate exposure and credit exposure.
HYG
High-yield credit
High-yield corporate bonds, more equity like and more sensitive to credit conditions.
GLD
Gold
Holds physical gold, often treated as a real asset or crisis/inflation hedge.
VNQ
Real estate
US real estate investment trusts, rate-sensitive and equity-sensitive.
DBC
Commodities
Broad commodities futures, often connected to inflation and global real activity.
UUP
Dollar signal only
US dollar, used as an information variable rather than a tradable portfolio asset here.
2.2 behaviour of these ETFs
SPY, QQQ, and IWM are all equity ETFs, but they don’t behave the same. SPY is the broad large cap core. QQQ is more growth and technology heavy, so it usually benefits when investors reward long duration growth cash flows, innovations, and cap leadership. IWM is small cap exposure, so it tends to be more sensitive to growth and financing conditions
EFA and EEM give international exposure. EFA is developed market equity, while EEM is emerging market equity. Emerging markets are often more sensitive to dollar strength, global liquidity and commodity cycles. That’s why UUP is useful as a signal asset. If the dollar is strengthening aggressively, emerging markets can face pressure even if they look attractive based on their local currency.
The bond ETFs: SHY is short Treasury exposure, IEF is intermediate Treasury exposure, and TLT is long Treasury exposure. As we can remembe from the notebook 1 and fixed income risk measures implementation, The longer the duration, the more sensitive the ETF is to interest rate changes. If rates rise sharply, long duration bonds can suffer large falls in their value. If rates fall during a growth scare, duration can help stabilize the portfolio.
Credit ETFs sit between bonds and equities. LQD is investment grade credit, while HYG is high yield credit. HYG can behave like a risky asset because high-yield bonds carry more credit and default exposure. In calm markets, high yield can earn carry. In stress markets, it can sell off with equities because credit spreads widen.
GLD, VNQ, and DBC have real asset and inflation sensitive exposures. Gold can behave as a diversifier when confidence in financial assets weakens and the situation is uncertain, although it isn’t a perfect hedge. VNQ is real estate and is sensitive to both growth and rates. DBC is broad commodities and can perform well during inflationary commodity cycles, but it can also be very painful in long disinflationary periods.
We use these economic relationships to design the views below, linking each long-short position to the behavior of the assets and the market conditions behind it.
Show code
start_date ="2013-01-01"assets = ["SPY", "QQQ", "IWM", "EFA", "EEM", "TLT", "IEF","SHY", "AGG", "LQD", "HYG", "GLD", "VNQ", "DBC"]signal_only_assets = ["UUP"]sleeve_map = {"SPY": "US equity", "QQQ": "US equity", "IWM": "US equity","EFA": "International equity", "EEM": "International equity","TLT": "Duration bonds", "IEF": "Duration bonds", "SHY": "Short bonds","AGG": "Core bonds", "LQD": "Credit", "HYG": "Credit","GLD": "Real assets", "VNQ": "Real assets", "DBC": "Commodities","UUP": "Dollar"}close_after_start = close_all.loc[pd.Timestamp(start_date):].copy()missing_assets = [ticker for ticker in assets if ticker notin close_after_start.columns]missing_signal_assets = [ticker for ticker in signal_only_assets if ticker notin close_after_start.columns]if missing_assets or missing_signal_assets:raiseValueError(f"missing ETF close columns: tradable={missing_assets}, signal_only={missing_signal_assets}")filter_rows = []for ticker in assets + signal_only_assets: px = close_after_start[ticker] valid = px.dropna() first_valid = px.first_valid_index() missing_pct = np.nan if first_valid isNoneelsefloat(px.loc[first_valid:].isna().mean()) filter_rows.append({"ticker": ticker, "observations": len(valid),"first_date": valid.index.min(), "last_date": valid.index.max(),"missing_pct_after_first_valid": missing_pct,"tradable": ticker in assets})display(pd.DataFrame(filter_rows).round(4))first_all_valid = close_after_start[assets].dropna(how="any").index.min()if pd.isna(first_all_valid):raiseValueError("no date where all tradable ETF close columns are available")close = close_after_start.loc[first_all_valid:, assets].ffill(limit=3).dropna(how="any")returns = prices_to_returns(close, kind="simple").replace([np.inf, -np.inf], np.nan).dropna(how="all").fillna(0.0)close = close.reindex(returns.index).ffill(limit=3)signal_tickers =list(dict.fromkeys(assets + signal_only_assets))signal_close = close_after_start.loc[first_all_valid:, signal_tickers].ffill(limit=3).reindex(returns.index).ffill(limit=3)signal_returns = signal_close.pct_change(fill_method=None).replace([np.inf, -np.inf], np.nan).reindex(returns.index).fillna(0.0)
We made monthly rebalance dates and we can see that even though our data starts from 2013, we need a lookback window for estimating covariance and mean, so the first rebalance date happens at the first month of 2016.
As we can see, QQQ has been the strongest asset in the sample, with an annualized return close to 20% and volatility around 21%. SPY also performs strongly, with lower return and lower volatility than QQQ. IWM has a positive return but much higher drawdown, which is typical of small cap exposure especially in our sample where large cap growth leadership has been very strong.
The international equity ETFs are weaker. EFA and EEM both have large drawdowns and volatility, but their returns are much lower than U.S. equities. The differences in return and drawdown show why we need to consider the individual markets when choosing our equity weights. International exposure may still be useful, but an efficient model should not blindly assume that all equities and markets deserve the same allocation.
The bond ETFs show very different behavior. SHY has very low volatility around \(1.4\%\) and a much smaller drawdown around \(-5.7\%\), but the return is also low. TLT is the opposite of what someone might expect from “bonds”. It has a very large drawdown around \(-48.5\%\) in this sample because as we said earlier, long duration bonds are sensitive to interest rate changes. TLT fell hard by the post-2021 rate shock.
DBC has a very large drawdown around \(-62.2\%\), which can be from the long weak cycles. GLD and VNQ are somewhere between diversifiers and risk assets. They have meaningful volatility and large drawdowns, but they also bring different economic exposure.
An efficient active model like Black-Litterman should know when to lean into growth, when to prefer real assets, when to reduce long duration, and when to stay close to the benchmark so it can control risk and gain the high return based on the state and information and the uncertainty of that information given to it each time it’s rabalancing.
As we can see, volatility jumps during crisis periods, especially around the COVID shock. Equity and real asset ETFs show much larger volatility spikes than short duration bonds. SHY remains the lowest volatility instrument throughout the sample.
U.S. equity ETFs are highly correlated with each other. SPY and QQQ move very closely, and IWM also has strong equity market correlation. Developed and emerging market equity ETFs also connect strongly to the US equity. This means that simply owning SPY, QQQ, IWM, EFA, and EEM is not as diversified as the number of tickers suggests.
The bond holding ETFs are different. TLT, IEF, SHY, AGG, and LQD have their own internal structure. TLT and IEF are strongly related because both are Treasury duration exposures. TLT has negative correlation with SPY around \(-0.19\), which shows why long Treasuries historically helped in some equity selloffs. But the correlation is not fixed forever. In inflation shocks, stocks and bonds can fall together.
AGG and LQD are also connected to the broader bond and credit market. HYG behaves more like risky credit and often correlates more with equities than with pure Treasuries.
Gold and commodities contain a different kind of exposure. GLD has weaker relationships with most assets, which can make it a good diversifier. DBC has a different inflation and commodity cycle profile, so it can help when inflation sensitive assets are leading, but it can also be very difficult to hold during long weak commodity regimes.
If the optimizer only sees a covariance matrix and expected returns, it may concentrate in a few assets. The human portfolio manager might add economic sanity to the model instead of just relying on estimation. With Black-Litterman, we can combine those economic views with the estimated covariance matrix and a starting portfolio when choosing our allocation.
3) Building the strategic benchmark and Prior
If we remember the bayes rule from earlier, we want to update our belief when new information arrives. We first need an intial belief or a starting point that is going to change with the new information.
3.1 Benchmark portfolio
We first define a neutral portfolio as our benchmark. In Black-Litterman, this portfolio represents the starting belief about how capital should be allocated before we add any active views. Its weights give us a consistent starting point for the return estimates.
The benchmark we use is a portfolio with pre-defined fixed weights and the weights are based on what we think can be reasonable for these ETFs based on their performance, behaviour and what they hold. It gives meaningful weight to US equities, international equities, bonds, credit, gold, real estate, and commodities. It is also constrained by asset caps and constraints. These caps protect the optimization from unrealistic concentration on just one asset.
For example, even if QQQ has a strong historical return, we probably do not want a mult asset benchmark to become almost entirely QQQ. We are looking for diversification. Also, even if short Treasuries have low volatility, a diversified benchmark should not become a pure cash substitute. In that case we will end up with a very low volatility portfolio but almost the worst return.
We keep the optimizer within a multi-asset allocation structure by placing bounds and caps on the weights of each asset. These limits also make the resulting allocation easier to compare with the benchmark and its broad sleeves.
Since we already know from notebook 2, A benchmark weight vector looks like this:
As we can see, we set SPY as the largest position at about \(22.2\%\), followed by AGG around \(10.1\%\), EFA around \(9.1\%\), and several exposures around \(8.1\%\) or \(5.1\%\). Large weights go to liquid core exposures. SPY for broad US equity, AGG for core bonds, EFA for developed international equities, and intermediate/long Treasuries for duration. Smaller weights go to more specialized or volatile exposures such as DBC, VNQ, HYG, and IWM. This makes sense because those assets can be useful, but we usually do not want them to dominate the portfolio.
US equity is the largest section at about 34.3%. Duration bonds and international equities are also used enough. Core bonds, real assets, and credit each have visible allocations, while commodities are intentionally small.
This gives us a strategic benchmark with enough equity exposure to participate in growth and allocations to bonds, gold, real estate, credit, and commodities as well. Each sleeve has a role, with larger core positions balanced by smaller specialized exposures. We can now use these weights as the starting point for the active allocation and compare the views against it.
We now build the first Black-Litterman object: the prior expected return vector.
In ordinary mean-variance optimization, expected returns are usually the weakest input. Historical average returns are noisy, unstable, and extremely sensitive to the sample window. Black-Litterman avoids starting from raw sample means. Instead, we do a reverse action. We now have a portfolio with our defined weights, so there has to be a Mean-Variance that produces these weights with an unknown mean. We are looking for that mean given our weights. We want to see what expected returns would make this benchmark look optimal for an investor?
Here \(\mu\) is the expected return vector, \(\Sigma_t\) is the covariance matrix estimated at time \(t\), \(w_b\) is the benchmark weight vector, and \(\delta_t\) is the risk-aversion coefficient. Higher \(\delta\) means the investor wants more compensation for taking variance.
the unconstrained solution is \(\mathbf{w}_b^* = \frac{1}{\delta} \boldsymbol{\Sigma}^{-1} \boldsymbol{\mu}\), which can be changed to give:
The vector \(\mu_\text{eq}\) is the prior expected return vector. If an investor maximizes expected return minus a penalty for variance, then the expected return vector that supports a given benchmark is equal to covariance times benchmark weights. If an asset contributes a lot of risk to the benchmark, it must have a higher implied return to justify its place. If an asset is defensive or low risk, it’s implied return is usually lower.
3.2.2 Estimating Covariance \(\Sigma\) and Risk Aversion \(\delta\)
We also estimate \(\Sigma\) with EWMA covariance. As we analyzes in notebook 2, EWMA puts more weight on recent returns:
A high \(\lambda\) such as \(0.97\) makes the covariance estimate persistent but still responsive. This is useful for portfolio risk because covariance regimes change: 2020, 2022, and 2025 do not have the same market structure.
The parameter \(\delta\) is the market risk aversion coefficient which controls the overall scale of the implied returns. Economically, \(\delta\) is the additional expected return the representative investor requires per unit of portfolio variance. We get the \(\delta\) from benchmark too:
This is the Sharpe ratio of the benchmark expressed in variance units. It measures how many units of expected excess return the market demands per unit of variance. Rather than assuming a fixed \(\delta = 2.5\) (usual in literature), we estimate it dynamically at each rebalance date from the rolling historical performance of the benchmark portfolio itself.
you might be confused about using \(\lambda\) as the risk reversion parameter for Mean-Variance model in notebook 2, and here, using \(\delta\). We use \(\delta\) as the symbol for implied risk aversion, which means extracting it from an existing model/portfolio. But \(\lambda\) is used as the parameter of risk aversion for optimizing a model and getting to the optimal portfolio we want.
Note: The prior is not static. At each rebalance date we use the current EWMA covariance estimate, so \(\boldsymbol{\mu}_{\text{eq}}\) evolves with the market’s volatility and correlation structure.
In the latest prior estimates, we can see EEM at about \(21.5\%\), EFA around \(17.2\%\), GLD around \(16.5\%\), IWM around \(15.5\%\), QQQ around \(14.1\%\), and SPY around \(11.3\%\). These are annualized equilibrium returns implied by the benchmark, the latest covariance estimate, and the risk-aversion estimate.
The high prior estimates for EEM, EFA, GLD, IWM, QQQ, and SPY come from their covariance with the benchmark and the current risk-aversion estimate. They represent the equilibrium return required to support those benchmark exposures under the latest covariance estimate. The ranking can change at each rebalance as the covariance and risk-aversion estimates change. DBC having a negative prior in the latest table means the benchmark and covariance structure do not naturally reward DBC in the same way at that time, so any active overweight to commodities must come from views rather than from the prior alone. This means we say on default, investing on commodities doesn’t benefit us in the way we want it, but since we use Black-Litterman, our views might show us things we don’t kno, guiding us to invest on them in the right time.
GLD receiving a high prior is interesting. Gold doesn’t always have high historical average return, but in a covariance-based prior it can receive more implied return when it contributes useful portfolio risk control maybe as a diversification power. This is one of the reasons Black-Litterman is more stable than raw historical mean estimation.
In periods when an asset becomes more volatile or more connected to the benchmark, its implied prior can move. In 2020 and parts of 2022 which are stress regimes, implied returns change a lot because covariance and benchmark risk conditions changed suddenly. This is better than using one fixed prior for the whole sample. If market risk structure changes, the return should update too.
4) Building baseline strategies
We now run the baseline portfolio models from the portfolios we made in notebook 2, and compare them against the strategic benchmark.
A quick reminder from notebook 2: The minimum-variance problem is
\[
\min_w w^{\top}\Sigma w
\]
subject to the usual constraints like full investment, long-only weights, caps, and sleeve bounds.
The mean-variance problem is
\[
\max_w w^{\top}\mu-\lambda w^{\top}\Sigma w
\]
where \(\mu\) is the expected return vector. The problem is that \(\hat\mu\) is noisy. A small change in expected returns can create a large change in weights. This is why the baseline uses shrinkage-style inputs such as Ledoit-Wolf covariance and Bayes-Stein means as we implemented and explained in notebook 2, here we don’t repeat the code and use the library’s grid function for producing models.
candidate_rows = []for name, result in baseline_candidates.items(): m = selection.performance_metrics(result["returns"], result["nav"], rf_daily=risk_free_daily, annualization=annualization) candidate_rows.append({"strategy": name, "cagr": m["CAGR"], "ann_vol": m["Vol"],"sharpe": m["Sharpe"], "max_drawdown": m["Max Drawdown"],"avg_turnover": result["turnover"].mean()})baseline_candidate_summary = pd.DataFrame(candidate_rows).sort_values("sharpe", ascending=False)display(baseline_candidate_summary.round(4))
strategy
cagr
ann_vol
sharpe
max_drawdown
avg_turnover
0
Strategic Benchmark
0.0914
0.1014
0.5284
-0.2242
0.0144
3
MV (LedoitWolf, BayesStein)
0.0814
0.0847
0.4949
-0.1378
0.0369
1
Equal Weight
0.0848
0.0935
0.4842
-0.2038
0.0114
2
MinVar (OAS)
0.0665
0.0754
0.3543
-0.1973
0.0031
Show code
avg_turnover_table = pd.DataFrame({name: result["turnover"].mean() for name, result in baseline_results.items()}, index=["avg_turnover"]).Tdisplay(avg_turnover_table.round(4))baseline_nav = pd.DataFrame({name: result["nav"] for name, result in baseline_results.items()})baseline_dd = baseline_nav / baseline_nav.cummax() -1.0fig, axes = plt.subplots(2, 1, figsize=(9, 6), sharex=True)baseline_nav.plot(ax=axes[0])axes[0].set_title("benchmarks nav")axes[0].set_ylabel("nav")baseline_dd.plot(ax=axes[1], lw=1.1)axes[1].set_title("benchmarks drawdown")axes[1].set_ylabel("drawdown")axes[1].set_xlabel("date")plt.tight_layout() plt.show()
avg_turnover
Strategic Benchmark
0.0144
Equal Weight
0.0114
MinVar (OAS)
0.0031
MV (LedoitWolf, BayesStein)
0.0369
As we can see, the strategic benchmark has the highest Sharpe ratio, around \(0.52\), with a CAGR around \(9.1\%\) and volatility around \(10.1\%\).
The MV Model using Ledoit-Wolf and Bayes-Stein has the lowest drawdown among these baselines, around \(-13.8\%\), and volatility around \(8.4\%\), but its CAGR is also lower than the benchmark. So it’s safer in path terms, but it doesn’t beat the benchmark on return and Sharpe. Equal weight is close but slightly weaker. The minimum-variance portfolio has lower volatility around \(7.6\%\), but its CAGR is only around \(6.8\%\) and its Sharpe is around \(0.38\).
This is the issue that MinVar has in these assets: there are low-volatility assets that also have very low returns. SHY is almost one of the safest investments in the universe, but it’s also one of the lowest-return assets. A model like minimum variance will overweight it to reduce volatility. But reducing variance doesn’t automatically create a better return-risk tradeoff if the portfolio gives up too much return. That’s why it ends up as the lowest-Sharpe model in our benchmarks. For these assets specifically, taking return more seriously is more important than the stock market that we tested in notebook 2, because there were a lot of very high return-high risk stocks in nasdaq market and controlling risk was really important to a better performance. But here, we have all ETFs having lower volatility compared to the stocks, but also not that high of return, so having higher return in a way to control risk enough, is important, because in this specific set of assets, reducing volatility too much, will hurt returns too much.
Another note is that our own benchmark already has the best Sharpe among the baseline models. Even if the views don’t add anything, or even move the benchmark in the wrong direction, the Black-Litterman model can still look better than some of the purely statistical baselines. We therefore compare BL with MinVar and Equal Weight, and also compare it directly with the strategic benchmark to see whether the views and learned confidence actually add return and Sharpe over the starting portfolio on the same dates and after the assumed trading costs.
Benchmark-relative return and risk give us a direct measure of what the active views add to the strong starting portfolio. We can use those results to see how much improvement comes from the views after taking active risk and trading costs.
5) market signals
Before making a Black-Litterman view, we need a way to describe the market environment. We use signals that are common in asset allocation: momentum, trend, volatility, drawdown quality, dividend yield, relative strength, and correlation.
Momentum signals: Computed as the cumulative return of an asset over a trailing window, excluding the most recent month (the standard momentum definition avoids short-term reversal), for example, this is how a 6 month momentum is calculated:
We compute this for 1-month, 3-month including recent, 6-to-1 month, and 12-to-1 month.
Trend signals: The price of each asset relative to its \(n\)-day simple moving average (SMA), normalized by the SMA: \[
\text{Trend}_n = \frac{\text{Price} - \text{SMA}_n}{\text{SMA}_n}
\]
A positive 200-day trend means the asset is in a long-term uptrend. Negative means a downtrend. This is close to momentum, but it gives us information about recent performance in a different way: it compares the latest price with the asset’s own moving average instead of only comparing today’s price with an old price.
Volatility and drawdown signals: Rolling 63-day volatility and trailing 252-day drawdown capture the risk profile and stress state of each asset.
Composite score: Each asset gets an overall score combining the above signals, which ranks assets from most to least attractive for overweighting relative to the benchmark. This score is used as a sorting tool inside view-generation rules. But these scores might not be in the same scale, and we have to make them comparable to each other.
Therefore we use cross-sectional z-scores. A z-score shows us whether an asset’s signal is strong or weak relative to the other assets at the same date. A z-score turns different things into comparable units. The basic idea is
We combine these indicators so that the ranking uses several aspects of asset behaviour instead of relying on a single noisy measure at each rebalance date. Momentum, trend, volatility, and drawdown quality each capture a different part of the market state.
Market state signals: Besides per-asset signals, we compute a set of market-wide state variables that describe the current market environment: - fraction of risky assets (SPY, QQQ, IWM, EEM, HYG, VNQ, DBC) above their 200-day SMA, measures broad market health - current SPY drawdown from its 252-day high indicates whether equities are in a stress environment and helps us adjust the views for the asset allocation. - rolling 126-day correlation between SPY and TLT helps us understand whether Treasury duration is providing diversification against equity moves - relative cumulative returns of one ETF versus another over 63 days is the market’s current relative performance signal that directly feeds into views
We winsorize the cross-sectional inputs at each date before turning them into z-scores. That keeps one extreme ETF reading from dominating the whole signal table, while still preserving the ranking information that the view rules need.
Show code
risky_asset_names = ["SPY", "QQQ", "IWM", "EFA", "EEM", "HYG", "VNQ", "DBC"]cyclical_asset_names = ["IWM", "EEM", "VNQ", "HYG"]defensive_asset_names = ["TLT", "IEF", "SHY", "AGG", "LQD", "GLD"]def winsorized_zscore(series): s = pd.Series(series, dtype=float).replace([np.inf, -np.inf], np.nan) good = s.dropna() clipped = s.clip(good.quantile(0.05), good.quantile(0.95)) std = clipped.std(ddof=0)ifnot np.isfinite(std) or std <=1e-12:return pd.Series(0.0, index=s.index)return ((clipped - clipped.mean()) / std).fillna(0.0)def cumulative_return(window): clean = pd.Series(window, dtype=float).dropna()return np.nan if clean.empty elsefloat((1.0+ clean).prod() -1.0)def trailing_dividend_yield(dividends, close_hist, ticker, lookback=252):if dividends isNoneor dividends.empty:return np.nan div_window = dividends[ticker].reindex(close_hist.index).fillna(0.0).tail(lookback) px = close_hist[ticker].dropna()returnfloat(div_window.sum() / px.iloc[-1]) iflen(px) and px.iloc[-1] >0else np.nandef relative_cumulative_return(ret_hist, long_asset, short_asset, lookback=63):iflen(ret_hist) < lookback:return np.nan long_ret = cumulative_return(ret_hist[long_asset].tail(lookback)) short_ret = cumulative_return(ret_hist[short_asset].tail(lookback))return long_ret - short_ret if np.isfinite(long_ret) and np.isfinite(short_ret) else np.nandef trailing_pair_correlation(ret_hist, asset_a, asset_b, lookback=126):iflen(ret_hist) < lookback:return np.nan window = ret_hist[[asset_a, asset_b]].tail(lookback).dropna()iflen(window) <max(30, lookback //3):return np.nanif window[asset_a].std(ddof=1) <=1e-12or window[asset_b].std(ddof=1) <=1e-12:return np.nanreturnfloat(window[asset_a].corr(window[asset_b]))def trailing_average_correlation(ret_hist, names, lookback=126): names = [name for name in names if name in ret_hist.columns]iflen(names) <2orlen(ret_hist) < lookback:return np.nan window = ret_hist[names].tail(lookback).dropna(how="any")iflen(window) <max(30, lookback //3):return np.nan corr = window.corr().replace([np.inf, -np.inf], np.nan) vals = corr.where(~np.eye(len(corr), dtype=bool)).stack().dropna()returnfloat(vals.mean()) iflen(vals) else np.nandef trend_breadth_from_close(close_hist, names, ma_window=200, offset=0): names = [name for name in names if name in close_hist.columns] hist = close_hist.iloc[:len(close_hist) - offset] if offset andlen(close_hist) > offset else close_hist vals = []for ticker in names: px = hist[ticker].dropna()iflen(px) < ma_window:continue ma = px.tail(ma_window).mean()if ma >0: vals.append(px.iloc[-1] / ma -1.0)returnfloat((pd.Series(vals) >0).mean()) if vals else np.nan
From the latest signals (2026-06-17) table, we can see that EEM has the strongest latest score by a wide margin. It has strong recent momentum, strong medium-term momentum, a positive trend, and a small drawdown. DBC, IWM, QQQ, SPY and EFA also score positively, while several bond and credit ETFs score negatively.
The latest market state points in the same direction. The risky median score is above the defensive median score, so the risk-on score is positive. Risky trend breadth and cyclical trend breadth are both strong. SPY is not in a deep drawdown, and the equity stress flag is false. The commodity-versus-duration signal is also strong because DBC has been outperforming duration-sensitive bond exposures.
Together, these indicators give us a picture of the current market before we form the views: risk-on, commodity-positive, cyclicals reasonably strong, bonds relatively weak, and dollar strength present but not dominant enough to overturn the other signals. Now we want to use these signals and states to generate the views that directly affect the Black-Litterman model. We can trace each view back to the state measures and the underlying ETF signals.
The useful thing about this signal layer is that it stays separate from the optimizer. We can inspect the market state first, then inspect the views, and only after that look at posterior returns and final weights. That makes mistakes much easier to diagnose.
6) Black-Litterman views
We now translate market signals into views. These views are the connection between the market information and our portfolio decisions.
A view in Black-Litterman is a statement of the form: “I believe that asset (or portfolio) \(A\) will outperform asset (or portfolio) \(B\) by \(q\) per year.” Formally, a view is a pair \((\mathbf{p}, q)\) where \(\mathbf{p} \in \mathbb{R}^N\) is a portfolio vector encoding which assets are long and which are short, and \(q \in \mathbb{R}\) is the expected excess return of that portfolio vector \(\mathbf{p}\).
The portfolio vector \(\mathbf{p}\) for a simple relative view (asset \(A\) outperforms asset \(B\)) is: \[
p_i = \begin{cases} +1/|L| & \text{if asset } i \text{ is long} \\ -1/|S| & \text{if asset } i \text{ is short} \\ 0 & \text{otherwise} \end{cases}
\]
where \(|L|\) and \(|S|\) are the number of long and short assets in the view. This equal-weighting within each side keeps views comparable in scale. The vector \(\mathbf{p}\) sums to zero (long equals short), which means the view expresses a relative return rather than an absolute return level.
If we have \(K\) views and \(N\) assets, we’re working with a view matrix\(P\) that has shape \(K\times N\). Each row of \(P\) defines one long-short or relative-return portfolio. The view vector \(q\) has length \(K\) and contains the expected return attached to each view.
For one view \(k\), the statement is
\[
P_k\mu=q_k
\]
For \(K\) views, all rows stack into the view matrix:
If the view is “QQQ and SPY should outperform HYG and VNQ,” then the portfolio row could look like a long-short spread with positive weights on QQQ and SPY, negative weights on HYG and VNQ, and zeros elsewhere. The row sums to zero when it’s a relative view.
This relative view expresses an expectation that one set of assets will outperform another. A portfolio manager can use that expected spread directly without assigning a separate forecast to every asset. The row of \(P\) records the assets that enter the long and short sides.
We build several economically motivated view families that can track market states and regimes repeatedly. Each family uses signals to select the long and short assets, so the view can change as the market environment changes.
How do we write these views in math form for the model? We extract a portfolio vector from the idea and attach an expected return to that portfolio. We can’t just say “SPY is going to outperform EEM” inside the BL equation, but we can say “I have a portfolio \(\mathbf{p}\) with weight 50% on SPY and -50% on EEM, with expected return \(q\) equal to \(0.50\mu_{\mathrm{SPY}}-0.50\mu_{\mathrm{EEM}}\)”.
If the universe is ordered as SPY, QQQ, IWM, EFA, EEM, TLT, IEF, SHY, AGG, LQD, HYG, GLD, VNQ, DBC, then one inflation-style view could look like:
A Black-Litterman view can describe an expected relative return for that long-short portfolio. We can add the view to the benchmark-implied prior without separately forecasting the standalone expected return of every asset in the universe.
We interpret the signs in \(\mathbf{p}\) as the long and short sides of each view portfolio. Positive entries are the side we want to reward, negative entries are the side we expect to lag, and zeros are assets that aren’t directly part of the view. The posterior can still move the zero-weight assets indirectly through covariance, but the original belief is expressed only through the long-short row.
6.1. constructing the views
1. Liquidity Leadership: We look for periods when large-cap US growth (SPY, QQQ) is outperforming cyclical and small-cap assets (IWM, EEM, VNQ, HYG) on a trend and momentum basis. Economically, this corresponds to a narrow market leadership regime when investors are concentrating risk in the most liquid, high-quality names rather than spreading into cyclicals. This is often seen in late-cycle or uncertainty environments. We go long the leaders (SPY, QQQ) and short the lagging cyclicals when this view is active.
2. Dual Momentum: This is a cross-sectional momentum view. The assets with the best recent risk-adjusted momentum scores are expected to continue outperforming the assets with the worst scores. We form the view by going long the top-ranked assets and short the bottom-ranked assets across all 14 ETFs. This can work well in practice even though it’s completely backward-looking, because cross-asset trends often persist longer than single-stock noise.
3. Inflation Rotation: We look for periods when real assets (GLD, DBC) are outperforming fixed income (IEF, AGG) on a trend basis, signalling an inflationary environment. Economically, when commodities and gold are trending up relative to bonds, the market is pricing in higher inflation expectations. We go long the real assets and short the bonds in this view.
4. Risk-Adjusted Momentum: A more sophisticated version of dual momentum that addresses volatility too. Assets are ranked by their Sharpe-ratio-like signal (return per unit of recent volatility). High Sharpe-ratio assets are expected to continue outperforming low Sharpe-ratio assets.
5. Credit Switch: When HYG (high yield) is significantly outperforming LQD (investment grade) on a relative basis, the credit market is in a risk-on state. We go long high yield and short investment grade, taking the view that credit spreads will continue to compress. When HYG is underperforming, the signal reverses.
6. Reflation Breadth: Here we look for a broad economic reflation. A regime where small caps outperform large caps (IWM > SPY), high yield outperforms investment grade, cyclical trend breadth is high, and commodities are trending. When all these conditions are confirmed together, we go long cyclicals (IWM, EEM, VNQ, HYG, DBC) and short SPY and TLT. This is close to a risk-on reflation macro regime.
7. Growth-Duration Barbell: We look for periods when equity momentum (QQQ/SPY relative performance) and bond momentum (TLT/IEF) are strong together, suggesting the market is rewarding both growth and duration. This is a barbell view: long growth equity and long-duration bonds, short the middle exposures such as IWM, EEM, and HYG. This regime covers much of the 2014-2020 period.
8. Correlation Stress: When the stock-bond correlation turns positive (both stocks and bonds are falling together, as happened in 2022), the traditional diversification benefit of bonds breaks down. We reduce long duration exposure (TLT, IEF) and favour real assets and short duration (GLD, SHY) as alternative diversifiers for the portfolio.
9. International Rotation: We track the relative performance of emerging markets (EEM) versus developed international (EFA). When EEM is outperforming EFA on momentum, it signals a risk-on, dollar-weak, commodity-supportive environment. We go long EEM and short EFA in this view. When the dollar (UUP) is strengthening, we control this view so it doesn’t fire too easily and create a false emerging-market signal.
10. Duration Quality: We monitor the relative performance of long-duration bonds (TLT) versus short-duration (SHY) to determine the slope of the term premium. A positive signal (TLT outperforming SHY) shows the market is pricing in falling rates and suggests an overweight in long duration.
These view families are intentionally not symmetric in frequency. Momentum views should appear often because they describe broad ranking behaviour. Stress and inflation views should be rarer because they represent more specific regimes. That difference later affects how much data each view has for learned confidence.
Show code
view_caps = {"liquid_leadership": 0.08, "dual_momentum": 0.10, "inflation_rotation": 0.05,"risk_adj_momentum": 0.10, "credit_switch": 0.05, "reflation_breadth": 0.10,"growth_duration": 0.08, "correlation_stress": 0.08,"international_rotation": 0.10, "duration_quality": 0.08}view_names = {"liquid_leadership": "Liquidity leadership", "dual_momentum": "Dual momentum","inflation_rotation": "Inflation rotation", "risk_adj_momentum": "Risk-adjusted momentum","credit_switch": "Credit switch", "reflation_breadth": "Reflation breadth","growth_duration": "Growth-duration barbell", "correlation_stress": "Correlation stress","international_rotation": "International rotation", "duration_quality": "Duration quality"}q_strength_scale =1.25def p_series_from_assets(long_assets, short_assets, asset_list=assets): p = pd.Series(0.0, index=asset_list, dtype=float) long_assets =list(dict.fromkeys([asset for asset in long_assets if asset in p.index])) short_assets =list(dict.fromkeys([asset for asset in short_assets if asset in p.index and asset notin long_assets]))ifnot long_assets ornot short_assets:returnNone p.loc[long_assets] =1.0/len(long_assets) p.loc[short_assets] =-1.0/len(short_assets)return pdef make_view_row(view_id, view, long_assets, short_assets, view_strength, risk_orientation, source="view_rule", priority=0.50, confluence_score=None, view_state=None): long_assets =list(dict.fromkeys([asset for asset in long_assets if asset in assets])) short_assets =list(dict.fromkeys([asset for asset in short_assets if asset in assets and asset notin long_assets]))if view_id notin view_caps ornot long_assets ornot short_assets ornot np.isfinite(view_strength):returnNone view_strength =float(abs(view_strength))if view_strength <=1e-8:returnNone cap = view_caps[view_id] q_tilt =float(cap * math.tanh(view_strength / q_strength_scale)) p = p_series_from_assets(long_assets, short_assets)if p isNone:returnNone priority =float(np.clip(priority, 0.0, 1.0)) confluence = view_strength /3.0if confluence_score isNoneelse confluence_score confluence =float(np.clip(confluence, 0.0, 1.0)) view = view or view_names.get(view_id, view_id.replace("_", " ").title())return {"view_id": view_id, "view": view, "view_state": view_state or risk_orientation,"long_assets": long_assets, "short_assets": short_assets,"signal_value": view_strength, "raw_strength": view_strength,"view_strength": view_strength, "confluence_score": confluence,"q_tilt": q_tilt, "q": q_tilt, "confidence": np.nan,"risk_orientation": risk_orientation, "source": source,"priority": priority, "view_priority": priority,"p_vector": p.round(6).to_dict()}def best_assets(signal_table, candidates, n=1, require_positive=False): names = [asset for asset in candidates if asset in signal_table.index] frame = signal_table.reindex(names).sort_values("score", ascending=False)if require_positive: frame = frame[(frame["score"] >0) | (frame["trend_200"] >0) | (frame["mom_3_0"] >0)]returnlist(frame.head(n).index)def worst_assets(signal_table, candidates, n=1, require_weak=False): names = [asset for asset in candidates if asset in signal_table.index] frame = signal_table.reindex(names).sort_values("score")if require_weak: frame = frame[(frame["score"] <0) | (frame["trend_200"] <0) | (frame["mom_3_0"] <0)]returnlist(frame.head(n).index)def sleeve_limited_assets(signal_table, candidates, n=3, side="long", per_sleeve=1): names = [asset for asset in candidates if asset in signal_table.index] frame = signal_table.reindex(names).sort_values("score", ascending=(side =="short")) picks, sleeve_counts = [], {}for asset, row in frame.iterrows(): sleeve = row.get("sleeve", sleeve_map.get(asset, "Other"))if sleeve_counts.get(sleeve, 0) >= per_sleeve:continueif side =="long"and row["trend_200"] <=0and row["mom_3_0"] <=0:continueif side =="short"and row["trend_200"] >=0and row["mom_3_0"] >=0:continue picks.append(asset) sleeve_counts[sleeve] = sleeve_counts.get(sleeve, 0) +1iflen(picks) >= n:breakreturn picks
As we can see in the latest state, we have 5 active views built from the signals and strong enough, based on \(q\) tilt, to be kept: liquidity leadership, dual momentum, risk-adjusted momentum, international rotation, and duration quality. Together, these views cover market leadership, momentum, international rotation, and duration.
Dual momentum is the strongest view by annualized tilt. We go long the strongest assets such as QQQ, DBC, EEM, and VNQ, and short assets such as TLT and GLD. This is a clean cross-asset relative-strength view. Risk-adjusted momentum is the second-strongest \(q\)-tilt view and is almost similar to dual momentum, but it shorts VNQ instead of longing it. That’s the effect of including volatility and drawdown quality in the momentum score.
International rotation goes long EEM against EFA, which shows emerging-market leadership compared to developed international equities.
Duration quality has a small tilt, which also makes sense. It shows that inside fixed income, shorter or higher-quality exposures look more attractive than long-duration TLT when duration risk doesn’t look worth taking.
We first convert the signals into interpretable relative-return views, then use those views in the portfolio. This lets us inspect the long and short sides and the expected spread before we move to the allocation.
6.2 View Strength and the Tilt Cap
Before adding the view, the prior already implies an expected return spread:
where \(\boldsymbol{\mu}_{\text{prior}}\) is the Black-Litterman prior return vector. This is what the benchmark-implied prior already believes about the long-short spread.
We don’t replace the prior with the view starting from zero. We add a tilt on top of the prior spread:
So the final tilt is the part that actually makes the posterior move away from the prior in the direction of the view.
The tilt \(q_{\text{tilt}}\) is the amount that the view moves expected returns away from what the prior already implies for this portfolio. If the prior already assigns the right weights, the tilt is zero and the view adds nothing.
We also need to control the size of each view as signals change. Each view family has a maximum allowed view size that applies to its active return tilt. This gives the family a fixed scale for translating signal strength into an annualized return view. This avoids one strong-looking signal from creating an unrealistic return estimate. If the signal strength is \(s_i\) and the family cap is \(c_i\), we use a smooth function:
The hyperbolic tangent is useful because it maps signal strength to \((-1, 1)\) and grows quickly at first and then flattens. A signal can become stronger, but the view can’t become too large. This is more realistic than a linear rule because financial signals don’t become twice as trustworthy just because the z-score is twice as large. A small signal creates a small active return view. A strong signal creates a larger active return view. But once the signal is already strong, the cap prevents overconfidence.
For example, the liquid leadership view has a cap of 8%, meaning even an infinitely strong signal can’t push the expected return tilt for that view above 8% per year. The \(q_{\text{scale}} = 1.25\) parameter controls how quickly the tilt saturates. Smaller values mean the tilt reaches its cap faster for a given signal strength.
Having all 10 views active all the time would cause problems, so we use a selection step to keep the most useful views with meaningful \(q_\text{tilt}\).
The cap sets the maximum active-return size for a view family before confidence is learned. We use the learned confidence to set view uncertainty later, while the cap controls how large the raw return tilt is allowed to become.
8.3 Candidate view log through time
After defining the view rules, we generate candidate views at every rebalance date and record the view activity through time. This gives us two diagnostics:
how many views tend to fire in each month,
which view families dominate the historical sample.
The frequency of a view family matters because it changes the reliability problem. A view that fires 126 times has enough observations to estimate payoff quality with some usefulness. A view that fires 4 times can still be economically important, but its learned confidence has to be treated with caution.
In the q-tilt heatmap we can compare how strong each view is at the dates when it appears, as well as how frequently it becomes active. A view that fires often with tiny \(q\) is less influential than a view that fires less often with large \(q\).
where \(K_t\) changes over time. Black-Litterman can handle a changing number of views because the posterior update only requires the current \(P_t\), \(\mathbf{q}_t\), and \(\Omega_t\).
This log is also useful for debugging. If a view fires in a period where its economic story doesn’t make sense, we can inspect the signal inputs for that date rather than only looking at the final portfolio weight.
Show code
max_active_views =len(view_caps)def view_signature(row): long_side =tuple(sorted(row.get("long_assets", []))) short_side =tuple(sorted(row.get("short_assets", [])))return long_side, short_sidedef combine_and_rank_views(simple_views): rows, seen = [], set()for original in simple_views: sig = view_signature(original)if sig in seen:continue seen.add(sig) row =dict(original) cap = view_caps.get(row.get("view_id"), 0.020)if cap >0: q_score =float(np.clip(abs(row.get("q_tilt", 0.0)) / cap, 0.0, 1.0))else: q_score =0.0 row["q_score"] = q_score row["adjusted_strength"] = (0.55* row.get("confluence_score", 0.0) +0.25* q_score+0.20* row.get("view_priority", 0.0)) rows.append(row)returnsorted(rows, key=lambda item: item["adjusted_strength"], reverse=True)[:max_active_views]
Candidate view counts usually sit around 4 to 6, but the count changes through time as the market state changes. At each monthly rebalance, the signals determine which view families have an active candidate. The number of views therefore reflects the opportunities identified on that date, while the benchmark prior remains available when fewer candidates are active.
Risk-adjusted momentum and Dual momentum dominate the candidate set. Risk-adjusted momentum fires 126 times with average q-tilt around 0.0883. Dual momentum fires 119 times with average q-tilt around 0.0867. These are the broadest and most persistent view families, so it makes sense that they appear often.
Duration quality fires 78 times, but with a smaller average tilt around 0.0334. Liquidity leadership, international rotation, and credit-switch risk-on also fire frequently. Reflation breadth and inflation rotation are more regime-specific. Correlation stress only fires 4 times, which is exactly the kind of view that should stay rare.
The q-tilt heatmap clusters by regime. Momentum views are active through most of the sample, while inflation and correlation views concentrate in specific regimes. The payoff histories available for confidence learning differ across these families: frequent views provide more observations, while rare views receive stronger conservative caps.
Correlation stress responds to periods when stocks and bonds move together, while reflation breadth combines cyclical, credit, and commodity signals. These families enter the candidate set when their specific conditions line up, adding views about the market environment alongside the broader cross-asset momentum rankings.
9) View payoff history
We now build a payoff history for each view family, which lets us connect the confidence assigned to a view with the performance of earlier instances of that view.
Classical Black-Litterman asks the user to choose confidence manually. In practice, that is hard. Saying “I am 70% confident in this view” sounds precise, but it often has no statistical meaning unless we tie it back to realized performance. We instead measure how the view has actually performed whenever it fired in the past.
For each view at date \(t\), we look forward \(h=21\) trading days and compute the cumulative return of each asset:
This puts payoff values on the same annual scale as \(q\) and the posterior expected returns.
A view earns a positive payoff when the long side beats the short side over the next month. That is the direct test of whether the view was useful.
We form each view using the signals available on its date, then measure its payoff using the following returns. The long-short performance over that forward window tells us whether the view was right, and the realized payoff becomes available only after the window ends.
9.1 Hit rate, payoff IR, t-stat, and information coefficient
Once we have payoff history, we summarize each view family.
The hit rate is the fraction of realized payoffs that were positive:
The payoff IR compares a view family’s average payoff with the standard deviation of those payoffs. Like a Sharpe-style ratio, it expresses the average outcome per unit of payoff variability.
The t-statistic is:
\[
t_k = IR_k\sqrt{n_k}
\]
It rewards both consistency and sample size. A view with decent IR and many observations earns more trust than a view with decent IR from only a few observations.
The information coefficient checks whether stronger signal values actually lead to stronger future payoffs:
A positive information coefficient means stronger signals have tended to accompany larger subsequent payoffs, while a weak coefficient gives us less support for scaling \(q\) aggressively as the signal grows.
The hit rate counts how often payoffs are positive, giving each positive outcome the same contribution to that fraction. The information coefficient instead relates the strength of each signal to the size of its realized payoff, so the two measures can give us different information when we decide how much to scale \(q\).
We can use the realized payoff histories to see which of our view families have earned more trust.
The best broad evidence comes from Dual momentum and Risk-adjusted momentum. Dual momentum has 117 observations, hit rate around 57.26%, recent hit rate 100%, average annualized payoff around 4.78%, and payoff IR around 0.1106. Risk-adjusted momentum has 124 observations, hit rate around 57.26%, recent hit rate 66.67%, and payoff IR around 0.0893. These are not huge IR values, but for noisy monthly cross-asset views, they are meaningful enough to deserve moderate confidence.
Credit switch: risk-on has a strong profile: hit rate around 65.12%, average payoff around 15.04%, and payoff IR around 0.3733. This suggests that when credit confirms a risk-on environment, the view has historically been useful.
Reflation breadth also looks strong, with hit rate around 71.43% and payoff IR around 0.3905, but it has only 21 observations. This is useful but less statistically mature than momentum.
Some views are weak. Duration quality has hit rate around 35.53% and negative payoff IR. International rotation has hit rate around 44.44% and negative payoff IR. Credit switch: risk-off has very poor hit rate around 23.08%. These views can still be selected if current signals support them, but learned confidence needs to shrink them.
Correlation stress has excellent-looking numbers, with hit rate 75% and payoff IR 0.5361, but only 4 observations. We shouldn’t give it high confidence just because the few examples worked. The sample is too small.
The payoff histories let us compare each family’s economic rationale with what happened after its past signals. We use those realized outcomes to adjust confidence, giving the model a way to respond when an appealing view has a weak payoff record in the available history.
10) Learned confidence
Now we convert payoff statistics into a confidence number. Confidence is bounded between 0.30 and 0.90:
\[
0.30 \leq \omega_k \leq 0.90
\]
Here \(\omega_k\) is not the same as the Black-Litterman uncertainty matrix \(\Omega\). It is a human-readable confidence score. Later we convert it into \(\Omega\).
The learned confidence formula starts from a skeptical baseline:
\[
\omega_k^{base}=0.40
\]
Then it adds evidence from t-stat, payoff IR, information coefficient, hit rate, sample size, and confluence:
So views with tiny sample size get almost no sample-size bonus, while views with enough history get the full bonus.
The nonlinear \(\tanh\) terms keep a single statistic from pushing confidence to an extreme value. Their bounded response keeps the contribution of each input controlled across views. A very high t-stat helps, but it saturates. A very bad IR hurts, but it also saturates. This keeps confidence in a controlled range.
We also apply safety rules:
if \(n_k<8\), confidence is capped at 0.52,
if payoff IR is weak and hit rate is weak, confidence is capped at 0.46,
if recent hit rate is below 35%, confidence is penalized,
risk-on views are penalized slightly during equity stress,
protective views need stress-period evidence to earn high stress confidence.
We use the realized payoff record to set confidence for each view family. A view with stronger historical outcomes can receive more influence in the posterior, while a weak record reduces the confidence attached to new instances of that family.
So \(\omega_k\) should be read as a model trust score, not as a literal probability that the view will be right next month. It controls how much noise we assign to the view when we build \(\Omega\).
We can compare the learned confidence with the payoff history for each active view.
Dual momentum has confidence around 0.5857, the highest in the latest active set. It has 117 observations, hit rate 57.26%, recent hit rate 100%, and positive payoff IR. We assign it more confidence because it has both a long payoff history and good recent behavior.
Liquidity leadership also receives high confidence around 0.5802, supported by hit rate 62% and positive payoff IR. Risk-adjusted momentum gets confidence around 0.5431, which is moderate and sensible: it has a good sample and positive hit rate, but the information coefficient is negative, so signal magnitude hasn’t translated cleanly into larger payoffs.
International rotation is held down to 0.3309. It has 45 observations, but hit rate is below 50% and payoff IR is negative. We still allow the view into the candidate set, but its lower confidence means it won’t dominate the posterior.
Duration quality is floored at 0.3000 because its history is weak: hit rate around 35.53%, recent hit rate 33.33%, and negative payoff IR. The current state points toward short duration quality, so the view is active, while its weak payoff history keeps the learned confidence low. We need to be careful because this view hasn’t worked well historically.
We generate a view from the current signal rules and then use its realized payoff history to determine how much influence it receives. The signal and its historical reliability therefore enter at different steps.
A strong current signal can receive low confidence when the same view family’s past payoffs have been weak. The signal determines the direction and current strength of the candidate, and the payoff history determines how much confidence we attach to that view.
10.1 View selection and redundancy control
After computing confidence, we still need to select views. If too many overlapping views enter the posterior, the same economic bet can be counted multiple times.
The \(q\)-score measures how large the raw tilt is relative to its cap:
\[
q\text{-score}_k = \frac{|q_{tilt,k}|}{c_k}
\]
The payoff-quality score combines hit rate, recent hit rate, payoff IR, and sample score. Confluence measures how many independent signal dimensions support the view.
Then we remove redundant views using cosine similarity between exposure vectors:
If two views have similarity above 0.85, we keep the stronger one. Dual momentum and risk-adjusted momentum can sometimes point to almost the same long-short basket, so this threshold helps us avoid using the same exposure twice. Without redundancy control, the posterior would double-count the same bet.
We also cap selected views at 7 and limit too many same-direction views. That keeps the posterior from becoming a pile of correlated risk-on bets.
The diversification term gives a small preference to views that bring broader or less repeated exposures. This gives a view with a different combination of long and short assets a chance to add information to the selected set.
From the latest selection table, all five candidate views are kept. The ranking is sensible:
Risk-adjusted momentum, selected score 0.6742.
Dual momentum, selected score 0.6703.
Liquidity leadership, selected score 0.5355.
International rotation, selected score 0.4651.
Duration quality, selected score 0.2759.
Risk-adjusted momentum ranks slightly above dual momentum because its confluence and selection score are strong, even though dual momentum has higher confidence. Duration quality is clearly the weakest selected view, and the score reflects that.
The long/short side of each final view is the part to inspect before the posterior. It lets us directly connect the model’s active ideas to later posterior shifts:
EEM and QQQ appear repeatedly on the long side.
TLT appears repeatedly on the short side.
GLD is short in the momentum views, but later the full posterior can still be affected by covariance interactions.
International rotation directly pushes EEM above EFA, but with low confidence.
These long and short baskets give us the exact active bets that enter the posterior, so we can relate each expected-return shift to the views that were selected.
We can use the selected-view table to check those economic bets and their confidence together. Before seeing the posterior, we can already tell which economic ideas are about to influence the model and which ones are only weakly trusted.
11) Learned \(q\) scaling and the view matrix
Payoff history affects both the view uncertainty matrix \(\Omega\) and the size of the active \(q\) tilt. We now look at how the reliability multiplier changes the tilt before it enters the posterior.
Both the prior spread and the reliability-scaled tilt determine \(q\). If the prior already strongly favors the long side, \(q\) can be high even with a modest active tilt. If the prior favors the short side, the active tilt has to overcome that base spread.
We use learned confidence in two places in the full process: it helps us scale the active tilt itself, and it later helps us build \(\Omega\). The first channel changes the size of the view target. The second channel changes how strongly BL listens to that target.
We can now compare the learned \(q\) values and their components for each of the five selected views.
Risk-adjusted momentum has base view spread about 0.0674, raw q tilt 0.0835, and a positive learned component 0.0083, giving final q tilt 0.0918. Its final view target \(q\) is 0.1592. We start with a prior that already likes this spread, and its view history supports adding a small boost.
Dual momentum is even more interesting. Its base spread is slightly negative at about -0.0103, but the raw tilt is 0.0916, boosted to the cap at 0.1000. The final target \(q\) becomes 0.0897. This means the active view fully overcomes the prior’s slight disagreement.
Liquidity leadership has base spread 0.1270 and final active tilt 0.0393, producing \(q\) around 0.1663. The prior and view agree.
International rotation has base spread 0.1123, but the learned component is negative -0.0501, shrinking final active tilt from 0.0771 to 0.0270. So even though EEM looks strong now, its historical payoff record forces a much smaller active push.
Duration quality is also heavily shrunk. Raw q tilt 0.0168 becomes only 0.0059 because hit rate and payoff IR are weak. The final tilt is roughly 35% of the raw value, close to the multiplier’s 0.35 floor.
We can also check the portfolio exposures in the resulting view matrix. For example, international rotation is simply \(+1\) EEM and \(-1\) EFA. Duration quality is \(+1/3\) SHY, \(+1/3\) AGG, \(+1/3\) IEF, and \(-1\) TLT. This matrix is what lets BL translate relative views into asset-level posterior return shifts.
We can check that the final \(q\) values still match the economic direction of the selected views. The learning layer should shrink or boost the signal; it should not quietly reverse the intended long-short belief.
12) From confidence to \(\Omega\)
Now we convert confidence into the Black-Litterman view uncertainty matrix \(\Omega\).
If we have \(N\) assets and \(K\) views, the return vector \(\boldsymbol{\mu}\) has \(N\) entries, \(P\) has \(K\) rows and \(N\) columns, and \(\mathbf{q}\) contains one expected spread return for each view. We write the view relationship as:
For view \(k\), \(\varepsilon_k\) is the gap between the actual expected return of its long-short portfolio and the stated view target \(q_k\). The vector \(\boldsymbol{\varepsilon}\) collects the errors from all \(K\) views. We model it as:
The matrix \(\Omega\) contains the covariances of those view errors. Its diagonal entry \(\Omega_{kk}\) is the error variance of view \(k\), while an off-diagonal entry describes how the errors of two views move together. A smaller \(\Omega_{kk}\) gives that view more influence in the posterior; a larger value gives it less influence. The entries are variances and covariances of return estimates, so their units are squared returns.
We start from the asset return covariance matrix \(\Sigma\). Multiplying by the view matrix on each side converts the \(N\)-asset covariance into a \(K\)-view covariance matrix:
\[
\Sigma_P = P\Sigma P^\top
\]
Each diagonal element measures the return variance of one long-short view portfolio. If \(\mathbf{p}_k\) is the vector of weights in view \(k\), its variance is:
The off-diagonal element for views \(i\) and \(j\) comes from \(\mathbf{p}_i^\top\Sigma\mathbf{p}_j\). It measures how the returns of their two view portfolios move together, so shared long or short exposures can appear as covariance between view portfolios.
We next use the learned confidence score to scale the uncertainty assigned to each view:
\[
s_k=\sqrt{\frac{1-\omega_k}{\omega_k}}
\]
Here \(\omega_k\) is the learned confidence score for view \(k\). The factor \(s_k\) is positive: a higher confidence reduces \(s_k\), while a lower confidence increases it. Since the matrix formula applies this factor on both sides, the variance of one view is scaled by \(s_k^2=(1-\omega_k)/\omega_k\).
We collect the \(K\) scaling factors in \(\mathbf{s}\), place them on the diagonal of a matrix, and multiply the view covariance on the left and right. With \(\tau=0.05\) setting the overall scale used in this project, the view uncertainty matrix is:
The variance of the view portfolio sets the scale, and the confidence ratio adjusts how much of that scale we assign to uncertainty. For example, if a view portfolio has variance \(0.04\) and \(\tau=0.05\), confidence \(0.50\) gives \(\Omega_{kk}=0.002\). With the same portfolio variance and confidence \(0.80\), the ratio falls to \(0.25\) and \(\Omega_{kk}=0.0005\). The higher-confidence view receives a smaller error variance.
For two different views, the off-diagonal entry is:
Both confidence factors scale the covariance between the two view portfolios. If their exposures move together, their view errors can be correlated as well; opposite exposures can produce a negative covariance. This construction combines:
the market covariance of the view portfolio,
the learned reliability of the view family.
The uncertainty assigned to a view depends on both its portfolio variance and its confidence score. A lower-confidence view can still have a smaller diagonal entry than another view if its underlying portfolio variance is sufficiently lower.
Before the diagonal adjustment, the positive confidence factors change each view’s uncertainty scale while preserving the pairwise correlations implied by \(P\Sigma P^\top\). The next calculation symmetrizes the matrix and adds a very small diagonal adjustment so that the later linear solve stays numerically stable.
For the latest active set, the \(\Omega\) numbers separate two effects: how much we trust the view family and how volatile the long-short view portfolio is.
Dual momentum has confidence 0.5857 and view variance about 0.0164, giving omega diagonal around 0.000580. This is relatively small, so the view receives meaningful weight in the posterior.
Risk-adjusted momentum has confidence 0.5431 and view variance 0.0227, giving omega diagonal around 0.000954. Even though confidence is moderate, the view portfolio variance is larger, so uncertainty is larger than dual momentum.
Liquidity leadership has similar confidence 0.5802, but view variance is higher at 0.0361, so omega diagonal rises to about 0.001307. The larger variance of the long-short portfolio gives this view more uncertainty even though its confidence is similar.
International rotation has low confidence 0.3309 and high view variance 0.0397, producing omega diagonal around 0.004011, the largest in the active set. So even though the current EEM signal is strong, the posterior will treat this view cautiously.
Duration quality has the lowest confidence 0.3000, but the view variance is tiny at 0.0040, so omega diagonal is only 0.000462. Low confidence increases the scaling factor, while the low variance of the underlying long-short portfolio keeps the resulting \(\Omega\) diagonal small. Both quantities enter the uncertainty calculation.
So the largest confidence score doesn’t always create the smallest omega diagonal. The volatility of the view portfolio matters too, which is why the same confidence can lead to different uncertainty across view families.
13) Black-Litterman posterior
Now we combine the equilibrium prior with the selected views. At this point the model has five objects: the prior mean \(\boldsymbol{\pi}\), the prior uncertainty scale \(\tau\Sigma\), the view matrix \(P\), the view target vector \(\mathbf{q}\), and the view uncertainty matrix \(\Omega\).
The prior says that the unknown expected-return vector \(\boldsymbol{\mu}\) is centered around the benchmark-implied return vector \(\boldsymbol{\pi}\):
Here \(\boldsymbol{\pi}\) is not a historical mean. It is the equilibrium return vector implied by the benchmark portfolio, covariance matrix, and risk aversion. The matrix \(\Sigma\) is the asset return covariance matrix, and \(\tau\) scales it down because this is uncertainty about expected returns, not the volatility of realized returns themselves. Smaller \(\tau\) means the prior is tighter and harder for views to move.
With each view, we don’t observe every asset’s expected return directly. We observe a portfolio spread, such as a long-short basket, which is why the view equation uses \(P\boldsymbol{\mu}\):
Each row of \(P\) is one view portfolio. Positive entries are the long side, negative entries are the short side, and zeros are assets not directly named in that view. The vector \(\mathbf{q}\) contains the return target for those view portfolios. The error term \(\boldsymbol{\varepsilon}\) is the part of the view that can be wrong. Its covariance matrix \(\Omega\) tells BL how noisy the views are. A larger diagonal value in \(\Omega\) means that view should pull the posterior less.
Bayes’ rule gives the posterior mean in precision form:
This formula is dense, so the pieces are worth reading slowly. The term \((\tau\Sigma)^{-1}\) is prior precision: a tight prior receives more weight. The term \(P^\top\Omega^{-1}P\) is view precision mapped back from view space into asset space. If \(\Omega\) is small, the views are precise and this term becomes larger. The right-hand side has the same two sources of information: prior precision times prior mean, plus view precision times view target. The inverse on the left turns those combined precision weights back into expected returns.
The same result can be written as a prior plus an adjustment:
This version is easier to connect to the notebook output. The posterior starts at \(\boldsymbol{\pi}\) and then moves only if the selected views disagree with the prior. The disagreement is the view gap:
\[
\mathbf{q}-P\boldsymbol{\pi}
\]
\(P\boldsymbol{\pi}\) is what the prior already believes about each long-short view portfolio. If \(q_k\) is above that prior-implied spread, the view is asking BL to raise expected returns on the long side relative to the short side. If \(q_k\) is below it, the view pushes in the opposite direction. If the gap is near zero, the view adds almost no new information because the prior already says the same thing.
The middle matrix controls how strongly that gap is transmitted into assets:
\(K\) works like a gain matrix. The \(\tau\Sigma P^\top\) part maps view shocks back to individual assets using covariance relationships. The \((P\tau\Sigma P^\top+\Omega)^{-1}\) part scales the shock down when the view portfolio is already uncertain or when learned confidence is low. This is why an asset can move even if it is not explicitly long or short in a view: covariance links it to assets that are in the view.
Next we calculate this shift and compare the prior and posterior returns. The resulting shifts are not yet a portfolio. They are the changes in expected returns after blending the equilibrium prior, the selected views, the covariance matrix, and the learned view confidence. Portfolio weights come one step later, when the optimizer turns these posterior returns into a constrained allocation.
Numerically, the update form is also convenient because the main inverse is in view space, not full asset space. In this notebook, the number of selected views is smaller than the number of assets, so solving the view-space system is cleaner and less fragile.
DBC gets the largest positive shift in the first posterior diagnostic, about +4.55%, because it appears on the long side of both momentum views and starts from a negative prior. QQQ gets +3.89%, IWM gets +2.65%, EEM gets +2.25%, and SPY gets +1.41%. These shifts match the latest risk-on and momentum signals.
GLD receives the largest negative shift, about -6.50%, because it’s short in the momentum views and its recent trend is weak. VNQ also gets a negative shift around -1.91% in that diagnostic because risk-adjusted momentum shorts it, even though dual momentum longs it. This is a good example of views partially conflicting and the posterior averaging them through \(P\), \(q\), and \(\Omega\).
TLT, IEF, AGG, and LQD receive small negative shifts. That aligns with duration quality and weak fixed-income signals, but the shifts are not extreme because some bond exposures remain part of the benchmark and the prior is still tight.
EEM is clipped at 0.3000 in some posterior outputs. That tells us the model has a return cap for numerical stability. The cap prevents extreme posterior expected-return inputs from driving the optimizer toward unusually large allocations.
The posterior shifts are consistent with the selected views. We combine those views with the equilibrium prior and the covariance structure, so the final changes in expected returns reflect all three inputs.
Before looking at weights, this isolates the belief update. If a posterior shift looks reasonable but the final weight doesn’t move much, the reason is usually in the optimizer constraints or covariance risk, not in the view construction.
14) Portfolio optimization from the posterior
Once we have posterior expected returns and posterior covariance, we solve a constrained mean-variance problem:
The active limit keeps each asset weight within 15 percentage points of its benchmark weight. A strong view can change the allocation while the portfolio stays close to the benchmark. If QQQ receives a strong posterior shift, the optimizer can overweight it, but only within the active budget.
We also include practical fallbacks. If the strict optimizer fails, we relax the active limit, remove sleeve constraints, or fall back to a tracking-error style solution. If all else fails, we keep previous weights. In the final BL backtest, we’ll see a zero fallback count across the full backtest, which confirms that the optimization path is stable.
Together, the asset caps, active limits, and sleeve constraints control concentration, distance from the benchmark, and dependence on one part of the market. Large posterior shifts therefore lead to allocations that still respect the structure of the portfolio.
At each monthly rebalance, we repeat the complete Black-Litterman process using the information then available:
take the current prior \(\boldsymbol{\pi}_t\) and covariance \(\Sigma_t\),
generate candidate views from current signals,
use only already-realized payoff history to learn confidence,
select non-redundant views,
build \(P_t\), \(\mathbf{q}_t\), and \(\Omega_t\),
compute \(\boldsymbol{\mu}_{post,t}\) and \(\Sigma_{post,t}\),
optimize weights under constraints,
hold until the next rebalance.
The no-lookahead part is essential. At date \(t\), learned confidence only uses payoffs whose end dates are before \(t\):
\[
\text{payoff\_end\_date}_{k,j} < t
\]
This prevents the confidence system from using future information. Without this condition, the learned confidence would be a form of leakage and the backtest would be invalid.
We store the selected views, confidence logs, posterior shifts, and portfolio weights at each rebalance. These logs are what allow us to analyze the model after the backtest instead of just looking at final performance.
The stored logs are also what make the later reliability analysis possible. Without them, we would only know the final return path, not which views caused each posterior and weight change.
Now we combine the baseline strategies and Learned-Confidence BL into one comparison.
We compare gross and net performance so we can separate trading returns from transaction costs. Gross CAGR ignores costs. Net CAGR includes 10 bps per trade. Cost drag is the total return lost to trading costs.
The BL model has 11.31% gross CAGR and 10.95% net CAGR. The cost drag is much larger than the benchmark because BL turnover is much larger: average monthly turnover is 27.12%, and total turnover is 33.90 over the backtest.
For comparison, the benchmark has 1.44% average monthly turnover and total turnover 1.80. MV has 3.69% average monthly turnover and total turnover 4.65. BL trades much more because views update monthly and the optimizer responds to posterior changes.
After the assumed trading costs, BL still keeps a higher net CAGR than the benchmark over the same backtest period: 10.95% versus 9.14%. So the active model’s extra turnover is expensive, but the gross alpha is large enough to survive the assumed transaction cost.
The model earns more, but it pays for that through turnover. That tradeoff has to be part of the evaluation.
The gross-to-net comparison includes the costs of implementing the changing allocations and their compounding through the return path. Average monthly turnover describes the typical amount traded at a rebalance, while total turnover accumulates those position changes across the backtest.
Show code
combined_results = {}combined_results.update(baseline_results)combined_results.update(bl_results)final_strategy_names = ["Strategic Benchmark", "Equal Weight", best_minvar, best_mv, "Learned-Confidence BL"]returns_by_strategy = {name: combined_results[name]["returns"] for name in final_strategy_names}nav_by_strategy = {name: combined_results[name]["nav"] for name in final_strategy_names}weights_by_strategy = {name: combined_results[name]["weights"] for name in final_strategy_names}turnover_by_strategy = {name: combined_results[name]["turnover"] for name in final_strategy_names}cost_by_strategy = {name: combined_results[name]["costs"] for name in final_strategy_names}def cagr_from_nav(nav): nav = pd.Series(nav).dropna()if nav.empty:return np.nan years =max(len(nav) / annualization, 1e-9)returnfloat(nav.iloc[-1] ** (1.0/ years) -1.0)gross_net_rows = []for name in final_strategy_names: result = combined_results[name] gross_net_rows.append({"strategy": name, "Gross CAGR": cagr_from_nav(result.get("gross_nav", result["nav"])),"Net CAGR": cagr_from_nav(result["nav"]),"Cost Drag": float(result["costs"].sum()) iflen(result["costs"]) else np.nan,"Average Monthly Turnover": float(result["turnover"].mean()) iflen(result["turnover"]) else np.nan,"Total Turnover": float(result["turnover"].sum()) iflen(result["turnover"]) else np.nan})gross_net_table = pd.DataFrame(gross_net_rows).set_index("strategy")display(gross_net_table.round(4))def full_strategy_metrics(name, result): metrics = selection.performance_metrics(result["returns"], result["nav"], rf_daily=risk_free_daily, annualization=annualization) weights = result["weights"] eff_n = (float((1.0/ weights.pow(2).sum(axis=1)).replace([np.inf, -np.inf], np.nan).mean())ifnot weights.empty else np.nan) avg_max_weight =float(weights.max(axis=1).mean()) ifnot weights.empty else np.nanreturn {"strategy": name, "CAGR": metrics["CAGR"], "Annualized Volatility": metrics["Vol"],"Sharpe": metrics["Sharpe"], "Sortino": metrics["Sortino"],"Max Drawdown": metrics["Max Drawdown"], "Calmar": metrics["Calmar"],"Avg Monthly Turnover": float(result["turnover"].mean()) iflen(result["turnover"]) else np.nan,"Total Turnover": float(result["turnover"].sum()) iflen(result["turnover"]) else np.nan,"Cost Drag": float(result["costs"].sum()) iflen(result["costs"]) else np.nan,"Effective Number of Holdings": eff_n, "Average Max Weight": avg_max_weight,"Fallback Count": result.get("fallback_count", 0)}summary_table = pd.DataFrame([full_strategy_metrics(name, combined_results[name]) for name in final_strategy_names] ).set_index("strategy")display(summary_table.round(4))def avg_active_risk_against_benchmark(weights): rows = [] bench = benchmark_w.reindex(assets).fillna(0.0)for dt, row in weights.iterrows(): dt = pd.Timestamp(dt)if dt notin cov_by_date:continue diff = row.reindex(assets).fillna(0.0).values - bench.values cov_ann = cov_by_date[dt].loc[assets, assets].values rows.append(float(np.sqrt(max(diff @ cov_ann @ diff, 0.0))))returnfloat(np.mean(rows)) if rows else np.nandef benchmark_relative_metrics(name): bench_ret = pd.Series(combined_results["Strategic Benchmark"]["returns"]).dropna() ret = pd.Series(combined_results[name]["returns"]).reindex(bench_ret.index).fillna(0.0) active = ret - bench_ret active_nav = (1.0+ active).cumprod() tracking_error =float(active.std(ddof=1) * math.sqrt(annualization)) if active.std(ddof=1) >1e-12else np.nan active_return =float(active.mean() * annualization) info_ratio = active_return / tracking_error if np.isfinite(tracking_error) and tracking_error >1e-12else np.nan corr =float(ret.corr(bench_ret)) if ret.std(ddof=1) >1e-12and bench_ret.std(ddof=1) >1e-12else np.nan beta =float(ret.cov(bench_ret) / bench_ret.var(ddof=1)) if bench_ret.var(ddof=1) >1e-12else np.nan weights = combined_results[name]["weights"].reindex(columns=assets).fillna(0.0) bench = benchmark_w.reindex(assets).fillna(0.0) avg_active_dist =float(weights.subtract(bench, axis=1).abs().sum(axis=1).mean()) ifnot weights.empty else np.nanreturn {"strategy": name, "active_return_ann": active_return,"tracking_error": tracking_error, "information_ratio": info_ratio,"active_max_drawdown": float((active_nav / active_nav.cummax() -1.0).min()),"hit_rate_vs_benchmark": float((active >0).mean()),"corr_to_benchmark": corr, "beta_to_benchmark": beta,"avg_active_weight_distance": avg_active_dist,"avg_active_risk": avg_active_risk_against_benchmark(weights)}benchmark_relative_table = pd.DataFrame([benchmark_relative_metrics(name) for name in final_strategy_namesif name !="Strategic Benchmark"]).set_index("strategy")display(benchmark_relative_table.round(4))
Gross CAGR
Net CAGR
Cost Drag
Average Monthly Turnover
Total Turnover
strategy
Strategic Benchmark
0.0916
0.0914
0.0026
0.0144
1.7991
Equal Weight
0.0849
0.0848
0.0022
0.0114
1.4385
MinVar (OAS)
0.0665
0.0665
0.0005
0.0031
0.3897
MV (LedoitWolf, BayesStein)
0.0819
0.0814
0.0062
0.0369
4.6481
Learned-Confidence BL
0.1131
0.1095
0.0602
0.2712
33.8951
CAGR
Annualized Volatility
Sharpe
Sortino
Max Drawdown
Calmar
Avg Monthly Turnover
Total Turnover
Cost Drag
Effective Number of Holdings
Average Max Weight
Fallback Count
strategy
Strategic Benchmark
0.0914
0.1014
0.5284
0.6493
-0.2242
0.4075
0.0144
1.7991
0.0026
9.7136
0.2222
0
Equal Weight
0.0848
0.0935
0.4842
0.5934
-0.2038
0.4159
0.0114
1.4385
0.0022
14.0000
0.0714
0
MinVar (OAS)
0.0665
0.0754
0.3543
0.4044
-0.1973
0.3370
0.0031
0.3897
0.0005
9.1680
0.2394
0
MV (LedoitWolf, BayesStein)
0.0814
0.0847
0.4949
0.6049
-0.1378
0.5904
0.0369
4.6481
0.0062
5.0557
0.3212
0
Learned-Confidence BL
0.1095
0.1066
0.6613
0.8336
-0.2154
0.5083
0.2712
33.8951
0.0602
5.8309
0.2660
0
active_return_ann
tracking_error
information_ratio
active_max_drawdown
hit_rate_vs_benchmark
corr_to_benchmark
beta_to_benchmark
avg_active_weight_distance
avg_active_risk
strategy
Equal Weight
-0.0083
0.0181
-0.4581
-0.1088
0.4774
0.9860
0.9094
0.4560
0.0170
MinVar (OAS)
-0.0268
0.0455
-0.5893
-0.2614
0.4705
0.9086
0.6764
0.7007
0.0400
MV (LedoitWolf, BayesStein)
-0.0116
0.0812
-0.1432
-0.2863
0.4858
0.6317
0.5277
1.1509
0.0687
Learned-Confidence BL
0.0169
0.0402
0.4218
-0.0667
0.5295
0.9266
0.9746
0.9059
0.0369
Learned-Confidence BL is the strongest model in the full strategy metrics table.
Learned-Confidence BL has:
10.95% CAGR, highest among all tested strategies,
10.66% volatility, only slightly above the strategic benchmark,
0.6613 Sharpe, clearly above the benchmark’s 0.5284,
0.8336 Sortino, also stronger than the benchmark,
-21.54% max drawdown, slightly better than the benchmark’s -22.42%,
0 fallback count, meaning the optimizer didn’t rely on emergency behavior.
BL improves return and risk-adjusted performance while its drawdown is slightly better than the benchmark’s. MV has the lowest drawdown at -13.78%, alongside a CAGR of 8.14% and a Sharpe ratio below BL’s. The models therefore differ in both their return paths and the balance they strike between return and risk.
The effective number of holdings is also informative. BL averages around 5.83 effective holdings, while the benchmark has 9.71 and equal weight has 14. So BL is more concentrated. That concentration is the cost of making active tilts. The active weight and sleeve constraints prevent it from becoming extreme, but the model is clearly taking directional relative bets.
The higher return and Sharpe come alongside a more concentrated active allocation and higher turnover at monthly rebalances. After the assumed costs, BL still has the higher return, and its maximum drawdown is slightly better than the benchmark’s over the same period.
For a fully invested portfolio, the effective number of holdings is \(1/\sum_i w_i^2\). The squared weights give more influence to large positions in the concentration measure, so the effective number falls when the allocation is concentrated even if the portfolio holds several other assets at smaller weights.
15.1 Benchmark-relative metrics
Absolute performance can hide whether the active model actually adds value over the benchmark. So we also compute active return, tracking error, information ratio, hit rate versus benchmark, correlation, beta, and active risk.
The BL model has annualized active return around 1.69%, tracking error around 4.02%, and information ratio around 0.4218. That is a meaningful active result. The monthly hit rate versus benchmark is 52.95%, so the model wins slightly more often than it loses.
The correlation to benchmark is 0.9266 and beta is 0.9746, so BL continues to move closely with the benchmark. Its active return comes from the tilts around that multi-asset portfolio, while the broad benchmark exposure remains in the allocation throughout the period measured by this comparison.
The information ratio divides active return by tracking error, so the value of 0.4218 puts the model’s annualized excess return in the context of its benchmark-relative volatility. We can read it alongside beta and correlation to see both the size of the active result and how closely the portfolio follows the benchmark.
BL finishes above the benchmark, equal weight, MinVar, and MV. The strategy doesn’t dominate every period, but it compounds better over the full sample. BL still participates in major stress events, which is expected because it remains long-only and benchmark-aware. It isn’t a market-neutral strategy.
The drawdown difference versus the benchmark isn’t huge. The BL model still falls during COVID and the 2022 regime, while its recovery and subsequent compounding are stronger. The difference in their return paths becomes more visible as the portfolios recover and continue to compound after those declines.
The main weakness visible from the path is turnover and responsiveness. Because we update the allocation monthly, our response can lag sudden events. A shock like COVID can hit before signals have time to adjust. But after the shock, we can use the updated signals to rotate the allocation faster than a static benchmark.
The path is consistent with the table: BL adds value over the full sample, but it remains exposed to broad market risk.
We can read the drawdown path together with turnover and active risk to judge the complete allocation. The goal is to rotate the benchmark weights enough to improve the return path while keeping the active exposure under control.
15.2 Tail risk and stress windows
The risk-report table compares daily 5% VaR and expected shortfall. Historical VaR is the empirical 5th percentile of returns:
\[
VaR_{5\%}^{hist}= -Q_{0.05}(r_t)
\]
Expected shortfall is the average loss beyond VaR:
The filtered historical simulation values are larger because they account for volatility clustering. BL has historical VaR around 1.02% and historical ES around 1.62%. Its FHS VaR is around 1.97% and FHS ES around 2.98%, higher than the benchmark. That tells us the active tilts increase conditional tail exposure in volatile regimes.
BL improves return and Sharpe, while MinVar has the lower VaR and ES among these strategies. The strategic benchmark sits between them on these tail-risk measures, so the allocation we prefer depends on how we weigh the return improvement against the left-tail losses.
We compare the stress windows because each one captures a different market regime:
2018 Q4 selloff, Fed tightening and equity derisking.
COVID crash, sudden exogenous shock.
2022 rates/inflation, stock-bond correlation failure and duration crash.
2023 growth rebound, recovery led by growth assets.
BL is designed to adapt to regimes, but it can’t react before a sudden shock appears in the data. Its strength is more visible in slower regime shifts like 2022 inflation and 2023 growth recovery.
We optimized posterior mean-variance under allocation constraints, so the resulting weights reflect both expected returns and covariance risk. MinVar can still look better on pure tail-risk measures because the optimization objectives place different emphasis on risk and return.
16) Selected-view reliability after the full backtest
After the full BL backtest, we analyze the selected views alongside the candidates. The rules generate candidate views, and the selected views are the ones that actually entered the posterior at each rebalance. We can then count how often each view influenced the resulting estimates.
Most candidate views survive selection, and the selected count is usually close to the candidate count. Redundancy and direction caps remove some views when their baskets overlap or when too many views point in the same direction, so the selected set can be smaller than the candidate set.
From the selected-view counts, we can see how often each view actually affected the posterior. Risk-adjusted momentum is selected 124 times, almost every rebalance. Dual momentum is selected 106 times. Duration quality is selected 77 times. Liquidity leadership, international rotation, credit switch, growth-duration, reflation, inflation, and correlation stress are less frequent.
The selected-share column helps us read the model’s personality. Risk-adjusted momentum has selected share 0.992, dual momentum 0.848, and duration quality 0.616. We’re therefore relying heavily on momentum for the active allocation, with fixed-income regime controls layered on top.
The frequent momentum views give the allocation a persistent trend-following component across assets. Momentum can also lag a reversal when the recent leaders change quickly. Learned confidence adjusts the influence of a view using its payoff history, while the underlying signal still follows its lookback window.
Selected counts show which candidate views actually enter the posterior after the redundancy and direction limits are applied. A family that fires often but is frequently removed has less influence on the posterior than its candidate count alone would suggest.
For the selected views, we can compare realized hit rates, average forward payoffs, and payoff IR across the view families.
Dual momentum has realized hit rate 60.00%, average forward payoff around 6.50% annualized, and payoff IR 0.1727. This is one of the strongest selected views because it has both sample size and positive realized performance.
Credit switch: risk-on is very strong among selected views: hit rate 65.12%, average payoff 15.04%, and payoff IR 0.3733. This supports the idea that credit leadership is a useful signal for risk-on allocation.
Reflation breadth has hit rate 71.43% and payoff IR 0.3905, but only 21 selected observations. This is promising, but less mature than the momentum views.
Liquidity leadership has hit rate 62% and payoff IR 0.1297, making it a useful but moderate view.
Weak selected views include Duration quality, International rotation, Inflation rotation, and Credit switch: risk-off. Duration quality has hit rate 35.53% and negative payoff IR. International rotation has hit rate 44.44% and negative payoff IR. Credit switch risk-off has very poor realized profile. These weak results explain why the learned-confidence system shrinks or floors these views.
These differences in realized performance are why we use learned confidence for the views. Without it, all selected views could be treated too similarly. With it, historically useful views get more influence, while weak views still exist but have lower posterior impact.
These view families can still describe specific economic regimes even when their overall payoff histories are weak. The learned-confidence rule keeps them available when their conditions appear, while giving their current targets less influence than families with stronger realized payoff records.
16.1 Posterior shift heatmap and latest posterior table
For the latest logged rebalance, we compare the benchmark weight, prior return, posterior return, and posterior shift for each tradable asset in the portfolio. The shift is:
\[
\Delta\mu_i = \mu_{post,i} - \pi_i
\]
The shift gives us the change in the expected return of each individual asset after the selected views have been combined with the prior.
In the latest full-log table, EEM receives a very large positive shift, about +9.86%, DBC about +9.76%, EFA about +9.47%, GLD about +11.84%, QQQ about +7.76%, and SPY about +3.94%. Some of these are direct view effects, while others come from covariance propagation through the posterior formula.
Across time, we can use these shifts to compare changes in expected-return beliefs with the final weights chosen by the optimizer at each rebalance date. A posterior shift can be strong, but the optimizer may still limit the final allocation because of caps, active weight limits, sleeve constraints, or covariance risk.
We can see visible regime blocks in the heatmap of posterior shifts. Around 2022, real assets and inflation-sensitive exposures receive stronger positive shifts, while duration is penalized. In growth-led periods, QQQ and equity-linked assets receive positive shifts. The shifts aren’t random noise. They cluster around economic regimes.
We can also see frequent sign changes for some assets, especially where momentum views and covariance effects interact. Those month-to-month shifts reflect how the signals and covariance estimates move as the allocation responds to changing market conditions.
Covariance propagation can move an asset’s posterior return even when that asset isn’t on either side of a particular view. The model updates the full expected-return vector jointly, using the covariance between the asset and the long-short view portfolios.
For the full walk-forward backtest, we bring together candidate counts, selected counts, payoff statistics, average confidence, stress-period behavior, and selected share. This gives us the history of each view family in one place.
A few rows stand out:
Risk-adjusted momentum: 125 candidates, 124 selected, hit rate 57.26%, average confidence 0.4515.
Dual momentum: 118 candidates, 106 selected, hit rate 57.26%, recent hit rate 100%, average confidence 0.4238.
Reflation breadth: hit rate 71.43%, payoff IR 0.3905, average confidence 0.6451, but smaller sample.
Correlation stress: hit rate 75% and high payoff IR, but only 4 observations, so it has limited statistical reliability.
Duration quality and International rotation: weak hit rates and negative payoff IR, so the confidence layer keeps them small.
After optimization, QQQ is at the active cap with +15.00% active weight. EFA is +10.91%, EEM +9.95%, SHY +9.95%, GLD +4.95%, and DBC +2.98%. The biggest underweights are SPY -12.97%, AGG -10.10%, TLT -8.08%, IEF -8.08%, and LQD -5.05%.
The latest weights give us a relative allocation across growth, international equity, real assets, bonds, and credit:
overweight QQQ, EEM, EFA, GLD, DBC, and SHY,
underweight broad SPY, core bonds, long duration, and credit,
keep active risk controlled around the benchmark.
Average active risk is around 3.69% and average turnover is 27.12%. That is a moderate active-risk profile but a high-turnover profile. For a monthly tactical model, that is believable, but costs need to be monitored.
We can compare these active weights with the selected views to see how their expected-return shifts were translated into positions. Asset caps, sleeve constraints, active limits, and covariance risk all affect the final allocation produced by the optimizer.
17) Sector ETF transfer with quantfinlab
Now we move the learned-confidence Black-Litterman workflow to sector ETFs through the packaged quantfinlab library. The decision problem changes here. In the main cross-asset universe, we can rotate between equities, bonds, credit, commodities, gold, and cash-like duration. In the sector universe, every tradable asset is still an equity ETF in the portfolio, so we’re choosing which parts of the stock market should lead or lag.
The sector data comes from the same repository data area: sector and cross-asset ETF data. The tradable universe is the 11 SPDR sector ETFs:
ETF
Sector
Usual behavior
Main drivers we care about here
XLK
Technology
Growth-heavy, high-duration equity exposure; often leads when mega-cap growth and risk appetite are strong.
QQQ leadership, falling or stable rates, strong momentum, benign volatility.
XLC
Communication services
Mix of platform, media, and telecom exposure; behaves partly like growth and partly like consumer/advertising cyclicality.
We can group the sectors into growth, cyclical, inflation-sensitive, and defensive exposures. XLK, XLC, and XLY are the growth/consumer-risk side. XLI, XLF, XLB, and XLE are more cyclical or reflation-sensitive. XLV, XLP, and XLU are defensive, although they still fall in broad equity selloffs. XLRE sits between real assets and rate-sensitive equity because property cash flows depend heavily on financing costs.
This makes the sector transfer more subtle than the main allocation problem. A defensive sector overweight is not the same as buying SHY or IEF. It only means we prefer defensive equities relative to other equities. If the whole stock market sells off, sector rotation can reduce damage, but it usually can’t remove equity beta completely.
The signal assets still include cross-asset indicators like SPY, QQQ, IWM, TLT, IEF, SHY, LQD, HYG, GLD, DBC, and UUP. That means we trade sectors, but we still use broad macro signals to decide which sector views make sense. QQQ/SPY leadership helps growth views. IWM/SPY and HYG/LQD help cyclical and credit-beta views. TLT, IEF, and SHY help rate-sensitive views. DBC and GLD help inflation and real-asset views. UUP helps identify dollar pressure, which can matter for commodities, materials, and multinational earnings.
The benchmark is equal weight:
\[
w_{b,i}=\frac{1}{11}\approx 0.0909
\]
This is a natural starting point for sector rotation because the sectors all belong to the same broad equity market. There isn’t an obvious strategic reason to heavily overweight one sector before signals arrive. Equal weight also makes the active bets easy to read: if XLK is above 9.09%, the model is explicitly saying technology should carry more than its neutral sector share.
We repeat the learned-confidence workflow with the extracted quantfinlab functions on this related but narrower universe. The sector application uses the same portfolio construction and view-confidence ideas with inputs and view families designed for equity sectors. If the method only works in the original cross-asset setup, it is less convincing. If it still gives reasonable sector rotations, then the view/confidence framework is more portable.
Sector coverage is clean: 2001 observations for all tradable and signal assets from 2018-07-02 to 2026-06-17, with no missing values after first valid. The sector return panel has 2000 daily observations, and the monthly rebalance sequence starts on 2020-07-31 because we need a 504-day lookback.
This shorter history is the main difference from the cross-asset test. The cross-asset model starts its active backtest around 2016. The sector transfer test effectively starts around mid-2020. That means the sector results are useful, but they are based on fewer regimes.
XLK is the strongest tradable sector in the latest signal table, with score 1.2947. XLI, XLE, and XLB are also positive. XLC and XLU are the weakest sectors, with scores around -0.6992 and -0.6984. XLY is also weak at -0.5827.
All tradable sector assets share equity beta, so we’re deciding which equity sectors should receive more or less weight. In the earlier cross-asset universe, the allocation could also move between equities, bonds, commodities, and cash-like assets; here the active decisions are rotations within the equity market.
Because all tradable assets are sectors, we can’t express a broad defensive move by buying Treasuries directly in this portfolio. Defensive information can still enter through signal assets, but the final trade has to be a sector rotation.
17.1 Sector view families
With the library, we use the following sector view families for this transfer to the sector universe. The logic is similar to the main project, but the economic interpretation is sector-specific.
The sector view families used in this application are:
Sector momentum, long the strongest sector momentum/volatility scores and short the weakest.
Growth leadership, long XLK, XLC, XLY when growth leadership is confirmed.
Defensive rotation, long XLP, XLU, XLV when equity stress is present.
Cyclical breadth, long XLI, XLF, XLB, XLE when participation is broad.
Credit beta, long credit-sensitive sectors when high yield confirms risk-on conditions.
Inflation beneficiaries, long XLE and XLB when commodities and inflation-sensitive assets lead.
Duration sensitive, long rate-sensitive sectors when duration conditions are supportive.
Small-cap risk-on, long domestic cyclicals when IWM/SPY leadership is positive.
Quality defensive, long XLV and XLP when volatility and defensive quality dominate.
Sector reversal, a short-horizon mean-reversion view after large sector extremes.
The latest active candidate views are growth leadership, cyclical breadth, credit beta, small-cap risk-on, and sector reversal. Growth leadership goes long XLK, XLC, XLY and short XLP, XLU, XLV. Cyclical breadth goes long XLI, XLF, XLB, XLE and short the same defensive basket. Sector reversal is different: it goes long XLE and XLC while shorting XLK and XLI.
The odd-looking number is cyclical breadth raw q around 0.6488, much larger than the other sector view tilts. That happens because the sector view specification has a much larger cap for this family. This doesn’t automatically mean the final portfolio goes all-in. The BL posterior, confidence, optimizer caps, and constraints still control the final weights.
The sector views express their long and short positions within the equity market. Their signals still use cross-asset information, but the portfolio responds by changing the relative weights of the sectors rather than buying the signal assets themselves.
17.2 Sector baselines and BL backtest
The sector grid selects MinVar with Ledoit-Wolf covariance and MV with EWMA covariance plus Bayes-Stein means. The selected sector baselines are:
The sector BL weight matrix has 71 monthly weight rows across 11 sectors. Candidate views total 310, and selected views total 280. That means about 90% of candidate sector views survive selection, similar to the main project.
The latest sector weights are concentrated. XLC is 23.18%, XLK 19.28%, XLY 17.92%, XLE 11.58%, XLB 9.60%, XLRE 9.51%, and XLU 7.84%. XLV, XLP, and XLF are at 0%, and XLI is very small.
This is a strong growth and cyclical allocation, with additional energy and materials exposure. The defensive sectors receive little weight in the latest portfolio. The latest active weights make this clearer: XLC is +14.09%, XLK +10.19%, XLY +8.83%, while XLV, XLP, and XLF are each -9.09% relative to equal weight.
The sector model is more aggressive than the cross-asset model because it can’t move into bonds or cash-like assets. All tradable assets are equity sectors, so active views express themselves as sector concentration.
In the sector allocation, the model expresses its views through overweights and underweights within equities. The cross-asset allocation can also move capital between different sleeves, whereas this portfolio remains invested in equity sectors even when the signals favor defensive positioning.
17.3 Sector performance and reliability
Learned-Confidence BL has the best overall profile among the sector strategies:
Benchmark: 14.60% net CAGR, 14.75% volatility, 0.7317 Sharpe, -19.25% drawdown.
MinVar: 9.59% net CAGR, 12.84% volatility, 0.4753 Sharpe, -18.70% drawdown.
MV: 13.51% net CAGR, 15.13% volatility, 0.6443 Sharpe, -16.38% drawdown.
This is a more modest improvement than the main cross-asset application, but it’s still positive. BL improves CAGR and Sharpe over the equal-weight sector benchmark, while also improving drawdown from -19.25% to -16.57%.
The active metrics are smaller than they look from turnover alone. BL active return is 0.62%, tracking error 5.30%, and information ratio 0.1169. The hit rate versus benchmark is only 50.27%. So sector BL adds value through rotation, with a modest active return relative to the tracking error taken by the portfolio over the sector backtest.
The sector view reliability table confirms this mixed picture. Growth leadership is strong, with hit rate 64.86%, average payoff 12.43%, payoff IR 0.2222, and average confidence 0.5581. Credit beta is also useful, with hit rate 58.33% and average confidence 0.5841. Duration sensitive is frequent but weak, with hit rate 48.89% and payoff IR near zero. Quality defensive and sector reversal are poor, with negative payoff IR and low average confidence.
Selection keeps 280 views out of 310 candidates, a 90.32% kept share, with 30 views removed by redundancy gating. Average selected views are about 3.94 per rebalance. That tells us the library implementation is applying the same redundancy logic rather than simply accepting every sector view.
The stress table is the final result here. In 2022 rates/inflation, the sector BL return is -9.13%, much better than the benchmark’s -18.20% and close to MV’s -7.30%. In the 2023 growth rebound, BL earns 13.19%, very close to the benchmark’s 13.79% and much better than MV’s 2.91%. In the 2024-2025 cycle, BL lags the benchmark and MV, with 28.10% versus benchmark 31.82% and MV 38.23%.
So the sector transfer works, but with a realistic tradeoff. BL helps in the 2022 sector-rotation stress regime and keeps up in the 2023 rebound, but it doesn’t dominate the 2024-2025 cycle. That is a good ending point for this project: learned-confidence BL is useful and portable, but its edge depends on the universe, the regime, and whether the selected views continue to earn their confidence.
We can use the same approach outside the original universe, but the smaller sector edge depends more on the regime. The narrower set of tradable assets also limits how much we can diversify away from equity risk during stressed periods.
