clean the ETF data and understand the market instruments.
build a strategic benchmark and the covariance estimates needed for portfolio risk.
introduce the Black-Litterman prior and define views.
build the view matrix, learn confidence from realized historical payoffs, and finally convert the posterior expected returns into portfolio weights.
repeat the same idea on sector ETFs through the library implementation.
1) mathematic background: Bayes Rule
Black-Litterman is easier to understand if we start with this topic. Imagine we have an initial belief about something uncertain. Then we observe new evidence. We should not throw away the initial belief completely, and we should not ignore the evidence either. We should update the belief.
That is the whole Bayesian idea. The prior is what we believed before the new information. The likelihood is how compatible the new information is with possible states of the world. The posterior is what we believe after combining both.
\[
P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}
\]
where \(P(A)\) is the prior (what we believed before seeing \(B\)), \(P(B \mid A)\) is the likelihood (how compatible the evidence \(B\) is with belief \(A\)), and \(P(A \mid B)\) is the posterior (our updated belief after seeing \(B\)). Written proportionally:
Or we can write it like this: \[
\mathrm{posterior\ belief}\propto\mathrm{prior\ belief}\times\mathrm{new\ evidence}
\]
Here, the uncertain object is not a class label or a coin probability. It is the vector of expected returns. Expected returns are hard to estimate, so we don’t want to trust historical averages too much. We can first define a benchmark that gives us a starting point or the prior. Then we use some views about the financial markets and economic situations that will have effect on expected returns. views give us new information. Then there is confidence for each view which decides how much the views should move us away from the benchmark.
So the process is: - start with a reasonable neutral belief - observe the new evidence - update the belief considering the confidence.
For this project, we could’ve used the same nasdaq data from notebook 2, but that wouldn’t be the best choice for Black-Litterman and the economic interpretation and building views wouldn’t be as good. So we use a dataset of famous ETFs (investement funds that are tradable like exchanges) which is collected from yahoo-finance API. A script for reproducing the dataset is available in data folder ([from here]). The dataset contains history of these ETFs from the start of 2010, but we use the data from the start of 2013 cause we don’t want to deal with missing prices of one of the ETFs. [from here]: https://github.com/ramtin-asadi/Quantitative-Finance-Lab/tree/main/data/core_cross_asset_etfs
Show code
raw = pd.read_csv("../data/core_cross_asset_etfs.csv")raw["date"] = pd.to_datetime(raw["date"], errors="coerce")raw = raw.dropna(subset=["date"]).sort_values("date")raw = raw.drop_duplicates(subset=["date"], keep="last").set_index("date")def extract_yfinance_field(raw, field, drop_empty=True): suffix =f"__{field}" cols = [col for col in raw.columns ifstr(col).endswith(suffix)] frame = raw[cols].apply(pd.to_numeric, errors="coerce").replace([np.inf, -np.inf], np.nan) frame.columns = [col.rsplit("__", 1)[0] for col in cols]if drop_empty: frame = frame.dropna(axis=1, how="all")return frame.reindex(columns=sorted(frame.columns))close_all = extract_yfinance_field(raw, "close")volume_all = extract_yfinance_field(raw, "volume")dividends_all = extract_yfinance_field(raw, "dividends", drop_empty=False)splits_all = extract_yfinance_field(raw, "stock_splits", drop_empty=False)def action_frame(frame, field): out = frame.copy() out.columns = pd.MultiIndex.from_product([out.columns, [field]])return outactions_all = pd.concat([action_frame(dividends_all, "dividends"), action_frame(splits_all, "stock_splits")], axis=1).sort_index(axis=1)
The loaded data has \(4107\) trading days and \(63\) adjusted close columns before we select the project’s ETFs. We are not going to use all \(63\) ETFs and most of them are for later projects and secondary data for this project like sector ETFs.
2.1 Choosing the ETFs
We now choose the ETFs we are going to analyze for this notebook. The tradable assets are:
Ticker
Role
What it represents
SPY
U.S. equity core
Large-cap US equities, commonly used as the broad US stock-market proxy.
QQQ
U.S. growth leadership
Nasdaq-100, usually more growth and technology heavy than SPY.
IWM
U.S. small caps
Russell 2000 small-cap US equities, more cyclical and domestically sensitive.
EFA
Developed international equity
Developed international equities outside North America.
EEM
Emerging markets equity
Higher-risk Emerging-market equities, usually more sensitive to global growth, dollar strength, and commodity cycles.
TLT
Long duration Treasuries
Long-duration US Treasuries (20+ years), very sensitive to changes in long-term interest rates.
IEF
Intermediate Treasuries
Intermediate Treasury (7-10 year), less duration risk than TLT.
SHY
Short Treasuries
Short-duration Treasury (1-3 year) and a defensive cash-like sleeve. Usually low volatility and low return.
AGG
Core bonds
Broad US aggregate bond, a core bond allocation.
LQD
Investment-grade credit
corporate bonds, combining rate exposure and credit exposure.
HYG
High-yield credit
High-yield corporate bonds, more equity like and more sensitive to credit conditions.
GLD
Gold
Holds physical gold, often treated as a real asset or crisis/inflation hedge.
VNQ
Real estate
US real estate investment trusts, rate-sensitive and equity-sensitive.
DBC
Commodities
Broad commodities futures, often connected to inflation and global real activity.
UUP
Dollar signal only
US dollar, used as an information variable rather than a tradable portfolio asset here.
2.2 behaviour of these ETFs
SPY, QQQ, and IWM are all equity ETFs, but they do not behave the same. SPY is the broad large cap core. QQQ is more growth and technology heavy, so it usually benefits when investors reward long duration growth cash flows, innovations, and cap leadership. IWM is small cap exposure, so it tends to be more sensitive to growth and financing conditions
EFA and EEM give international exposure. EFA is developed market equity, while EEM is emerging market equity. Emerging markets are often more sensitive to dollar strength, global liquidity and commodity cycles. That is why UUP is useful as a signal asset. If the dollar is strengthening aggressively, emerging markets can face pressure even if they look attractive based on their local currency.
The bond ETFs: SHY is short Treasury exposure, IEF is intermediate Treasury exposure, and TLT is long Treasury exposure. As we can remembe from the notebook 1 and fixed income risk measures implementation, The longer the duration, the more sensitive the ETF is to interest rate changes. If rates rise sharply, long duration bonds can suffer large falls in their value. If rates fall during a growth scare, duration can help stabilize the portfolio.
Credit ETFs sit between bonds and equities. LQD is investment grade credit, while HYG is high yield credit. HYG is especially important because it can behave like a risky asset. In calm markets, high yield can earn carry. In stress markets, it can sell off with equities because credit spreads widen.
GLD, VNQ, and DBC have real asset and inflation sensitive exposures. Gold can behave as a diversifier when confidence in financial assets weakens and the situation is uncertain, although it is not a perfect hedge. VNQ is real estate and is sensitive to both growth and rates. DBC is broad commodities and can perform well during inflationary commodity cycles, but it can also be very painful in long disinflationary periods.
This financial interpretation matters because the later views are not arbitrary and we use these relations to build economically reasonable views.
Show code
start_date ="2013-01-01"assets = ["SPY", "QQQ", "IWM", "EFA", "EEM", "TLT", "IEF","SHY", "AGG", "LQD", "HYG", "GLD", "VNQ", "DBC"]signal_only_assets = ["UUP"]sleeve_map = {"SPY": "US equity", "QQQ": "US equity", "IWM": "US equity","EFA": "International equity", "EEM": "International equity","TLT": "Duration bonds", "IEF": "Duration bonds", "SHY": "Short bonds","AGG": "Core bonds", "LQD": "Credit", "HYG": "Credit","GLD": "Real assets", "VNQ": "Real assets", "DBC": "Commodities","UUP": "Dollar"}close_after_start = close_all.loc[pd.Timestamp(start_date):].copy()missing_assets = [ticker for ticker in assets if ticker notin close_after_start.columns]missing_signal_assets = [ticker for ticker in signal_only_assets if ticker notin close_after_start.columns]if missing_assets or missing_signal_assets:raiseValueError(f"missing ETF close columns: tradable={missing_assets}, signal_only={missing_signal_assets}")filter_rows = []for ticker in assets + signal_only_assets: px = close_after_start[ticker] valid = px.dropna() first_valid = px.first_valid_index() missing_pct = np.nan if first_valid isNoneelsefloat(px.loc[first_valid:].isna().mean()) filter_rows.append({"ticker": ticker, "observations": len(valid),"first_date": valid.index.min(), "last_date": valid.index.max(),"missing_pct_after_first_valid": missing_pct,"tradable": ticker in assets})display(pd.DataFrame(filter_rows).round(4))first_all_valid = close_after_start[assets].dropna(how="any").index.min()if pd.isna(first_all_valid):raiseValueError("no date where all tradable ETF close columns are available")close = close_after_start.loc[first_all_valid:, assets].ffill(limit=3).dropna(how="any")returns = prices_to_returns(close, kind="simple").replace([np.inf, -np.inf], np.nan).dropna(how="all").fillna(0.0)close = close.reindex(returns.index).ffill(limit=3)signal_tickers =list(dict.fromkeys(assets + signal_only_assets))signal_close = close_after_start.loc[first_all_valid:, signal_tickers].ffill(limit=3).reindex(returns.index).ffill(limit=3)signal_returns = signal_close.pct_change(fill_method=None).replace([np.inf, -np.inf], np.nan).reindex(returns.index).fillna(0.0)
We made monthly rebalance dates and we can see that even though our data starts from 2013, we need a lookback window for estimating covariance and mean, so the first rebalance date happens at the first month of 2016.
As we can see, QQQ has been the strongest asset in the sample, with an annualized return close to 20% and volatility around 21%. SPY also performs strongly, with lower return and lower volatility than QQQ. IWM has a positive return but much higher drawdown, which is typical of small cap exposure especially in our sample where large cap growth leadership has been very strong.
The international equity ETFs are weaker. EFA and EEM both have large drawdowns and volatility, but their returns are much lower than U.S. equities. This already tells us that a simple global equity diversification story is not enough. International exposure may still be useful, but an efficient model should not blindly assume that all equities and markets deserve the same allocation.
The bond ETFs show very different behavior. SHY has very low volatility around \(1.4\%\) and a much smaller drawdown around \(-5.7\%\), but the return is also low. TLT is the opposite of what someone might expect from “bonds”. It has a very large drawdown around \(-48.5\%\) in this sample because as we said earlier, long duration bonds are sensitive to interest rate changes. TLT fell hard by the post-2021 rate shock.
DBC has a very large drawdown around \(-62.2\%\), which can be from the long weak cycles. GLD and VNQ are somewhere between diversifiers and risk assets. They have meaningful volatility and large drawdowns, but they also bring different economic exposure.
An efficient active model like Black-Litterman should know when to lean into growth, when to prefer real assets, when to reduce long duration, and when to stay close to the benchmark so it can control risk and gain the high return based on the state and information and the uncertainty of that information given to it each time it’s rabalancing.
As we can see, volatility jumps during crisis periods, especially around the COVID shock. Equity and real asset ETFs show much larger volatility spikes than short duration bonds. SHY remains the lowest volatility instrument throughout the sample.
U.S. equity ETFs are highly correlated with each other. SPY and QQQ move very closely, and IWM also has strong equity market correlation. Developed and emerging market equity ETFs also connect strongly to the US equity. This means that simply owning SPY, QQQ, IWM, EFA, and EEM is not as diversified as the number of tickers suggests.
The bond holding ETFs are different. TLT, IEF, SHY, AGG, and LQD have their own internal structure. TLT and IEF are strongly related because both are Treasury duration exposures. TLT has negative correlation with SPY around \(-0.19\), which shows why long Treasuries historically helped in some equity selloffs. But the correlation is not fixed forever. In inflation shocks, stocks and bonds can fall together.
AGG and LQD are also connected to the broader bond and credit market. HYG behaves more like risky credit and often correlates more with equities than with pure Treasuries.
Gold and commodities contain a different kind of exposure. GLD has weaker relationships with most assets, which can make it a good diversifier. DBC has a different inflation and commodity cycle profile, so it can help when inflation sensitive assets are leading, but it can also be very difficult to hold during long weak commodity regimes.
If the optimizer only sees a covariance matrix and expected returns, it may concentrate in a few assets. The human portfolio manager might add economic sanity to the model instead of just relying on estimation. This is why Black-Litterman can be great for combining economic ideas and views into a portfolio optimization model.
3) Building the strategic benchmark and Prior
If we remember the bayes rule from earlier, we want to update our belief when new information arrives. We first need an intial belief or a starting point that is going to change with the new information.
3.1 Benchmark portfolio
We First define a neutral portfolio. This is not supposed to be the smartest portfolio in the notebook. In Black-Litterman, the benchmark exists because it represents the starting belief about how capital should be allocated before any active views are added.
The benchmark we use is a portfolio with pre-defined fixed weights and the weights are based on what we think can be reasonable for these ETFs based on their performance, behaviour and what they hold. It gives meaningful weight to US equities, international equities, bonds, credit, gold, real estate, and commodities. It is also constrained by asset caps and constraints. These caps protect the optimization from unrealistic concentration on just one asset.
For example, even if QQQ has a strong historical return, we probably do not want a mult asset benchmark to become almost entirely QQQ. We are looking for diversification. Also, even if short Treasuries have low volatility, a diversified benchmark should not become a pure cash substitute. In that case we will end up with a very low volatility portfolio but almost the worst return.
We don’t want the optimizer to do whatever it wants. We want it to stay inside a realistic multi asset allocation structure. That’s why we use bounds and caps for weights on each asset.
Since we already know from notebook 2, A benchmark weight vector looks like this:
As we can see, we set SPY as the largest position at about \(22.2\%\), followed by AGG around \(10.1\%\), EFA around \(9.1\%\), and several exposures around \(8.1\%\) or \(5.1\%\). Large weights go to liquid core exposures. SPY for broad US equity, AGG for core bonds, EFA for developed international equities, and intermediate/long Treasuries for duration. Smaller weights go to more specialized or volatile exposures such as DBC, VNQ, HYG, and IWM. This makes sense because those assets can be useful, but we usually do not want them to dominate the portfolio.
US equity is the largest section at about 34.3%. Duration bonds and international equities are also used enough. Core bonds, real assets, and credit each have visible allocations, while commodities are intentionally small.
This looks like a reasonable strategic benchmark for a multi asset project. It is growth oriented enough to participate in equity markets, but it is not a pure equity portfolio to end up with too high volatility. It has bonds, gold, real estate, credit, and commodities. That makes it a useful starting point for active allocation.
We now build the first Black-Litterman object: the prior expected return vector.
In ordinary mean-variance optimization, expected returns are usually the weakest input. Historical average returns are noisy, unstable, and extremely sensitive to the sample window. Black-Litterman avoids starting from raw sample means. Instead, we do a reverse action. We now have a portfolio with our defined weights, so there has to be a Mean-Variance that produces these weights with an unknown mean. We are looking for that mean given our weights. We want to see what expected returns would make this benchmark look optimal for an investor?
Here \(\mu\) is the expected return vector, \(\Sigma_t\) is the covariance matrix estimated at time \(t\), \(w_b\) is the benchmark weight vector, and \(\delta_t\) is the risk-aversion coefficient. Higher \(\delta\) means the investor wants more compensation for taking variance.
the unconstrained solution is \(\mathbf{w}_b^* = \frac{1}{\delta} \boldsymbol{\Sigma}^{-1} \boldsymbol{\mu}\), which can be changed to give:
The vector \(\mu_\text{eq}\) is the prior expected return vector. If an investor maximizes expected return minus a penalty for variance, then the expected return vector that supports a given benchmark is equal to covariance times benchmark weights. If an asset contributes a lot of risk to the benchmark, it must have a higher implied return to justify its place. If an asset is defensive or low risk, it’s implied return is usually lower.
3.2.2 Estimating Covariance \(\Sigma\) and Risk Aversion \(\delta\)
We also estimate \(\Sigma\) with EWMA covariance. As we analyzes in notebook 2, EWMA puts more weight on recent returns:
A high \(\lambda\) such as \(0.97\) makes the covariance estimate persistent but still responsive. This is useful for portfolio risk because covariance regimes change: 2020, 2022, and 2025 do not have the same market structure.
The parameter \(\delta\) is the market risk aversion coefficient which controls the overall scale of the implied returns. Economically, \(\delta\) is the additional expected return the representative investor requires per unit of portfolio variance. We get the \(\delta\) from benchmark too:
This is the Sharpe ratio of the benchmark expressed in variance units. It measures how many units of expected excess return the market demands per unit of variance. Rather than assuming a fixed \(\delta = 2.5\) (usual in literature), we estimate it dynamically at each rebalance date from the rolling historical performance of the benchmark portfolio itself.
you might be confused about using \(\lambda\) as the risk reversion parameter for Mean-Variance model in notebook 2, and here, using \(\delta\). We use \(\delta\) as the symbol for implied risk aversion, which means extracting it from an existing model/portfolio. But \(\lambda\) is used as the parameter of risk aversion for optimizing a model and getting to the optimal portfolio we want.
Note: The prior is not static. At each rebalance date we use the current EWMA covariance estimate, so \(\boldsymbol{\mu}_{\text{eq}}\) evolves with the market’s volatility and correlation structure.
The latest prior table ranks EEM at about \(21.5\%\), EFA around \(17.2\%\), GLD around \(16.5\%\), IWM around \(15.5\%\), QQQ around \(14.1\%\), and SPY around \(11.3\%\). These are annualized equilibrium returns implied by the benchmark, the latest covariance estimate, and the risk-aversion estimate.
When the latest prior ranks EEM, EFA, GLD, IWM, QQQ, and SPY highly, it’s not simply saying these assets will have higher return, It’s saying that in the latest covariance estimate, and based on the benchmark exposures, those assets carry some kind of risk contribution that needs higher equilibrium return. DBC having a negative prior in the latest table means the benchmark and covariance structure do not naturally reward DBC in the same way at that time, so any active overweight to commodities must come from views rather than from the prior alone. This means we say on default, investing on commodities doesn’t benefit us in the way we want it, but since we use Black-Litterman, our views might show us things we don’t kno, guiding us to invest on them in the right time.
GLD receiving a high prior is interesting. Gold does not always have high historical average return, but in a covariance-based prior it can receive more implied return when it contributes useful portfolio risk control maybe as a diversification power. This is one of the reasons Black-Litterman is more stable than raw historical mean estimation.
In periods when an asset becomes more volatile or more connected to the benchmark, its implied prior can move. In 2020 and parts of 2022 which are stress regimes, implied returns change a lot because covariance and benchmark risk conditions changed suddenly. This is better than using one fixed prior for the whole sample. If market risk structure changes, the return should update too.
4) Building baseline strategies
We now run the baseline portfolio models from the portfolios we made in notebook 2, and compare them against the strategic benchmark.
A quick reminder from notebook 2: The minimum-variance problem is
\[
\min_w w^{\top}\Sigma w
\]
subject to the usual constraints like full investment, long-only weights, caps, and sleeve bounds.
The mean-variance problem is
\[
\max_w w^{\top}\mu-\lambda w^{\top}\Sigma w
\]
where \(\mu\) is the expected return vector. The problem is that \(\hat\mu\) is noisy. A small change in expected returns can create a large change in weights. This is why the baseline uses shrinkage-style inputs such as Ledoit-Wolf covariance and Bayes-Stein means as we implemented and explained in notebook 2, here we don’t repeat the code and use the library’s grid function for producing models.
candidate_rows = []for name, result in baseline_candidates.items(): m = selection.performance_metrics(result["returns"], result["nav"], rf_daily=risk_free_daily, annualization=annualization) candidate_rows.append({"strategy": name, "cagr": m["CAGR"], "ann_vol": m["Vol"],"sharpe": m["Sharpe"], "max_drawdown": m["Max Drawdown"],"avg_turnover": result["turnover"].mean()})baseline_candidate_summary = pd.DataFrame(candidate_rows).sort_values("sharpe", ascending=False)display(baseline_candidate_summary.round(4))
strategy
cagr
ann_vol
sharpe
max_drawdown
avg_turnover
0
Strategic Benchmark
0.0914
0.1014
0.5284
-0.2242
0.0144
3
MV (LedoitWolf, BayesStein)
0.0814
0.0847
0.4949
-0.1378
0.0369
1
Equal Weight
0.0848
0.0935
0.4842
-0.2038
0.0114
2
MinVar (OAS)
0.0665
0.0754
0.3543
-0.1973
0.0031
Show code
avg_turnover_table = pd.DataFrame({name: result["turnover"].mean() for name, result in baseline_results.items()}, index=["avg_turnover"]).Tdisplay(avg_turnover_table.round(4))baseline_nav = pd.DataFrame({name: result["nav"] for name, result in baseline_results.items()})baseline_dd = baseline_nav / baseline_nav.cummax() -1.0fig, axes = plt.subplots(2, 1, figsize=(9, 6), sharex=True)baseline_nav.plot(ax=axes[0])axes[0].set_title("benchmarks nav")axes[0].set_ylabel("nav")baseline_dd.plot(ax=axes[1], lw=1.1)axes[1].set_title("benchmarks drawdown")axes[1].set_ylabel("drawdown")axes[1].set_xlabel("date")plt.tight_layout() plt.show()
avg_turnover
Strategic Benchmark
0.0144
Equal Weight
0.0114
MinVar (OAS)
0.0031
MV (LedoitWolf, BayesStein)
0.0369
As we can see, the strategic benchmark has the highest Sharpe ratio, around \(0.52\), with a CAGR around \(9.1\%\) and volatility around \(10.1\%\).
The MV Model using Ledoit-Wolf and Bayes-Stein has the lowest drawdown among these baselines, around \(-13.8\%\), and volatility around \(8.4\%\), but its CAGR is also lower than the benchmark. So it’s safer in path terms, but it doesn’t beat the benchmark on return and Sharpe. Equal weight is close but slightly weaker. The minimum-variance portfolio has lower volatility around \(7.6\%\), but its CAGR is only around \(6.8\%\) and its Sharpe is around \(0.38\).
This is the issue that MinVar has in these assets: there are low-volatility assets that also have very low returns. SHY is almost one of the safest investments in the universe, but it’s also one of the lowest-return assets. A model like minimum variance will overweight it to reduce volatility. But reducing variance doesn’t automatically create a better return-risk tradeoff if the portfolio gives up too much return. That’s why it ends up as the lowest-Sharpe model in our benchmarks. For these assets specifically, taking return more seriously is more important than the stock market that we tested in notebook 2, because there were a lot of very high return-high risk stocks in nasdaq market and controlling risk was really important to a better performance. But here, we have all ETFs having lower volatility compared to the stocks, but also not that high of return, so having higher return in a way to control risk enough, is important, because in this specific set of assets, reducing volatility too much, will hurt returns too much.
Another note is that our own benchmark already has the best Sharpe among the baseline models. Even if the views don’t add anything, or even move the benchmark in the wrong direction, the Black-Litterman model can still look better than some of the purely statistical baselines. So the serious test is not only BL versus MinVar or Equal Weight. We have to check the difference between the strategic benchmark and the BL model to see whether views and learned confidence actually add return and Sharpe over the benchmark.
This is also why the later benchmark-relative tables are not optional diagnostics. They are the main way to separate useful active views from a portfolio that only looks good because the starting benchmark was already strong.
5) market signals
Before making a Black-Litterman view, we need a way to describe the market environment. We use signals that are common in asset allocation: momentum, trend, volatility, drawdown quality, dividend yield, relative strength, and correlation.
Momentum signals: Computed as the cumulative return of an asset over a trailing window, excluding the most recent month (the standard momentum definition avoids short-term reversal), for example, this is how a 6 month momentum is calculated:
We compute this for 1-month, 3-month including recent, 6-to-1 month, and 12-to-1 month.
Trend signals: The price of each asset relative to its \(n\)-day simple moving average (SMA), normalized by the SMA: \[
\text{Trend}_n = \frac{\text{Price} - \text{SMA}_n}{\text{SMA}_n}
\]
A positive 200-day trend means the asset is in a long-term uptrend. Negative means a downtrend. This is close to momentum, but it gives us information about recent performance in a different way: it compares the latest price with the asset’s own moving average instead of only comparing today’s price with an old price.
Volatility and drawdown signals: Rolling 63-day volatility and trailing 252-day drawdown capture the risk profile and stress state of each asset.
Composite score: Each asset gets an overall score combining the above signals, which ranks assets from most to least attractive for overweighting relative to the benchmark. This score is used as a sorting tool inside view-generation rules. But these scores might not be in the same scale, and we have to make them comparable to each other.
Therefore we use cross-sectional z-scores. A z-score shows us whether an asset’s signal is strong or weak relative to the other assets at the same date. A z-score turns different things into comparable units. The basic idea is
The point is not to make one perfect signal. The point is to avoid relying on a single noisy indicator and to cover different sides of the behaviour of these assets. Momentum, trend, volatility, and drawdown quality each capture a different part of the market state.
Market state signals: Besides per-asset signals, we compute a set of market-wide state variables that describe the current market environment: - fraction of risky assets (SPY, QQQ, IWM, EEM, HYG, VNQ, DBC) above their 200-day SMA, measures broad market health - current SPY drawdown from its 252-day high detects whether we’re in an equity stress environment which can be very important for weighting in this set of assets. - rolling 126-day correlation between SPY and TLT is important for understanding whether diversification is working - relative cumulative returns of one ETF versus another over 63 days is the market’s current relative performance signal that directly feeds into views
In the code we winsorize the cross-sectional inputs before turning them into z-scores. That keeps one extreme ETF reading from dominating the whole signal table, while still preserving the ranking information that the view rules need.
Show code
risky_asset_names = ["SPY", "QQQ", "IWM", "EFA", "EEM", "HYG", "VNQ", "DBC"]cyclical_asset_names = ["IWM", "EEM", "VNQ", "HYG"]defensive_asset_names = ["TLT", "IEF", "SHY", "AGG", "LQD", "GLD"]def winsorized_zscore(series): s = pd.Series(series, dtype=float).replace([np.inf, -np.inf], np.nan) good = s.dropna() clipped = s.clip(good.quantile(0.05), good.quantile(0.95)) std = clipped.std(ddof=0)ifnot np.isfinite(std) or std <=1e-12:return pd.Series(0.0, index=s.index)return ((clipped - clipped.mean()) / std).fillna(0.0)def cumulative_return(window): clean = pd.Series(window, dtype=float).dropna()return np.nan if clean.empty elsefloat((1.0+ clean).prod() -1.0)def trailing_dividend_yield(dividends, close_hist, ticker, lookback=252):if dividends isNoneor dividends.empty:return np.nan div_window = dividends[ticker].reindex(close_hist.index).fillna(0.0).tail(lookback) px = close_hist[ticker].dropna()returnfloat(div_window.sum() / px.iloc[-1]) iflen(px) and px.iloc[-1] >0else np.nandef relative_cumulative_return(ret_hist, long_asset, short_asset, lookback=63):iflen(ret_hist) < lookback:return np.nan long_ret = cumulative_return(ret_hist[long_asset].tail(lookback)) short_ret = cumulative_return(ret_hist[short_asset].tail(lookback))return long_ret - short_ret if np.isfinite(long_ret) and np.isfinite(short_ret) else np.nandef trailing_pair_correlation(ret_hist, asset_a, asset_b, lookback=126):iflen(ret_hist) < lookback:return np.nan window = ret_hist[[asset_a, asset_b]].tail(lookback).dropna()iflen(window) <max(30, lookback //3):return np.nanif window[asset_a].std(ddof=1) <=1e-12or window[asset_b].std(ddof=1) <=1e-12:return np.nanreturnfloat(window[asset_a].corr(window[asset_b]))def trailing_average_correlation(ret_hist, names, lookback=126): names = [name for name in names if name in ret_hist.columns]iflen(names) <2orlen(ret_hist) < lookback:return np.nan window = ret_hist[names].tail(lookback).dropna(how="any")iflen(window) <max(30, lookback //3):return np.nan corr = window.corr().replace([np.inf, -np.inf], np.nan) vals = corr.where(~np.eye(len(corr), dtype=bool)).stack().dropna()returnfloat(vals.mean()) iflen(vals) else np.nandef trend_breadth_from_close(close_hist, names, ma_window=200, offset=0): names = [name for name in names if name in close_hist.columns] hist = close_hist.iloc[:len(close_hist) - offset] if offset andlen(close_hist) > offset else close_hist vals = []for ticker in names: px = hist[ticker].dropna()iflen(px) < ma_window:continue ma = px.tail(ma_window).mean()if ma >0: vals.append(px.iloc[-1] / ma -1.0)returnfloat((pd.Series(vals) >0).mean()) if vals else np.nan
From the latest signals (2026-06-17) table, we can see that EEM has the strongest latest score by a wide margin. It has strong recent momentum, strong medium-term momentum, a positive trend, and a small drawdown. DBC, IWM, QQQ, SPY and EFA also score positively, while several bond and credit ETFs score negatively.
The latest market state points in the same direction. The risky median score is above the defensive median score, so the risk-on score is positive. Risky trend breadth and cyclical trend breadth are both strong. SPY is not in a deep drawdown, and the equity stress flag is false. The commodity-versus-duration signal is also strong because DBC has been outperforming duration-sensitive bond exposures.
This is not yet a portfolio decision. It is an information layer. We are learning what the current market looks like: risk-on, commodity-positive, cyclicals reasonably strong, bonds relatively weak, and dollar strength present but not dominant enough to overturn the other signals. Now we want to use these signals and states to generate the views that directly affect the Black-Litterman model.
The useful thing about this signal layer is that it stays separate from the optimizer. We can inspect the market state first, then inspect the views, and only after that look at posterior returns and final weights. That makes mistakes much easier to diagnose.
6) Black-Litterman views
We now translate market signals into views. This is the conceptual center of the notebook.
A view in Black-Litterman is a statement of the form: “I believe that asset (or portfolio) \(A\) will outperform asset (or portfolio) \(B\) by \(q\) per year.” Formally, a view is a pair \((\mathbf{p}, q)\) where \(\mathbf{p} \in \mathbb{R}^N\) is a portfolio vector encoding which assets are long and which are short, and \(q \in \mathbb{R}\) is the expected excess return of that portfolio vector \(\mathbf{p}\).
The portfolio vector \(\mathbf{p}\) for a simple relative view (asset \(A\) outperforms asset \(B\)) is: \[
p_i = \begin{cases} +1/|L| & \text{if asset } i \text{ is long} \\ -1/|S| & \text{if asset } i \text{ is short} \\ 0 & \text{otherwise} \end{cases}
\]
where \(|L|\) and \(|S|\) are the number of long and short assets in the view. This equal-weighting within each side keeps views comparable in scale. The vector \(\mathbf{p}\) sums to zero (long equals short), which means the view expresses a relative return rather than an absolute return level.
If we have \(K\) views and \(N\) assets, we’re working with a view matrix\(P\) that has shape \(K\times N\). Each row of \(P\) defines one long-short or relative-return portfolio. The view vector \(q\) has length \(K\) and contains the expected return attached to each view.
For one view \(k\), the statement is
\[
P_k\mu=q_k
\]
For \(K\) views, all rows stack into the view matrix:
If the view is “QQQ and SPY should outperform HYG and VNQ,” then the portfolio row could look like a long-short spread with positive weights on QQQ and SPY, negative weights on HYG and VNQ, and zeros elsewhere. The row sums to zero when it’s a relative view.
This is useful because the view is not simply bullish on the whole market. It says one set of assets should beat another set, which is exactly the kind of information a portfolio manager often has.
We don’t only use simple “this asset will outperform that one” views. Those can be useful when someone has a very specific belief or information about the near future, but here we want views that can track market states and regimes repeatedly. So we make several economically motivated view families.
How do we write these views in math form for the model? We extract a portfolio vector from the idea and attach an expected return to that portfolio. We can’t just say “SPY is going to outperform EEM” inside the BL equation, but we can say “I have a portfolio \(\mathbf{p}\) with weight 50% on SPY and -50% on EEM, with expected return \(q\) equal to \(0.50\mu_{\mathrm{SPY}}-0.50\mu_{\mathrm{EEM}}\)”.
If the universe is ordered as SPY, QQQ, IWM, EFA, EEM, TLT, IEF, SHY, AGG, LQD, HYG, GLD, VNQ, DBC, then one inflation-style view could look like:
That is why Black-Litterman views are so useful. They allow us to include views and predictions inside a portfolio without needing to forecast every single asset’s standalone return.
The signs in \(\mathbf{p}\) are important. Positive entries are the side we want to reward, negative entries are the side we expect to lag, and zeros are assets that are not directly part of the view. The posterior can still move the zero-weight assets indirectly through covariance, but the original belief is expressed only through the long-short row.
6.1. constructing the views
1. Liquidity Leadership: This view shows when large-cap US growth (SPY, QQQ) is outperforming cyclical and small-cap assets (IWM, EEM, VNQ, HYG) on a trend and momentum basis. Economically, this corresponds to a narrow market leadership regime when investors are concentrating risk in the most liquid, high-quality names rather than spreading into cyclicals. This is often seen in late-cycle or uncertainty environments. The view goes long the leaders (SPY, QQQ) and short the lagging cyclicals.
2. Dual Momentum: This is a cross-sectional momentum view. The assets with the best recent risk-adjusted momentum scores are expected to continue outperforming the assets with the worst scores. The view is formed by going long the top-ranked assets and short the bottom-ranked across all 14 ETFs. This can work well in practice even though it’s completely backward-looking, because cross-asset trends often persist longer than single-stock noise.
3. Inflation Rotation: This view shows when real assets (GLD, DBC) are outperforming fixed income (IEF, AGG) on a trend basis, signalling an inflationary environment. Economically, when commodities and gold are trending up relative to bonds, the market is pricing in higher inflation expectations. The view goes long real assets and short bonds.
4. Risk-Adjusted Momentum: A more sophisticated version of dual momentum that addresses volatility too. Assets are ranked by their Sharpe-ratio-like signal (return per unit of recent volatility). High Sharpe-ratio assets are expected to continue outperforming low Sharpe-ratio assets.
5. Credit Switch: When HYG (high yield) is significantly outperforming LQD (investment grade) on a relative basis, the credit market is in risk-on state. This view goes long high yield and short investment grade, betting that credit spreads will continue to compress. When HYG is underperforming, the signal reverses.
6. Reflation Breadth: This view detects a broad economic reflation. A regime where small caps outperform large caps (IWM > SPY), high yield outperforms investment grade, cyclical trend breadth is high, and commodities are trending. When all these conditions are confirmed together, the view goes long cyclicals (IWM, EEM, VNQ, HYG, DBC) and short SPY, TLT. This is close to a risk-on reflation macro regime.
7. Growth-Duration Barbell: This view shows when equity momentum (QQQ/SPY relative performance) and bond momentum (TLT/IEF) are strong together, suggesting the market is rewarding both growth and duration. This is a barbell view: long growth equity and long-duration bonds, short the middle exposures such as IWM, EEM, and HYG. This regime covers much of the 2014-2020 period.
8. Correlation Stress: When the stock-bond correlation turns positive (both stocks and bonds are falling together, as happened in 2022), the traditional diversification benefit of bonds breaks down. This view reduces long duration exposure (TLT, IEF) and favours real assets and short duration (GLD, SHY) as alternative diversifiers.
9. International Rotation: This view tracks the relative performance of emerging markets (EEM) versus developed international (EFA). When EEM is outperforming EFA on momentum, it signals a risk-on, dollar-weak, commodity-supportive environment. The view goes long EEM and short EFA. When the dollar (UUP) is strengthening, we control this view so it doesn’t fire too easily and create a false emerging-market signal.
10. Duration Quality: This view monitors the relative performance of long-duration bonds (TLT) versus short-duration (SHY) to determine the slope of the term premium. A positive signal (TLT outperforming SHY) shows the market is pricing in falling rates and suggests an overweight in long duration.
These view families are intentionally not symmetric in frequency. Momentum views should appear often because they describe broad ranking behaviour. Stress and inflation views should be rarer because they represent more specific regimes. That difference later affects how much data each view has for learned confidence.
Show code
view_caps = {"liquid_leadership": 0.08, "dual_momentum": 0.10, "inflation_rotation": 0.05,"risk_adj_momentum": 0.10, "credit_switch": 0.05, "reflation_breadth": 0.10,"growth_duration": 0.08, "correlation_stress": 0.08,"international_rotation": 0.10, "duration_quality": 0.08}view_names = {"liquid_leadership": "Liquidity leadership", "dual_momentum": "Dual momentum","inflation_rotation": "Inflation rotation", "risk_adj_momentum": "Risk-adjusted momentum","credit_switch": "Credit switch", "reflation_breadth": "Reflation breadth","growth_duration": "Growth-duration barbell", "correlation_stress": "Correlation stress","international_rotation": "International rotation", "duration_quality": "Duration quality"}q_strength_scale =1.25def p_series_from_assets(long_assets, short_assets, asset_list=assets): p = pd.Series(0.0, index=asset_list, dtype=float) long_assets =list(dict.fromkeys([asset for asset in long_assets if asset in p.index])) short_assets =list(dict.fromkeys([asset for asset in short_assets if asset in p.index and asset notin long_assets]))ifnot long_assets ornot short_assets:returnNone p.loc[long_assets] =1.0/len(long_assets) p.loc[short_assets] =-1.0/len(short_assets)return pdef make_view_row(view_id, view, long_assets, short_assets, view_strength, risk_orientation, source="view_rule", priority=0.50, confluence_score=None, view_state=None): long_assets =list(dict.fromkeys([asset for asset in long_assets if asset in assets])) short_assets =list(dict.fromkeys([asset for asset in short_assets if asset in assets and asset notin long_assets]))if view_id notin view_caps ornot long_assets ornot short_assets ornot np.isfinite(view_strength):returnNone view_strength =float(abs(view_strength))if view_strength <=1e-8:returnNone cap = view_caps[view_id] q_tilt =float(cap * math.tanh(view_strength / q_strength_scale)) p = p_series_from_assets(long_assets, short_assets)if p isNone:returnNone priority =float(np.clip(priority, 0.0, 1.0)) confluence = view_strength /3.0if confluence_score isNoneelse confluence_score confluence =float(np.clip(confluence, 0.0, 1.0)) view = view or view_names.get(view_id, view_id.replace("_", " ").title())return {"view_id": view_id, "view": view, "view_state": view_state or risk_orientation,"long_assets": long_assets, "short_assets": short_assets,"signal_value": view_strength, "raw_strength": view_strength,"view_strength": view_strength, "confluence_score": confluence,"q_tilt": q_tilt, "q": q_tilt, "confidence": np.nan,"risk_orientation": risk_orientation, "source": source,"priority": priority, "view_priority": priority,"p_vector": p.round(6).to_dict()}def best_assets(signal_table, candidates, n=1, require_positive=False): names = [asset for asset in candidates if asset in signal_table.index] frame = signal_table.reindex(names).sort_values("score", ascending=False)if require_positive: frame = frame[(frame["score"] >0) | (frame["trend_200"] >0) | (frame["mom_3_0"] >0)]returnlist(frame.head(n).index)def worst_assets(signal_table, candidates, n=1, require_weak=False): names = [asset for asset in candidates if asset in signal_table.index] frame = signal_table.reindex(names).sort_values("score")if require_weak: frame = frame[(frame["score"] <0) | (frame["trend_200"] <0) | (frame["mom_3_0"] <0)]returnlist(frame.head(n).index)def sleeve_limited_assets(signal_table, candidates, n=3, side="long", per_sleeve=1): names = [asset for asset in candidates if asset in signal_table.index] frame = signal_table.reindex(names).sort_values("score", ascending=(side =="short")) picks, sleeve_counts = [], {}for asset, row in frame.iterrows(): sleeve = row.get("sleeve", sleeve_map.get(asset, "Other"))if sleeve_counts.get(sleeve, 0) >= per_sleeve:continueif side =="long"and row["trend_200"] <=0and row["mom_3_0"] <=0:continueif side =="short"and row["trend_200"] >=0and row["mom_3_0"] >=0:continue picks.append(asset) sleeve_counts[sleeve] = sleeve_counts.get(sleeve, 0) +1iflen(picks) >= n:breakreturn picks
As we can see in the last state, we have 5 active views that are built from the signals and are strong enough, based on \(q\) tilt, to be kept: liquid leadership, dual momentum, inflation rotation, risk-adjusted momentum, international rotation, and duration quality.
Dual momentum is the strongest view by annualized tilt. It goes long the strongest assets such as QQQ, DBC, EEM, and VNQ, and short assets such as TLT and GLD. This is a clean cross-asset relative-strength view. Risk-adjusted momentum is the second-strongest \(q\)-tilt view and is almost similar to dual momentum, but it shorts VNQ instead of longing it. That is the effect of including volatility and drawdown quality in the momentum score.
International rotation goes long EEM against EFA, which shows emerging-market leadership compared to developed international equities.
Duration quality has a small tilt, which also makes sense. It shows that inside fixed income, shorter or higher-quality exposures look more attractive than long-duration TLT when duration risk doesn’t look worth taking.
The key point is that we don’t let one signal directly control the portfolio. It first converts signals into interpretable relative-return views. That makes the model easier to analyze and easier to fix.
6.2 View Strength and the Tilt Cap
Before adding the view, the prior already implies an expected return spread:
where \(\boldsymbol{\mu}_{\text{prior}}\) is the Black-Litterman prior return vector. This is what the benchmark-implied prior already believes about the long-short spread.
The view doesn’t replace the prior from zero. We add a tilt on top of the prior spread:
So the final tilt is the part that actually makes the posterior move away from the prior in the direction of the view.
The tilt \(q_{\text{tilt}}\) is the amount that the view moves expected returns away from what the prior already implies for this portfolio. If the prior already assigns the right weights, the tilt is zero and the view adds nothing.
Now how can we know which view is most likely important and which state is most likely happening at each time? Each view has a maximum allowed view size. This avoids one strong-looking signal from creating an unrealistic return estimate. If the signal strength is \(s_i\) and the family cap is \(c_i\), we use a smooth function:
The hyperbolic tangent is useful because it maps signal strength to \((-1, 1)\) and grows quickly at first and then flattens. A signal can become stronger, but the view can’t become too large. This is more realistic than a linear rule because financial signals don’t become twice as trustworthy just because the z-score is twice as large. A small signal creates a small active return view. A strong signal creates a larger active return view. But once the signal is already strong, the cap prevents overconfidence.
For example, the liquid leadership view has a cap of 8%, meaning even an infinitely strong signal can’t push the expected return tilt for that view above 8% per year. The \(q_{\text{scale}} = 1.25\) parameter controls how quickly the tilt saturates. Smaller values mean the tilt reaches its cap faster for a given signal strength.
Having all 10 views active all the time would cause problems, so we use a selection step to keep the most useful views with meaningful \(q_\text{tilt}\).
The cap is not a confidence number. It’s a maximum active-return size for that view family before confidence is even learned. Confidence later decides how noisy the view is; the cap decides how large the raw view is allowed to become.
8.3 Candidate view log through time
After defining the view rules, we generate candidate views at every rebalance date and record the view activity through time. This gives us two diagnostics:
how many views tend to fire in each month,
which view families dominate the historical sample.
The frequency of a view family matters because it changes the reliability problem. A view that fires 126 times has enough observations to estimate payoff quality with some usefulness. A view that fires 4 times can still be economically important, but its learned confidence has to be treated with caution.
The q-tilt heatmap adds another layer: not only whether a view appears, but how strong its tilt is when it appears. A view that fires often with tiny \(q\) is less influential than a view that fires less often with large \(q\).
where \(K_t\) changes over time. Black-Litterman can handle a changing number of views because the posterior update only requires the current \(P_t\), \(\mathbf{q}_t\), and \(\Omega_t\).
This log is also useful for debugging. If a view fires in a period where its economic story doesn’t make sense, we can inspect the signal inputs for that date rather than only looking at the final portfolio weight.
Show code
max_active_views =len(view_caps)def view_signature(row): long_side =tuple(sorted(row.get("long_assets", []))) short_side =tuple(sorted(row.get("short_assets", [])))return long_side, short_sidedef combine_and_rank_views(simple_views): rows, seen = [], set()for original in simple_views: sig = view_signature(original)if sig in seen:continue seen.add(sig) row =dict(original) cap = view_caps.get(row.get("view_id"), 0.020)if cap >0: q_score =float(np.clip(abs(row.get("q_tilt", 0.0)) / cap, 0.0, 1.0))else: q_score =0.0 row["q_score"] = q_score row["adjusted_strength"] = (0.55* row.get("confluence_score", 0.0) +0.25* q_score+0.20* row.get("view_priority", 0.0)) rows.append(row)returnsorted(rows, key=lambda item: item["adjusted_strength"], reverse=True)[:max_active_views]
Candidate view counts usually sit around 4 to 6, but the count moves through time. This is good because the view engine is responding to the market state instead of forcing a fixed number of views. If the count were always fixed, the view engine would be forcing views even when the market didn’t support them. If the count were usually zero, the model would behave too much like the benchmark.
Risk-adjusted momentum and Dual momentum dominate the candidate set. Risk-adjusted momentum fires 126 times with average q-tilt around 0.0883. Dual momentum fires 119 times with average q-tilt around 0.0867. These are the broadest and most persistent view families, so it makes sense that they appear often.
Duration quality fires 78 times, but with a smaller average tilt around 0.0334. Liquidity leadership, international rotation, and credit-switch risk-on also fire frequently. Reflation breadth and inflation rotation are more regime-specific. Correlation stress only fires 4 times, which is exactly the kind of view that should stay rare.
The q-tilt heatmap clusters by regime. Momentum views are active through most of the sample, while inflation and correlation views concentrate in specific regimes. That distinction is important for the learned-confidence system: frequent views get more statistical learning, rare views need stronger conservative caps.
The rare views should not be ignored just because they have fewer observations. They often describe the exact regimes where a static benchmark struggles. But the learned-confidence layer has to treat them differently from the high-frequency momentum views.
9) View payoff history
This is the part that makes the project different from a standard Black-Litterman implementation.
Classical Black-Litterman asks the user to choose confidence manually. In practice, that is hard. Saying “I am 70% confident in this view” sounds precise, but it often has no statistical meaning unless we tie it back to realized performance. We instead measure how the view has actually performed whenever it fired in the past.
For each view at date \(t\), we look forward \(h=21\) trading days and compute the cumulative return of each asset:
This puts payoff values on the same annual scale as \(q\) and the posterior expected returns.
A view earns a positive payoff when the long side beats the short side over the next month. That is the direct test of whether the view was useful.
The payoff uses returns after the view date, not returns before it. That is the small but important no-lookahead detail: the current signals create the view, and only future long-short performance decides whether the view was right.
9.1 Hit rate, payoff IR, t-stat, and information coefficient
Once we have payoff history, we summarize each view family.
The hit rate is the fraction of realized payoffs that were positive:
This is basically a Sharpe ratio for the view itself. It tells us how much average payoff the view produced per unit of payoff noise.
The t-statistic is:
\[
t_k = IR_k\sqrt{n_k}
\]
It rewards both consistency and sample size. A view with decent IR and many observations earns more trust than a view with decent IR from only a few observations.
The information coefficient checks whether stronger signal values actually lead to stronger future payoffs:
This is useful because a view can have a decent hit rate, but if larger signal strength doesn’t lead to larger payoff, the magnitude of \(q\) is less trustworthy.
The information coefficient is especially useful for sizing. A view can be right more than half the time, but if stronger signals don’t lead to stronger payoffs, then the model should be careful about scaling \(q\) too aggressively from signal strength alone.
The payoff history table is one of the most important outputs in the project.
The best broad evidence comes from Dual momentum and Risk-adjusted momentum. Dual momentum has 117 observations, hit rate around 57.26%, recent hit rate 100%, average annualized payoff around 4.78%, and payoff IR around 0.1106. Risk-adjusted momentum has 124 observations, hit rate around 57.26%, recent hit rate 66.67%, and payoff IR around 0.0893. These are not huge IR values, but for noisy monthly cross-asset views, they are meaningful enough to deserve moderate confidence.
Credit switch: risk-on has a strong profile: hit rate around 65.12%, average payoff around 15.04%, and payoff IR around 0.3733. This suggests that when credit confirms a risk-on environment, the view has historically been useful.
Reflation breadth also looks strong, with hit rate around 71.43% and payoff IR around 0.3905, but it has only 21 observations. This is useful but less statistically mature than momentum.
Some views are weak. Duration quality has hit rate around 35.53% and negative payoff IR. International rotation has hit rate around 44.44% and negative payoff IR. Credit switch: risk-off has very poor hit rate around 23.08%. These views can still be selected if current signals support them, but learned confidence needs to shrink them.
Correlation stress has excellent-looking numbers, with hit rate 75% and payoff IR 0.5361, but only 4 observations. We shouldn’t give it high confidence just because the few examples worked. The sample is too small.
This table also gives us a first warning about view families that sound economically reasonable but don’t pay consistently in this sample. That is exactly the kind of mismatch learned confidence is supposed to catch.
10) Learned confidence
Now we convert payoff statistics into a confidence number. Confidence is bounded between 0.30 and 0.90:
\[
0.30 \leq \omega_k \leq 0.90
\]
Here \(\omega_k\) is not the same as the Black-Litterman uncertainty matrix \(\Omega\). It is a human-readable confidence score. Later we convert it into \(\Omega\).
The learned confidence formula starts from a skeptical baseline:
\[
\omega_k^{base}=0.40
\]
Then it adds evidence from t-stat, payoff IR, information coefficient, hit rate, sample size, and confluence:
So views with tiny sample size get almost no sample-size bonus, while views with enough history get the full bonus.
The nonlinear \(\tanh\) terms are important. They prevent a single statistic from exploding confidence. A very high t-stat helps, but it saturates. A very bad IR hurts, but it also saturates. This keeps confidence in a controlled range.
We also apply safety rules:
if \(n_k<8\), confidence is capped at 0.52,
if payoff IR is weak and hit rate is weak, confidence is capped at 0.46,
if recent hit rate is below 35%, confidence is penalized,
risk-on views are penalized slightly during equity stress,
protective views need stress-period evidence to earn high stress confidence.
This is a major improvement over manually chosen confidence because the model has to earn trust from actual realized payoffs instead of receiving it by assumption.
So \(\omega_k\) should be read as a model trust score, not as a literal probability that the view will be right next month. It controls how much noise we assign to the view when we build \(\Omega\).
Dual momentum has confidence around 0.5857, the highest in the latest active set. It has 117 observations, hit rate 57.26%, recent hit rate 100%, and positive payoff IR. The model trusts it more because it has both a long history and good recent behavior.
Liquidity leadership also receives high confidence around 0.5802, supported by hit rate 62% and positive payoff IR. Risk-adjusted momentum gets confidence around 0.5431, which is moderate and sensible: it has a good sample and positive hit rate, but the information coefficient is negative, so signal magnitude hasn’t translated cleanly into larger payoffs.
International rotation is held down to 0.3309. It has 45 observations, but hit rate is below 50% and payoff IR is negative. The model still allows the view to exist, but it won’t let the view dominate the posterior.
Duration quality is floored at 0.3000 because its history is weak: hit rate around 35.53%, recent hit rate 33.33%, and negative payoff IR. That is a very important output. The view fires because the current state says short duration quality matters, but the learned confidence system says, “be careful, this view hasn’t worked well historically.”
This is the difference between a rule-based allocation system and learned-confidence BL. The rule creates the view. The payoff history controls how much the view is allowed to matter.
The important behaviour is the separation between current signal strength and learned trust. A view can look strong today and still receive low confidence if its historical payoff record is weak.
10.1 View selection and redundancy control
After computing confidence, we still need to select views. If too many overlapping views enter the posterior, the same economic bet can be counted multiple times.
The \(q\)-score measures how large the raw tilt is relative to its cap:
\[
q\text{-score}_k = \frac{|q_{tilt,k}|}{c_k}
\]
The payoff-quality score combines hit rate, recent hit rate, payoff IR, and sample score. Confluence measures how many independent signal dimensions support the view.
Then we remove redundant views using cosine similarity between exposure vectors:
If two views have similarity above 0.85, we keep the stronger one. This matters because dual momentum and risk-adjusted momentum can sometimes point to almost the same long-short basket. Without redundancy control, the posterior would double-count the same bet.
We also cap selected views at 7 and limit too many same-direction views. That keeps the posterior from becoming a pile of correlated risk-on bets.
The diversification term is not meant to reward random views. It gives a small preference to views that bring a broader or less repeated exposure, so the selected set is not just several versions of the same trade.
From the latest selection table, all five candidate views are kept. The ranking is sensible:
Risk-adjusted momentum, selected score 0.6742.
Dual momentum, selected score 0.6703.
Liquidity leadership, selected score 0.5355.
International rotation, selected score 0.4651.
Duration quality, selected score 0.2759.
Risk-adjusted momentum ranks slightly above dual momentum because its confluence and selection score are strong, even though dual momentum has higher confidence. Duration quality is clearly the weakest selected view, and the score reflects that.
The long/short side of each final view is the part to inspect before the posterior. It lets us directly connect the model’s active ideas to later posterior shifts:
EEM and QQQ appear repeatedly on the long side.
TLT appears repeatedly on the short side.
GLD is short in the momentum views, but later the full posterior can still be affected by covariance interactions.
International rotation directly pushes EEM above EFA, but with low confidence.
So the view set isn’t just a black box. We can read the exact active bets before seeing the posterior.
This is also why keeping the selected-view table in the notebook is useful. Before seeing the posterior, we can already tell which economic ideas are about to influence the model and which ones are only weakly trusted.
11) Learned \(q\) scaling and the view matrix
Confidence affects \(\Omega\), but we also use payoff history to scale the active \(q\) tilt itself. This is another important part that needs clear interpretation.
This separation is crucial. If the prior already strongly favors the long side, \(q\) can be high even with a modest active tilt. If the prior favors the short side, the active tilt has to overcome that base spread.
This means learned confidence affects the model in two places: it helps scale the active tilt itself, and it later helps build \(\Omega\). The first channel changes the size of the view target. The second channel changes how strongly BL listens to that target.
The learned \(q\) mechanism is easiest to read view by view.
Risk-adjusted momentum has base view spread about 0.0674, raw q tilt 0.0835, and a positive learned component 0.0083, giving final q tilt 0.0918. Its final view target \(q\) is 0.1592. The model is saying: the prior already likes this spread, and the view history supports a small boost.
Dual momentum is even more interesting. Its base spread is slightly negative at about -0.0103, but the raw tilt is 0.0916, boosted to the cap at 0.1000. The final target \(q\) becomes 0.0897. This means the active view fully overcomes the prior’s slight disagreement.
Liquidity leadership has base spread 0.1270 and final active tilt 0.0393, producing \(q\) around 0.1663. The prior and view agree.
International rotation has base spread 0.1123, but the learned component is negative -0.0501, shrinking final active tilt from 0.0771 to 0.0270. So even though EEM looks strong now, its historical payoff record forces a much smaller active push.
Duration quality is also heavily shrunk. Raw q tilt 0.0168 becomes only 0.0059 because hit rate and payoff IR are weak. This is exactly the behavior we want.
The view matrix output confirms the portfolio exposures. For example, international rotation is simply \(+1\) EEM and \(-1\) EFA. Duration quality is \(+1/3\) SHY, \(+1/3\) AGG, \(+1/3\) IEF, and \(-1\) TLT. This matrix is what lets BL translate relative views into asset-level posterior return shifts.
The key check is that the final \(q\) values still match the economic direction of the views. The learning layer should shrink or boost the signal; it should not quietly reverse the intended long-short belief.
12) From confidence to \(\Omega\)
Now we convert confidence into the Black-Litterman view uncertainty matrix \(\Omega\).
So \(\Omega\) controls how noisy the views are. Small \(\Omega_{kk}\) means view \(k\) is trusted. Large \(\Omega_{kk}\) means view \(k\) is noisy and the posterior won’t move much toward it.
We first compute the covariance of the view portfolios:
\[
\Sigma_P = P\Sigma P^\top
\]
The diagonal element is the variance of the long-short view portfolio:
This construction is better than assigning arbitrary diagonal numbers because it respects both sources of uncertainty:
the market covariance of the view portfolio,
the learned reliability of the view family.
A high-volatility long-short view naturally has more uncertainty. A low-confidence view gets even more uncertainty.
Using the full view covariance instead of only arbitrary diagonal numbers also lets overlapping views share uncertainty. If two views point to similar baskets, their covariance appears inside \(P\Sigma P^\top\).
For the latest active set, the \(\Omega\) numbers separate two effects: how much we trust the view family and how volatile the long-short view portfolio is.
Dual momentum has confidence 0.5857 and view variance about 0.0164, giving omega diagonal around 0.000580. This is relatively small, so the view receives meaningful weight in the posterior.
Risk-adjusted momentum has confidence 0.5431 and view variance 0.0227, giving omega diagonal around 0.000954. Even though confidence is moderate, the view portfolio variance is larger, so uncertainty is larger than dual momentum.
Liquidity leadership has similar confidence 0.5802, but view variance is higher at 0.0361, so omega diagonal rises to about 0.001307. This is the covariance structure speaking: the view is more volatile, so it gets more view uncertainty.
International rotation has low confidence 0.3309 and high view variance 0.0397, producing omega diagonal around 0.004011, the largest in the active set. So even though the current EEM signal is strong, the posterior will treat this view cautiously.
Duration quality has the lowest confidence 0.3000, but the view variance is tiny at 0.0040, so omega diagonal is only 0.000462. This is a subtle point: low confidence increases uncertainty, but the underlying long-short portfolio is much less volatile, so the final \(\Omega\) isn’t huge. BL combines both effects.
So the largest confidence score doesn’t always create the smallest omega diagonal. The volatility of the view portfolio matters too, which is why the same confidence can lead to different uncertainty across view families.
13) Black-Litterman posterior
Now we combine the equilibrium prior with the selected views. At this point the model has five objects: the prior mean \(\boldsymbol{\pi}\), the prior uncertainty scale \(\tau\Sigma\), the view matrix \(P\), the view target vector \(\mathbf{q}\), and the view uncertainty matrix \(\Omega\).
The prior says that the unknown expected-return vector \(\boldsymbol{\mu}\) is centered around the benchmark-implied return vector \(\boldsymbol{\pi}\):
Here \(\boldsymbol{\pi}\) is not a historical mean. It is the equilibrium return vector implied by the benchmark portfolio, covariance matrix, and risk aversion. The matrix \(\Sigma\) is the asset return covariance matrix, and \(\tau\) scales it down because this is uncertainty about expected returns, not the volatility of realized returns themselves. Smaller \(\tau\) means the prior is tighter and harder for views to move.
A view does not observe every asset’s expected return directly. It observes a portfolio spread, such as a long-short basket. That is why the view equation uses \(P\boldsymbol{\mu}\):
Each row of \(P\) is one view portfolio. Positive entries are the long side, negative entries are the short side, and zeros are assets not directly named in that view. The vector \(\mathbf{q}\) contains the return target for those view portfolios. The error term \(\boldsymbol{\varepsilon}\) is the part of the view that can be wrong. Its covariance matrix \(\Omega\) tells BL how noisy the views are. A larger diagonal value in \(\Omega\) means that view should pull the posterior less.
Bayes’ rule gives the posterior mean in precision form:
This formula is dense, so the pieces are worth reading slowly. The term \((\tau\Sigma)^{-1}\) is prior precision: a tight prior receives more weight. The term \(P^\top\Omega^{-1}P\) is view precision mapped back from view space into asset space. If \(\Omega\) is small, the views are precise and this term becomes larger. The right-hand side has the same two sources of information: prior precision times prior mean, plus view precision times view target. The inverse on the left turns those combined precision weights back into expected returns.
The same result can be written as a prior plus an adjustment:
This version is easier to connect to the notebook output. The posterior starts at \(\boldsymbol{\pi}\) and then moves only if the selected views disagree with the prior. The disagreement is the view gap:
\[
\mathbf{q}-P\boldsymbol{\pi}
\]
\(P\boldsymbol{\pi}\) is what the prior already believes about each long-short view portfolio. If \(q_k\) is above that prior-implied spread, the view is asking BL to raise expected returns on the long side relative to the short side. If \(q_k\) is below it, the view pushes in the opposite direction. If the gap is near zero, the view adds almost no new information because the prior already says the same thing.
The middle matrix controls how strongly that gap is transmitted into assets:
\(K\) works like a gain matrix. The \(\tau\Sigma P^\top\) part maps view shocks back to individual assets using covariance relationships. The \((P\tau\Sigma P^\top+\Omega)^{-1}\) part scales the shock down when the view portfolio is already uncertain or when learned confidence is low. This is why an asset can move even if it is not explicitly long or short in a view: covariance links it to assets that are in the view.
The next code cell computes exactly this object. The displayed shift table is not yet a portfolio. It is the change in expected returns after blending the equilibrium prior, the selected views, the covariance matrix, and the learned view confidence. Portfolio weights come one step later, when the optimizer turns these posterior returns into a constrained allocation.
Numerically, the update form is also convenient because the main inverse is in view space, not full asset space. In this notebook, the number of selected views is smaller than the number of assets, so solving the view-space system is cleaner and less fragile.
DBC gets the largest positive shift in the first posterior diagnostic, about +4.55%, because it appears on the long side of both momentum views and starts from a negative prior. QQQ gets +3.89%, IWM gets +2.65%, EEM gets +2.25%, and SPY gets +1.41%. These shifts match the latest risk-on and momentum signals.
GLD receives the largest negative shift, about -6.50%, because it’s short in the momentum views and its recent trend is weak. VNQ also gets a negative shift around -1.91% in that diagnostic because risk-adjusted momentum shorts it, even though dual momentum longs it. This is a good example of views partially conflicting and the posterior averaging them through \(P\), \(q\), and \(\Omega\).
TLT, IEF, AGG, and LQD receive small negative shifts. That aligns with duration quality and weak fixed-income signals, but the shifts are not extreme because some bond exposures remain part of the benchmark and the prior is still tight.
EEM is clipped at 0.3000 in some posterior outputs. That tells us the model has a return cap for numerical stability. Clipping is important because posterior expected returns can otherwise become too large for the optimizer.
The key interpretation: the posterior is coherent with the view set, but it’s not simply copying the views. It blends views with the equilibrium prior and covariance structure.
Before looking at weights, this isolates the belief update. If a posterior shift looks reasonable but the final weight doesn’t move much, the reason is usually in the optimizer constraints or covariance risk, not in the view construction.
14) Portfolio optimization from the posterior
Once we have posterior expected returns and posterior covariance, we solve a constrained mean-variance problem:
The active limit is very important. It stops the posterior from becoming an all-in allocation just because one view is strong. If QQQ receives a strong posterior shift, the optimizer can overweight it, but only within the active budget.
We also include practical fallbacks. If the strict optimizer fails, we relax the active limit, remove sleeve constraints, or fall back to a tracking-error style solution. If all else fails, we keep previous weights. The output later confirms that the final BL backtest has zero fallback count, so the optimization path is stable.
These constraints are not cosmetic. Without them, a few large posterior shifts could produce a portfolio that is technically optimal but financially unrealistic for this notebook: too concentrated, too far from the benchmark, or too dependent on one sleeve.
The full walk-forward loop repeats the complete Black-Litterman process at each monthly rebalance:
take the current prior \(\boldsymbol{\pi}_t\) and covariance \(\Sigma_t\),
generate candidate views from current signals,
use only already-realized payoff history to learn confidence,
select non-redundant views,
build \(P_t\), \(\mathbf{q}_t\), and \(\Omega_t\),
compute \(\boldsymbol{\mu}_{post,t}\) and \(\Sigma_{post,t}\),
optimize weights under constraints,
hold until the next rebalance.
The no-lookahead part is essential. At date \(t\), learned confidence only uses payoffs whose end dates are before \(t\):
\[
\text{payoff\_end\_date}_{k,j} < t
\]
This prevents the confidence system from using future information. Without this condition, the learned confidence would be a form of leakage and the backtest would be invalid.
The model stores selected views, confidence logs, posterior shifts, and weights. These logs are what allow us to analyze the model after the backtest instead of just looking at final performance.
The stored logs are also what make the later reliability analysis possible. Without them, we would only know the final return path, not which views caused each posterior and weight change.
Now we combine the baseline strategies and Learned-Confidence BL into one comparison.
The gross/net table separates trading performance from transaction costs. Gross CAGR ignores costs. Net CAGR includes 10 bps per trade. Cost drag is the total return lost to trading costs.
The BL model has 11.31% gross CAGR and 10.95% net CAGR. The cost drag is much larger than the benchmark because BL turnover is much larger: average monthly turnover is 27.12%, and total turnover is 33.90 over the backtest.
This cost difference is real. The benchmark has only 1.44% average monthly turnover and total turnover 1.80. MV has 3.69% average monthly turnover and total turnover 4.65. BL trades much more because views update monthly and the optimizer responds to posterior changes.
The important point is that BL still keeps a higher net CAGR than the benchmark after costs: 10.95% versus 9.14%. So the active model’s extra turnover is expensive, but the gross alpha is large enough to survive the assumed transaction cost.
The model earns more, but it pays for that through turnover. That tradeoff has to be part of the evaluation.
This is the point where turnover stops being an implementation detail and becomes part of the model result. A view system that improves gross returns but loses the edge after costs would not be convincing.
Show code
combined_results = {}combined_results.update(baseline_results)combined_results.update(bl_results)final_strategy_names = ["Strategic Benchmark", "Equal Weight", best_minvar, best_mv, "Learned-Confidence BL"]returns_by_strategy = {name: combined_results[name]["returns"] for name in final_strategy_names}nav_by_strategy = {name: combined_results[name]["nav"] for name in final_strategy_names}weights_by_strategy = {name: combined_results[name]["weights"] for name in final_strategy_names}turnover_by_strategy = {name: combined_results[name]["turnover"] for name in final_strategy_names}cost_by_strategy = {name: combined_results[name]["costs"] for name in final_strategy_names}def cagr_from_nav(nav): nav = pd.Series(nav).dropna()if nav.empty:return np.nan years =max(len(nav) / annualization, 1e-9)returnfloat(nav.iloc[-1] ** (1.0/ years) -1.0)gross_net_rows = []for name in final_strategy_names: result = combined_results[name] gross_net_rows.append({"strategy": name, "Gross CAGR": cagr_from_nav(result.get("gross_nav", result["nav"])),"Net CAGR": cagr_from_nav(result["nav"]), "Cost Drag": float(result["costs"].sum()) iflen(result["costs"]) else np.nan,"Average Monthly Turnover": float(result["turnover"].mean()) iflen(result["turnover"]) else np.nan,"Total Turnover": float(result["turnover"].sum()) iflen(result["turnover"]) else np.nan})gross_net_table = pd.DataFrame(gross_net_rows).set_index("strategy")display(gross_net_table.round(4))def full_strategy_metrics(name, result): metrics = selection.performance_metrics(result["returns"], result["nav"], rf_daily=risk_free_daily, annualization=annualization) weights = result["weights"] eff_n = (float((1.0/ weights.pow(2).sum(axis=1)).replace([np.inf, -np.inf], np.nan).mean())ifnot weights.empty else np.nan) avg_max_weight =float(weights.max(axis=1).mean()) ifnot weights.empty else np.nanreturn {"strategy": name,"CAGR": metrics["CAGR"],"Annualized Volatility": metrics["Vol"],"Sharpe": metrics["Sharpe"],"Sortino": metrics["Sortino"],"Max Drawdown": metrics["Max Drawdown"],"Calmar": metrics["Calmar"],"Avg Monthly Turnover": float(result["turnover"].mean()) iflen(result["turnover"]) else np.nan,"Total Turnover": float(result["turnover"].sum()) iflen(result["turnover"]) else np.nan,"Cost Drag": float(result["costs"].sum()) iflen(result["costs"]) else np.nan,"Effective Number of Holdings": eff_n,"Average Max Weight": avg_max_weight,"Fallback Count": result.get("fallback_count", 0), }summary_table = pd.DataFrame([full_strategy_metrics(name, combined_results[name]) for name in final_strategy_names] ).set_index("strategy")display(summary_table.round(4))def avg_active_risk_against_benchmark(weights): rows = [] bench = benchmark_w.reindex(assets).fillna(0.0)for dt, row in weights.iterrows(): dt = pd.Timestamp(dt)if dt notin cov_by_date:continue diff = row.reindex(assets).fillna(0.0).values - bench.values cov_ann = cov_by_date[dt].loc[assets, assets].values rows.append(float(np.sqrt(max(diff @ cov_ann @ diff, 0.0))))returnfloat(np.mean(rows)) if rows else np.nandef benchmark_relative_metrics(name): bench_ret = pd.Series(combined_results["Strategic Benchmark"]["returns"]).dropna() ret = pd.Series(combined_results[name]["returns"]).reindex(bench_ret.index).fillna(0.0) active = ret - bench_ret active_nav = (1.0+ active).cumprod() tracking_error =float(active.std(ddof=1) * math.sqrt(annualization)) if active.std(ddof=1) >1e-12else np.nan active_return =float(active.mean() * annualization) info_ratio = active_return / tracking_error if np.isfinite(tracking_error) and tracking_error >1e-12else np.nan corr =float(ret.corr(bench_ret)) if ret.std(ddof=1) >1e-12and bench_ret.std(ddof=1) >1e-12else np.nan beta =float(ret.cov(bench_ret) / bench_ret.var(ddof=1)) if bench_ret.var(ddof=1) >1e-12else np.nan weights = combined_results[name]["weights"].reindex(columns=assets).fillna(0.0) bench = benchmark_w.reindex(assets).fillna(0.0) avg_active_dist =float(weights.subtract(bench, axis=1).abs().sum(axis=1).mean()) ifnot weights.empty else np.nanreturn {"strategy": name,"active_return_ann": active_return,"tracking_error": tracking_error,"information_ratio": info_ratio,"active_max_drawdown": float((active_nav / active_nav.cummax() -1.0).min()),"hit_rate_vs_benchmark": float((active >0).mean()),"corr_to_benchmark": corr,"beta_to_benchmark": beta,"avg_active_weight_distance": avg_active_dist,"avg_active_risk": avg_active_risk_against_benchmark(weights), }benchmark_relative_table = pd.DataFrame([benchmark_relative_metrics(name) for name in final_strategy_namesif name !="Strategic Benchmark"]).set_index("strategy")display(benchmark_relative_table.round(4))
Gross CAGR
Net CAGR
Cost Drag
Average Monthly Turnover
Total Turnover
strategy
Strategic Benchmark
0.0916
0.0914
0.0026
0.0144
1.7991
Equal Weight
0.0849
0.0848
0.0022
0.0114
1.4385
MinVar (OAS)
0.0665
0.0665
0.0005
0.0031
0.3897
MV (LedoitWolf, BayesStein)
0.0819
0.0814
0.0062
0.0369
4.6481
Learned-Confidence BL
0.1131
0.1095
0.0602
0.2712
33.8951
CAGR
Annualized Volatility
Sharpe
Sortino
Max Drawdown
Calmar
Avg Monthly Turnover
Total Turnover
Cost Drag
Effective Number of Holdings
Average Max Weight
Fallback Count
strategy
Strategic Benchmark
0.0914
0.1014
0.5284
0.6493
-0.2242
0.4075
0.0144
1.7991
0.0026
9.7136
0.2222
0
Equal Weight
0.0848
0.0935
0.4842
0.5934
-0.2038
0.4159
0.0114
1.4385
0.0022
14.0000
0.0714
0
MinVar (OAS)
0.0665
0.0754
0.3543
0.4044
-0.1973
0.3370
0.0031
0.3897
0.0005
9.1680
0.2394
0
MV (LedoitWolf, BayesStein)
0.0814
0.0847
0.4949
0.6049
-0.1378
0.5904
0.0369
4.6481
0.0062
5.0557
0.3212
0
Learned-Confidence BL
0.1095
0.1066
0.6613
0.8336
-0.2154
0.5083
0.2712
33.8951
0.0602
5.8309
0.2660
0
active_return_ann
tracking_error
information_ratio
active_max_drawdown
hit_rate_vs_benchmark
corr_to_benchmark
beta_to_benchmark
avg_active_weight_distance
avg_active_risk
strategy
Equal Weight
-0.0083
0.0181
-0.4581
-0.1088
0.4774
0.9860
0.9094
0.4560
0.0170
MinVar (OAS)
-0.0268
0.0455
-0.5893
-0.2614
0.4705
0.9086
0.6764
0.7007
0.0400
MV (LedoitWolf, BayesStein)
-0.0116
0.0812
-0.1432
-0.2863
0.4858
0.6317
0.5277
1.1509
0.0687
Learned-Confidence BL
0.0169
0.0402
0.4218
-0.0667
0.5295
0.9266
0.9746
0.9059
0.0369
Learned-Confidence BL is the strongest model in the full strategy metrics table.
Learned-Confidence BL has:
10.95% CAGR, highest among all tested strategies,
10.66% volatility, only slightly above the strategic benchmark,
0.6613 Sharpe, clearly above the benchmark’s 0.5284,
0.8336 Sortino, also stronger than the benchmark,
-21.54% max drawdown, slightly better than the benchmark’s -22.42%,
0 fallback count, meaning the optimizer didn’t rely on emergency behavior.
This is a strong result. BL improves return and risk-adjusted performance without increasing drawdown relative to the benchmark. It doesn’t achieve the lowest drawdown. MV has the best drawdown at -13.78%, but MV’s CAGR is only 8.14% and its Sharpe is lower than BL.
The effective number of holdings is also informative. BL averages around 5.83 effective holdings, while the benchmark has 9.71 and equal weight has 14. So BL is more concentrated. That concentration is the cost of making active tilts. The active weight and sleeve constraints prevent it from becoming extreme, but the model is clearly taking directional relative bets.
So the performance improvement isn’t free. It comes from more concentrated, higher-turnover active allocation. The result is convincing because the improvement survives costs and doesn’t create a worse drawdown than the benchmark.
The concentration result is not automatically bad, but it’s a risk we need to name. BL earns its improvement by taking clearer active positions than the benchmark, so the higher effective concentration is part of the strategy, not an accident.
15.1 Benchmark-relative metrics
Absolute performance can hide whether the active model actually adds value over the benchmark. So we also compute active return, tracking error, information ratio, hit rate versus benchmark, correlation, beta, and active risk.
The BL model has annualized active return around 1.69%, tracking error around 4.02%, and information ratio around 0.4218. That is a meaningful active result. The monthly hit rate versus benchmark is 52.95%, so the model wins slightly more often than it loses.
The correlation to benchmark is 0.9266 and beta is 0.9746. This tells us BL is still fundamentally a benchmark-aware multi-asset portfolio. It isn’t becoming a completely different strategy. The alpha comes from active tilts around the benchmark, not from abandoning the benchmark.
The information ratio is the most useful number in this table because it scales active return by the amount of benchmark-relative risk we had to take. A positive active return with huge tracking error would be much less impressive.
BL finishes above the benchmark, equal weight, MinVar, and MV. The strategy doesn’t dominate every period, but it compounds better over the full sample. BL still participates in major stress events, which is expected because it remains long-only and benchmark-aware. It isn’t a market-neutral strategy.
The drawdown difference versus the benchmark is not huge. The BL model still falls during COVID and the 2022 regime. But the recovery and subsequent compounding are better. That matters more than trying to avoid every drawdown.
The main weakness visible from the path is turnover and responsiveness. Because the model updates monthly, it can lag sudden events. A shock like COVID can hit before signals have time to adjust. But after the shock, the signal engine can rotate faster than a static benchmark.
The path is consistent with the table: BL adds value over the full sample, but it remains exposed to broad market risk.
This is why the drawdown chart should be read together with the turnover and active-risk tables. The strategy is not trying to avoid every loss; it’s trying to rotate the benchmark allocation enough to improve the full path.
15.2 Tail risk and stress windows
The risk-report table compares daily 5% VaR and expected shortfall. Historical VaR is the empirical 5th percentile of returns:
\[
VaR_{5\%}^{hist}= -Q_{0.05}(r_t)
\]
Expected shortfall is the average loss beyond VaR:
The filtered historical simulation values are larger because they account for volatility clustering. BL has historical VaR around 1.02% and historical ES around 1.62%. Its FHS VaR is around 1.97% and FHS ES around 2.98%, higher than the benchmark. That tells us the active tilts increase conditional tail exposure in volatile regimes.
This is an honest limitation. BL improves return and Sharpe, but it doesn’t minimize tail risk. MinVar remains the cleanest tail-risk strategy, with lower VaR and ES. The strategic benchmark sits in the middle.
The stress plots are important because each stress window tests a different regime:
2018 Q4 selloff, Fed tightening and equity derisking.
COVID crash, sudden exogenous shock.
2022 rates/inflation, stock-bond correlation failure and duration crash.
2023 growth rebound, recovery led by growth assets.
BL is designed to adapt to regimes, but it can’t react before a sudden shock appears in the data. Its strength is more visible in slower regime shifts like 2022 inflation and 2023 growth recovery.
The tail-risk result is not a failure of BL by itself. It tells us what objective we optimized. We optimized posterior mean-variance with constraints, not CVaR or drawdown minimization, so MinVar can still look better in pure tail-risk tables.
16) Selected-view reliability after the full backtest
After the full BL backtest, we analyze the selected views, not just the candidate views. This matters because candidate views are generated by rules, while selected views are the ones that actually entered the posterior.
Most candidate views survive selection, but not all. The selected count is usually close to candidate count, but redundancy and direction caps remove some views. This means the view engine is active but not blindly accepting every signal.
The selected-view summary tells us how often each view actually affected the posterior. Risk-adjusted momentum is selected 124 times, almost every rebalance. Dual momentum is selected 106 times. Duration quality is selected 77 times. Liquidity leadership, international rotation, credit switch, growth-duration, reflation, inflation, and correlation stress are less frequent.
The selected-share column helps us read the model’s personality. Risk-adjusted momentum has selected share 0.992, dual momentum 0.848, and duration quality 0.616. This project’s active engine is therefore heavily momentum-driven, with fixed-income regime controls layered on top.
That is good and bad. It is good because cross-asset momentum has empirical support and works across many regimes. It is risky because momentum can lag reversals. The learned-confidence layer helps, but it doesn’t remove that structural lag.
This table is more honest than only looking at candidate views. A view that fires often but is repeatedly removed by redundancy control is not actually shaping the posterior as much as its candidate count suggests.
The selected payoff summary is even more useful.
Dual momentum has realized hit rate 60.00%, average forward payoff around 6.50% annualized, and payoff IR 0.1727. This is one of the strongest selected views because it has both sample size and positive realized performance.
Credit switch: risk-on is very strong among selected views: hit rate 65.12%, average payoff 15.04%, and payoff IR 0.3733. This supports the idea that credit leadership is a useful signal for risk-on allocation.
Reflation breadth has hit rate 71.43% and payoff IR 0.3905, but only 21 selected observations. This is promising, but less mature than the momentum views.
Liquidity leadership has hit rate 62% and payoff IR 0.1297, making it a useful but moderate view.
Weak selected views include Duration quality, International rotation, Inflation rotation, and Credit switch: risk-off. Duration quality has hit rate 35.53% and negative payoff IR. International rotation has hit rate 44.44% and negative payoff IR. Credit switch risk-off has very poor realized profile. These weak results explain why the learned-confidence system shrinks or floors these views.
This output is exactly why learned confidence is necessary. Without it, all selected views could be treated too similarly. With it, historically useful views get more influence, while weak views still exist but have lower posterior impact.
The weak views are not necessarily useless forever. They may still be economically meaningful in specific regimes, but this sample says they should not get the same influence as the views with stronger realized payoff records.
16.1 Posterior shift heatmap and latest posterior table
The posterior clarity table compares benchmark weight, prior return, posterior return, and posterior shift for the latest logged rebalance. The shift is:
\[
\Delta\mu_i = \mu_{post,i} - \pi_i
\]
This is the cleanest way to understand what the view system is doing at the asset level.
In the latest full-log table, EEM receives a very large positive shift, about +9.86%, DBC about +9.76%, EFA about +9.47%, GLD about +11.84%, QQQ about +7.76%, and SPY about +3.94%. Some of these are direct view effects, while others come from covariance propagation through the posterior formula.
Across time, these shifts form one of the best diagnostics in the project because they separate expected-return belief changes from final weights. A posterior shift can be strong, but the optimizer may still limit the final allocation because of caps, active weight limits, sleeve constraints, or covariance risk.
The heatmap has visible regime blocks. Around 2022, real assets and inflation-sensitive exposures receive stronger positive shifts, while duration is penalized. In growth-led periods, QQQ and equity-linked assets receive positive shifts. The shifts are not random noise. They cluster around economic regimes.
That said, the heatmap is also noisy. Some assets switch sign frequently, especially where momentum views and covariance effects interact. This is a normal feature of monthly tactical allocation.
This is also where covariance propagation becomes visible. An asset can receive a posterior shift even if it’s not on the long or short side of a particular view, because the model updates the whole expected-return vector jointly.
The final reliability table combines candidate counts, selected counts, payoff statistics, average confidence, stress-period behavior, and selected share. It is the model audit.
A few rows stand out:
Risk-adjusted momentum: 125 candidates, 124 selected, hit rate 57.26%, average confidence 0.4515.
Dual momentum: 118 candidates, 106 selected, hit rate 57.26%, recent hit rate 100%, average confidence 0.4238.
Reflation breadth: hit rate 71.43%, payoff IR 0.3905, average confidence 0.6451, but smaller sample.
Correlation stress: hit rate 75% and high payoff IR, but only 4 observations, so it has limited statistical reliability.
Duration quality and International rotation: weak hit rates and negative payoff IR, so the confidence layer keeps them small.
After optimization, QQQ is at the active cap with +15.00% active weight. EFA is +10.91%, EEM +9.95%, SHY +9.95%, GLD +4.95%, and DBC +2.98%. The biggest underweights are SPY -12.97%, AGG -10.10%, TLT -8.08%, IEF -8.08%, and LQD -5.05%.
The latest weight pattern is not simply “risk-on everything.” It is a relative allocation:
overweight QQQ, EEM, EFA, GLD, DBC, and SHY,
underweight broad SPY, core bonds, long duration, and credit,
keep active risk controlled around the benchmark.
Average active risk is around 3.69% and average turnover is 27.12%. That is a moderate active-risk profile but a high-turnover profile. For a monthly tactical model, that is believable, but costs need to be monitored.
The active-weight table is the final check that the view logic survived optimization. If the views said one thing but the active weights showed the opposite, we would need to inspect caps, sleeve constraints, and covariance risk.
17) Sector ETF transfer with quantfinlab
Now we move the learned-confidence Black-Litterman workflow to sector ETFs through the packaged quantfinlab library. The decision problem changes here. In the main cross-asset universe, the model can rotate between equities, bonds, credit, commodities, gold, and cash-like duration. In the sector universe, every tradable asset is still an equity ETF, so the model is choosing which parts of the stock market should lead or lag.
The sector data comes from the same repository data area: sector and cross-asset ETF data. The tradable universe is the 11 SPDR sector ETFs:
ETF
Sector
Usual behavior
Main drivers we care about here
XLK
Technology
Growth-heavy, high-duration equity exposure; often leads when mega-cap growth and risk appetite are strong.
QQQ leadership, falling or stable rates, strong momentum, benign volatility.
XLC
Communication services
Mix of platform, media, and telecom exposure; behaves partly like growth and partly like consumer/advertising cyclicality.
The important grouping is growth, cyclical, inflation-sensitive, and defensive. XLK, XLC, and XLY are the growth/consumer-risk side. XLI, XLF, XLB, and XLE are more cyclical or reflation-sensitive. XLV, XLP, and XLU are defensive, although they still fall in broad equity selloffs. XLRE sits between real assets and rate-sensitive equity because property cash flows depend heavily on financing costs.
This makes the sector transfer more subtle than the main allocation problem. A defensive sector overweight is not the same as buying SHY or IEF. It only means we prefer defensive equities relative to other equities. If the whole stock market sells off, sector rotation can reduce damage, but it usually can’t remove equity beta completely.
The signal assets still include cross-asset indicators like SPY, QQQ, IWM, TLT, IEF, SHY, LQD, HYG, GLD, DBC, and UUP. That means we trade sectors, but we still use broad macro signals to decide which sector views make sense. QQQ/SPY leadership helps growth views. IWM/SPY and HYG/LQD help cyclical and credit-beta views. TLT, IEF, and SHY help rate-sensitive views. DBC and GLD help inflation and real-asset views. UUP helps identify dollar pressure, which can matter for commodities, materials, and multinational earnings.
The benchmark is equal weight:
\[
w_{b,i}=\frac{1}{11}\approx 0.0909
\]
This is a natural starting point for sector rotation because the sectors all belong to the same broad equity market. There isn’t an obvious strategic reason to heavily overweight one sector before signals arrive. Equal weight also makes the active bets easy to read: if XLK is above 9.09%, the model is explicitly saying technology should carry more than its neutral sector share.
We use the library version here intentionally. The point is not to reimplement every function again, but to check whether the extracted quantfinlab version can repeat the same learned-confidence idea on a related but narrower universe. If the method only works in the original cross-asset setup, it is less convincing. If it still gives reasonable sector rotations, then the view/confidence framework is more portable.
Sector coverage is clean: 2001 observations for all tradable and signal assets from 2018-07-02 to 2026-06-17, with no missing values after first valid. The sector return panel has 2000 daily observations, and the monthly rebalance sequence starts on 2020-07-31 because we need a 504-day lookback.
This shorter history is the main difference from the cross-asset test. The cross-asset model starts its active backtest around 2016. The sector transfer test effectively starts around mid-2020. That means the sector results are useful, but they are based on fewer regimes.
XLK is the strongest tradable sector in the latest signal table, with score 1.2947. XLI, XLE, and XLB are also positive. XLC and XLU are the weakest sectors, with scores around -0.6992 and -0.6984. XLY is also weak at -0.5827.
This is different from the cross-asset universe. In sectors, all assets share equity beta, so the model is mostly deciding which equity sectors to own, not whether to own equities, bonds, commodities, or cash-like assets. That makes sector BL more about rotation and less about broad asset allocation.
Because all tradable assets are sectors, the model can’t express a broad defensive move by buying Treasuries directly. Defensive information can still enter through signal assets, but the final trade has to be a sector rotation.
17.1 Sector view families
The sector view specification table lists the view families used by the library. The logic is similar to the main project, but the economic interpretation is sector-specific.
The most important sector views are:
Sector momentum, long the strongest sector momentum/volatility scores and short the weakest.
Growth leadership, long XLK, XLC, XLY when growth leadership is confirmed.
Defensive rotation, long XLP, XLU, XLV when equity stress is present.
Cyclical breadth, long XLI, XLF, XLB, XLE when participation is broad.
Credit beta, long credit-sensitive sectors when high yield confirms risk-on conditions.
Inflation beneficiaries, long XLE and XLB when commodities and inflation-sensitive assets lead.
Duration sensitive, long rate-sensitive sectors when duration conditions are supportive.
Small-cap risk-on, long domestic cyclicals when IWM/SPY leadership is positive.
Quality defensive, long XLV and XLP when volatility and defensive quality dominate.
Sector reversal, a short-horizon mean-reversion view after large sector extremes.
The latest active candidate views are growth leadership, cyclical breadth, credit beta, small-cap risk-on, and sector reversal. Growth leadership goes long XLK, XLC, XLY and short XLP, XLU, XLV. Cyclical breadth goes long XLI, XLF, XLB, XLE and short the same defensive basket. Sector reversal is different: it goes long XLE and XLC while shorting XLK and XLI.
The odd-looking number is cyclical breadth raw q around 0.6488, much larger than the other sector view tilts. That happens because the sector view specification has a much larger cap for this family. This doesn’t automatically mean the final portfolio goes all-in. The BL posterior, confidence, optimizer caps, and constraints still control the final weights.
The sector view families therefore have a different job from the cross-asset views. They are not choosing between stocks, bonds, commodities, and gold; they are choosing which parts of the equity market should carry more or less weight.
17.2 Sector baselines and BL backtest
The sector grid selects MinVar with Ledoit-Wolf covariance and MV with EWMA covariance plus Bayes-Stein means. The selected sector baselines are:
The sector BL weight matrix has 71 monthly weight rows across 11 sectors. Candidate views total 310, and selected views total 280. That means about 90% of candidate sector views survive selection, similar to the main project.
The latest sector weights are concentrated. XLC is 23.18%, XLK 19.28%, XLY 17.92%, XLE 11.58%, XLB 9.60%, XLRE 9.51%, and XLU 7.84%. XLV, XLP, and XLF are at 0%, and XLI is very small.
This is a strong growth/cyclical allocation, with some energy and materials exposure. It is not a defensive sector portfolio. The latest active weights make this clearer: XLC is +14.09%, XLK +10.19%, XLY +8.83%, while XLV, XLP, and XLF are each -9.09% relative to equal weight.
The sector model is more aggressive than the cross-asset model because it can’t move into bonds or cash-like assets. All tradable assets are equity sectors, so active views express themselves as sector concentration.
This concentration is exactly why the sector transfer is a harder test. In the cross-asset universe, the model can reduce risk by moving across sleeves. In sectors, it mostly has to express disagreement through overweights and underweights inside equities.
17.3 Sector performance and reliability
Learned-Confidence BL has the best overall profile among the sector strategies:
Benchmark: 14.60% net CAGR, 14.75% volatility, 0.7317 Sharpe, -19.25% drawdown.
MinVar: 9.59% net CAGR, 12.84% volatility, 0.4753 Sharpe, -18.70% drawdown.
MV: 13.51% net CAGR, 15.13% volatility, 0.6443 Sharpe, -16.38% drawdown.
This is a more modest improvement than the main cross-asset application, but it’s still positive. BL improves CAGR and Sharpe over the equal-weight sector benchmark, while also improving drawdown from -19.25% to -16.57%.
The active metrics are smaller than they look from turnover alone. BL active return is 0.62%, tracking error 5.30%, and information ratio 0.1169. The hit rate versus benchmark is only 50.27%. So sector BL adds value, but the active edge is not huge. It is a small improvement through rotation, not a dramatic alpha engine.
The sector view reliability table confirms this mixed picture. Growth leadership is strong, with hit rate 64.86%, average payoff 12.43%, payoff IR 0.2222, and average confidence 0.5581. Credit beta is also useful, with hit rate 58.33% and average confidence 0.5841. Duration sensitive is frequent but weak, with hit rate 48.89% and payoff IR near zero. Quality defensive and sector reversal are poor, with negative payoff IR and low average confidence.
Selection keeps 280 views out of 310 candidates, a 90.32% kept share, with 30 views removed by redundancy gating. Average selected views are about 3.94 per rebalance. That tells us the library implementation is applying the same redundancy logic rather than simply accepting every sector view.
The stress table is the final result here. In 2022 rates/inflation, the sector BL return is -9.13%, much better than the benchmark’s -18.20% and close to MV’s -7.30%. In the 2023 growth rebound, BL earns 13.19%, very close to the benchmark’s 13.79% and much better than MV’s 2.91%. In the 2024-2025 cycle, BL lags the benchmark and MV, with 28.10% versus benchmark 31.82% and MV 38.23%.
So the sector transfer works, but with a realistic tradeoff. BL helps in the 2022 sector-rotation stress regime and keeps up in the 2023 rebound, but it doesn’t dominate the 2024-2025 cycle. That is a good ending point for this project: learned-confidence BL is useful and portable, but its edge depends on the universe, the regime, and whether the selected views continue to earn their confidence.
That is the right kind of ending for this notebook. The method works outside the original universe, but the edge is smaller and more regime-dependent, which is exactly what we should expect when the universe becomes narrower.
