Portfolio Optimization Explained + Python Code
The process of optimizing a portfolio is essentially selecting an asset distribution in order to achieve a certain goal. This optimal portfolio depends on the objective, but usually is associated with maximizing your returns or minimizing the risk. In this article, we’ll try to optimize a portfolio by maximizing the sharpe ratio of our portfolio.
We’ll first discuss about some basic concepts in portfolio theory, and then we’ll write a code to optimize the portfolio.
Modern Portfolio Theory (MPT)
A portfolio is a combination of investments. These investments can have different natures, such as stocks, bonds, cash, etc. Every asset within a same class has two main attributes that differ them from each other, which are the return and the variance or risk.
The Modern Portfolio Theory (or mean-variance analysis) is a concept in finance which propagates the idea of building a portfolio that maximizes the expected return for a given level of risk. A key part of the theory is that diversification in investing reduces the risk of your portfolio. Investors could achieve better results by choosing an optimal combination of investments based on an assessment of their own individual tolerance to risk.
Rate of Return
A portfolio’s return is the logarithmic change in value after a specified period. We can calculate the returns of each asset with:
For each asset, we can calculate its expected return, by taking the mean of daily returns in a sample:
We can think of a portfolio as a list, or vector, of weights that represent how our capital is distributed among the assets:
Then, the expected return of our portfolio would be the sum of weights of each asset multiplied by its expected return:
Risk
Each asset has its own variance. We can combine the variance of each asset by just adding up each independent variance. However, we also have to account for possible covariances between different assets.
In a portfolio with two assets, the variance would be:
For a portfolio with m assets, we first define a covariance matrix:
Then, we can calculate the variance of our portfolio as:
With the variance, we can also calculate the standard deviation of our portfolio.
Both are measures the volatility or risk in our portfolio.
Example with Two Stocks
Let’s use some real data as an example.
import yfinance as yf
import matplotlib.pyplot as plt
import numpy as np
# With yfinance, we retrieve historical closing prices of WMT and AMZN stocks
prices = yf.download(['WMT', 'AMZN'], start='2010-01-01', auto_adjust=False)['Adj Close']
# Calculate the daily returns of each stock
returns = np.log(prices / prices.shift())We will define two functions to calculate the return and the risk of our portfolio.
def portfolio_return(returns, weights):
"""
Takes the dataframe of daily returns and multiply by the respective weight,
while also annualizing by multiplying by the number of trading days.
"""
return np.sum(returns.mean() * weights) * 252
def portfolio_risk(returns, weights):
"""
Calculates the covariance matrix of the daily returns, adjusting for a year,
and then calculates the standard deviation of the portfolio using the given
weights.
"""
cov_matrix = returns.cov() * 252 # Calculate the covariance matrix
risk = np.sqrt(np.dot(weights.T, np.dot(cov_matrix, weights)))
return riskThen, for multiple combinations of weights, we can calculate the risk and return of those portfolios.
# Create a set of weights for the first stock
possible_weights = np.linspace(0, 1, 100)
portfolios = []
for w1 in possible_weights:
# The two weights sum up to 1 or 100%
w2 = 1 - w1
weights = np.array([w1, w2])
# Calculate the risk and return for a given combination of weights (portfolio)
curr_risk = portfolio_risk(returns, weights)
curr_return = portfolio_return(returns, weights)
portfolios.append([curr_risk, curr_return])
portfolios = np.array(portfolios)Also, we’ll highlight the portfolio with the lowest volatility and use it as a divider between portfolios with returns lower and higher than it.
# Look for the portfolio with the lowest volatility
min_risk_idx = np.argmin(portfolios[:, 0])
min_risk_risk = portfolios[min_risk_idx, 0]
min_risk_return = portfolios[min_risk_idx, 1]
# Divide the portfolios into efficient and inefficient according to the efficient frontier
efficient_mask = portfolios[:, 1] >= min_risk_return
efficient_portfolios = portfolios[efficient_mask]
inefficient_portfolios = portfolios[~efficient_mask]Finally, we can plot all the portfolios in a risk x return graph.
# Create a return x volatility plot for all portfolios
plt.figure(figsize=(12, 7))
plt.plot(inefficient_portfolios[:, 0], inefficient_portfolios[:, 1], c='gray', label='Inefficient Frontier')
plt.plot(efficient_portfolios[:, 0], efficient_portfolios[:, 1], c='red', label='Efficient Frontier')
# Highlight the portfolios with minimum risk and with 100% allocated to each stock
plt.scatter(min_risk_risk, min_risk_return, c='red', s=150, edgecolors='black', label='Min Volatility')
plt.scatter(portfolios[0, 0], portfolios[0, 1], c='green', s=150, edgecolors='black', label='100% AMZN')
plt.scatter(portfolios[-1, 0], portfolios[-1, 1], c='purple', s=150, edgecolors='black', label='100% WMT')
plt.title('Efficient Frontier: WMT & AMZN')
plt.xlabel('Annualized Volatility (Risk)')
plt.ylabel('Annualized Return')
plt.legend()
plt.grid(True)
plt.show()Efficient Frontier
As we can see int the plot above, the portfolios corresponding to the gray line, represent sub-optimal portfolios. That’s because there are other combinations of weights which have the same risk, but with higher returns. So the optimal portfolios, which are represented by the red line or the Efficient Frontier, are the portfolios which are worth analyzing.
The red marker is the portfolio with the lowest amount of risk. This portfolio divides the efficient frontier from the sub-optimal portfolios.
We can choose any portfolio within the efficient frontier, according to the MPT, this only depends on the risk we’re willing to take, since higher risk equals higher returns. So, one way we could optimize our portfolio is by defining a level of volatility, and looking for the distributions of weights that would result in the highest expected return.
Risk-free Assets
One thing we should also consider is the existence of risk-free assets. In practice, risk-free assets can be seen as the yield of bonds issued by stable governments. Even though, there is always a risk of a government defaulting, the odds are so low compared to the volatility of other assets, that it’s acceptable to think of them as risk-free. Since you could obtain a rate of return from zero risk, any other risky investment will have to have a higher rate of return in order to induce any investors to hold it. In other words, besides finding an optimal balance of risky assets in our portfolio, we can also allocate part of our capital in a risk-free asset.
Sharpe Ratio
Another way we can measure the performance of our portfolio is by looking at its Sharpe Ratio. The Sharpe Ratio is an indicator that combines the return (Rp) and standard deviation (σp) of our portfolio and the risk-free rate of return (Rf). In short, the Sharpe Ratio indicates how much extra return we get from every unit of risk taken.
With the Sharpe Ratio, we can compare different portfolios taking both risk and return into account. The higher the Sharpe Ratio, the better our portfolio did coverting its risk into excess returns. This means that we can maximize our Sharpe Ratio to find a optimal portfolio.
Capital Market Line
When we consider risk-free investments, our set of possible optimal portfolios change. The way we can visualize this is by first taking the rate of return of the risk-free asset and plotting in the volatility x return graph. Then, we trace a straight line, tangent to the efficient frontier and starting at the risk-free rate. This line is called the Capital Market Line (CML).
def sharpe_ratio(portfolio_return, portfolio_volatility, rf=0.042):
"""
Calculates the Sharpe Ratio by taking the return and volatility of
the portfolio and a risk-free rate of return.
"""
return (portfolio_return - rf) / portfolio_volatilitymax_sharpe = 0
max_efficient_portfolio = []
# Iterate through all the portfolios, looking for the one with the highest Sharpe Ratio
for risk, rp in portfolios:
portfolio_sharpe = sharpe_ratio(rp, risk)
if portfolio_sharpe > max_sharpe:
max_sharpe = portfolio_sharpe
max_efficient_portfolio = [risk, rp]# Create a plot with the efficient frontier + CML
plt.figure(figsize=(12, 7))
# Plot the efficient frontier from last example
plt.plot(efficient_portfolios[:, 0], efficient_portfolios[:, 1], c='red', label='Efficient Frontier')
# Plot a straight line between the risk-free rate and the optimal risky portfolio
plt.plot([0, max_efficient_portfolio[0]], [0.042, max_efficient_portfolio[1]], label='Capital Market Line')
plt.scatter(max_efficient_portfolio[0], max_efficient_portfolio[1], c='red', s=50, label='Tangency Portfolio')
plt.scatter(0, 0.042, c='black', s=50, label='Risk-free Rate of Return')
plt.title('Capital Market Line')
plt.xlabel('Annualized Volatility (Risk)')
plt.ylabel('Annualized Return')
plt.xlim(0, 0.4)
plt.legend()
plt.grid(True)
plt.show()The point where the CML and the Efficient Frontier meet represents the portfolio with the highest Sharpe Ratio, which means is also the most efficient portfolio of risky assets. This portfolio is also called the tangency portfolio. The starting point of the CML represents a portfolio only invested in a risk-free asset. Every portfolio that falls on top of the CML is a mix of the most efficient portfolio and the risk-free asset. For that reason, the efficient frontier doesn’t correspond to the set of optimal portfolios anymore, since there are options of portfolios with higher returns and lower risk when we combine risky and risk-free assets. Now, the set of optimal portfolios can be seen as the portfolios in the CML.
Get Andre Nagano’s stories in your inbox
Join Medium for free to get updates from this writer.
The formula for the Capital Market Line would be:
The term (E[Rt] — Rf)/σt is simply the Sharpe Ratio of the tangency or optimal risky portfolio, which is the slope of the CML. The formula says that the expected return of any portfolio on the line is the risk-free rate plus the risk of that portfolio multiplied by the sharpe ratio of the tangency portfolio, or “price of risk”.
Optimal Portfolio with Python
With all the basic concepts in mind, we can write a code to find an optimal portfolio in python. The code will look for a distribution of weights for a set of stocks, that will maximize the Sharpe Ratio of the portfolio.
Daily Returns
We will first retrieve the daily closing prices of the S&P500 companies using the yfinance library. We’ll get a list of the companies from the S&P500 wikipedia page.
import pandas as pd
import requests
# Scrape from the wikipedia page the list of S&P500 companies
url = 'https://en.wikipedia.org/wiki/List_of_S%26P_500_companies'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
}
r = requests.get(url, headers=headers)
table = pd.read_html(r.text)[0]
# Get just the tickers
tickers = table['Symbol'].tolist()# Retrieve the closing prices with yfinance
prices = yf.download(tickers, start='2020-01-01', auto_adjust=False)['Adj Close']
prices = prices.dropna(axis=1)
returns = np.log(prices / prices.shift())In our experiment, we will restrict our set to stocks from the S&P500 companies since 2020. In reality this is not a good practice. It’s always good to diversify our assets, with mid or small caps and even global stocks. Also, it’s recommended to use a bigger timeframe. There are techniques to deal with the impact of the pandemic as well.
But, it would take longer to run the tests. Since this is only for educational purposes, we don’t have to go that far.
Negative Sharpe Ratio
To optimize our portfolio, we have to maximize our Sharpe Ratio. However, since we’re using the minimize method from scipy.optimize library, we’ll define a function to calculate the negative Sharpe Ratio. So, instead of maximizing the Sharpe Ratio, we’ll minimize the negative Sharpe Ratio. This is called the loss function or cost function, which we want to minimize in order to get a better performance.
The loss function takes a array of weights (portfolio), the daily returns of stocks and the risk-free rate as inputs. First, it calculates the risk and volatility, given the weights, and then returns the negative Sharpe Ratio.
def neg_portfolio_sharpe(weights, returns, rf=0.0):
"""
Calculates the negative Sharpe Ratio given the weight distribution,
daily returns of each stock and a risk-free rate of return.
"""
p_returns = portfolio_return(returns, weights)
p_risk = portfolio_risk(returns, weights)
sharpe = (p_returns - rf) / p_risk
return -sharpeConstraints and Bounds
Next, we set some constraints and bounds for the optimization. There are multiple types of limitations we can add to our portfolio. In this example, we’ll set two different limitations.
The first is a constraint that the sum of all weights is equal to 1. That means that our capital will be 100% allocated.
The second is a boundary that for each weight the minimum value is 0 and the maximum is 0.5. This is because we won’t allow short positions, so we won’t have negative weights. Also, we don’t want a single asset to have a weight greater than 50%, since we want a diversified portfolio.
# This constraint means that the sum of all weights minus 1 must be equal to zero
constraints = {'type': 'eq', 'fun': lambda weights: np.sum(weights) - 1}
# This creates an array of (0, 0.5) for each stock
bounds = [(0, 0.5) for _ in range(returns.shape[1])]Initial Weights
For every optimization problem, we have to start at an initial guess of what the weights would be. The optimization algorithm will work from this starting point and look for changes in the weights that would result higher Sharpe Ratio (or lower negative Sharpe Ratio). For our initial weights, we’ll just have a uniform distribution among all the stocks. A more robust approach would be to create a set of random initial weights, and use those as starting points, reducing the bias and the likelihood falling on a local minimum. However, since this is a first experiment, we’ll stick to a more basic technique.
initial_weights = np.array([1 / returns.shape[1]] * returns.shape[1])Minimize
The optimization is done using the optimize.minimize function from the scipy library. We’ll input the loss function, initial weights, extra arguments for the loss function (returns and risk-free rate), which type of solver, constraints and bounds.
The method SLSQP is an iterative algorithm that is used for solving nonlinear optimization problems with constraints. It is widely used for its efficiency and robustness, being able to solve nonlinear problems with both equality and inequality constraints. Although, it can be quite sensitive to the initial guess. That’s why it’s recommended to test different starting points.
from scipy.optimize import minimize
optimized_results = minimize(neg_portfolio_sharpe, initial_weights, args=(returns, 0.02),
method='SLSQP', constraints=constraints, bounds=bounds)# The weights of the minimize results is the x attribute
optimized_weights = optimized_results.xFrom the results of the minimize function, we can get the parameter x, which refer to the weights that leads to the lowest negative sharpe ratio found. Some of the weights from the optimized results are very small compared, less than 0.1%, so we’ll discard them.
For our optimal portfolio, we can calculate the expected annualized return, volatility and Sharpe Ratio.
optimized_return = portfolio_return(returns, optimized_weights)
optimized_volatility = portfolio_risk(returns, optimized_weights)
optimized_sharpe = sharpe_ratio(optimized_return, optimized_volatility)
print('------ Optimal Portfolio ------')
print(f"Annualized Returns: {optimized_return:.2f}")
print(f"Annualized Volatility: {optimized_volatility:.2f}")
print(f"Annualized Sharpe Ratio: {optimized_sharpe:.2f}")
print(f'Number of Stocks in Portfolio: {len(optimized_results.x[optimized_results.x > 0.0001])}')A annualized return of 33% is exceptionally high, way higher than the average for a S&P500 index. However, a higher return is accompanied by a higher risk. Even though an annualized volatility of 20% is a moderately high risk, it’s expected from a portfolio composed only of stocks.
To put it in perspective, in any given year, there’s a roughly 68% chance that the portfolio’s return will be within one standard deviation of the average. This means you can expect the annual return to fall somewhere between 12% (33% — 21%) and 54% (33% + 21%).
A Sharpe Ratio of 1.39 indicates that the investment is generating more return than its associated risk, since it’s higher than 1.0. The portfolio is generating a high amount of return for every unit of risk it takes on, proving its efficience.
The plot above shows the composition of our portfolio. The portfolio is mainly distributed among nine different stocks, with a relatively small percentage allocated to other six. Given the set of almost 500 companies, this is not necessarily a diversified portfolio. In our portfolio optimization solution, this is the weight distribution that led to the highest Sharpe Ratio. However, there could be other solutions. Depending on the minimizing method and initial guess for the weights, another local minimum or the global minimum could be found, resulting in a different distribution of stocks.
(I can make another post explaining how I made this graph, but I basically followed this post on Python Graph Gallery)
Next Steps
As discussed earlier, this is an optimal portfolio with only risky assets, under the set constraints. We could also reduce the overall percentage of stocks and add risk-free assets to reduce the volatility, but also reducing the expected return.
There are many ways we could make this optimization more robust, increase the timeframe of historical data, add stocks with different backgrounds, test different starting weights and minimization algorithms. I’m planning to write about a genetic algorithm for the portfolio optimization in the future.
In the next story, I’ll use the Monte Carlo method to stress-test the performance of this portfolio.
Bibliography
For this article, these contents were used as references:







