# Efficient frontier doesn't look good

Hi I'm trying to draw an efficient frontier. Below is what I used. returns parameter consists of 9 column returns of portfolio. I selected 10,000 portfolios and this is how my efficient frontier looked like. This is not the usual frontier shape that is familiar to us.

Data set is 48_Industry_Portfolios_daily.csv, obtained from (http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html). The first 9 columns were selected

Can somone kindly explain me the issue.

def portfolio_annualised_performance(weights, mean_returns, cov_matrix):
returns = np.sum(mean_returns*weights ) *252
#print ('weights shape',weights.shape)
#print (' Returns ',returns)
std = np.sqrt(np.dot(weights.T, np.dot(cov_matrix, weights))) * np.sqrt(252)
#print ('Std ',std)
return std, returns

def random_portfolios(num_portfolios, mean_returns, cov_matrix, risk_free_rate):
results = np.zeros((3,num_portfolios))
weights_record = []
for i in range(num_portfolios):
weights = np.random.random(48)

weights /= np.sum(weights)
weights_record.append(weights)
portfolio_std_dev, portfolio_return = portfolio_annualised_performance(weights, mean_returns, cov_matrix)
results[0,i] = portfolio_std_dev
results[1,i] = portfolio_return
results[2,i] = (portfolio_return - risk_free_rate) / portfolio_std_dev
return results, weights_record

def monteCarlo_Simulation(returns):

#returns=returns.drop("Date")
returns=returns/100
stocks=list(returns)
stocks1=list(returns)
stocks1.insert(0,"ret")
stocks1.insert(1,"stdev")
stocks1.insert(2,"sharpe")
print (stocks)
#calculate mean daily return and covariance of daily returns
mean_daily_returns = returns.mean()
#print (mean_daily_returns)
cov_matrix = returns.cov()

#set number of runs of random portfolio weights
num_portfolios = 10000

#set up array to hold results
#We have increased the size of the array to hold the weight values for each stock
results = np.zeros((4+len(stocks)-1,num_portfolios))

for i in range(num_portfolios):
#select random weights for portfolio holdings
weights = np.array(np.random.random(len(stocks)))
#rebalance weights to sum to 1
weights /= np.sum(weights)

#calculate portfolio return and volatility
portfolio_return = np.sum(mean_daily_returns * weights) * 252
portfolio_std_dev = np.sqrt(np.dot(weights.T,np.dot(cov_matrix, weights))) * np.sqrt(252)

#store results in results array
results[0,i] = portfolio_return
results[1,i] = portfolio_std_dev
#store Sharpe Ratio (return / volatility) - risk free rate element excluded for simplicity
results[2,i] = results[0,i] / results[1,i]
#iterate through the weight vector and add data to results array
for j in range(len(weights)):
results[j+3,i] = weights[j]

print (results.T.shape)
#convert results array to Pandas DataFrame
results_frame = pd.DataFrame(results.T,columns=stocks1)

#locate position of portfolio with highest Sharpe Ratio
max_sharpe_port = results_frame.iloc[results_frame['sharpe'].idxmax()]
#locate positon of portfolio with minimum standard deviation
min_vol_port = results_frame.iloc[results_frame['stdev'].idxmin()]

#create scatter plot coloured by Sharpe Ratio
plt.figure(figsize=(10,10))
plt.scatter(results_frame.stdev,results_frame.ret,c=results_frame.sharpe,cmap='RdYlBu')
plt.xlabel('Volatility')
plt.ylabel('Returns')
plt.colorbar()
#plot red star to highlight position of portfolio with highest Sharpe Ratio
plt.scatter(max_sharpe_port[1],max_sharpe_port[0],marker=(2,1,0),color='r',s=1000)
#plot green star to highlight position of minimum variance portfolio
plt.scatter(min_vol_port[1],min_vol_port[0],marker=(2,1,0),color='g',s=1000)

print(max_sharpe_port)


Also I'm asked to compare portfolio variance using different regularizes and to use a validation methods to find the optimal parameters. Can we use python to do this?

• It would really help if others have your returns parameter so that they can run the code and debug. – Bob Jansen Feb 22 at 8:40
• Is there a way that I can attach my excel? – Hiru Feb 22 at 8:55
• You can upload a csv somewhere and link to it. – Bob Jansen Feb 22 at 9:56
• What did you do about pre-processing the lots of erroneous/missing values in the "soda" column? The comments below about taking more samples I do not consider correct, you should already see a reasonable shape with not that many samples due to their distribution. But if your input data is erroneous, then Im not sure what you will get... – Attack68 Feb 28 at 6:43
• Hi.... I selected data from 2005 onwards. So there are no missing values from 2005 onwards – Hiru Feb 28 at 7:19

As i understand your question you are confused as to why the expected parabola-shape of the frontier is not depicted clearly.

If you want to see the shape more clearly you can do one of two things:

1. Increase the number of random portfolios. As this numbers goes to infinity you will eventually plot all possible portfolio combinations, and your efficient frontier will be very visible.

2. Use the fact that all portfolios on the efficient frontier can be constructed by combinations of just two efficient portfolios (e.g. the max sharpe and the min. variance portfolios). You would do this by just constructing an array of portfolios ((return, std. dev)-pairs) with weights equal to $$weights=w*\pi_{max}+(1-w)*\pi_{min}$$ for an interval of $$w$$'s.

Instead of making 10,000 random portfolios to find the tangency and min.var. portfolios, you could also just solve for them using the equations

$$\mathbf{\pi}_{\max SR} = \frac{1}{\mathbf{1'\Sigma^{-1}\mu}} \mathbf{\Sigma^{-1}\mu}$$

$$\mathbf{\pi_{\min Var}} = \frac{1}{ \mathbf{1' \Sigma^{-1}1}} \mathbf{ \Sigma^{-1}1}$$

Where $$\mathbf{\Sigma^{-1}}$$ is the inverse of your variance-covariance matrix, $$\mathbf{\mu}$$ is your vector of expected returns, and $$\mathbf{1}$$ is a vector of 1's with the same length as your $$\mathbf{\mu}$$.

• Also if you really want to bring out the curve of the frontier visually, you could change the aspect ratio of the graph to make it wider. – jthg Feb 22 at 16:10
• Hi JTHG. Thanks alot for the answer. Well I went through the first step. still the frontier doesn't look like a parabola. I took 500,000 portfolios. I updated it in my answer – Hiru Feb 24 at 2:09
• You should try doing step 2, and overlaying that on the plot to see how it should look. I think the shape is somewhat unclear, but it is definently there in your updated answer. – jthg Feb 24 at 11:58
• @Hiru, if you're trying to produce something that looks 'perfectly efficient', you should simulate your portfolios using the same params (ie, trade-off between risk return). the efficient frontier is simply a theoretical framework and as such real portfolios don't always fall perfectly within the frontier. – Chris Feb 26 at 23:24
• @JTHG can u kindly provide me a resource where I can read more about the second option – Hiru Mar 1 at 23:35

Whilst this doesn't answer your question you may be interested to know that you can vectorise your simulation and greatly improve the efficiency of your code like this:

def portfolio_annualised_performance(weights, mean_returns, cov_matrix):
"""
Return annualised risk and return for a portfolio given asset weights and expected return and covriance

Args:
weights (ndarray): asset weights
mean_returns (ndarray): expected returns of each asset
cov_matrix (ndarray): covariance metrix of asset returns

Returns:
tuple: annualised risk (np.float64), annualised total return (np.float64)

Notes:
If weights is supplied as a 2D array, then outputs are not floats but 1D arrays.
"""
return (np.sqrt(np.einsum('...i,ij,...j->...', weights, cov_matrix, weights) * 252),
np.einsum('...i,i->...', weights, mean_returns) * 252)

def random_portfolios(size, mean_returns, cov_matrix, risk_free_rate):
"""
Simulates a number of portfolio through random weights and calculates performance metrics

Args:
size (int): number of simulations
mean_returns (ndarray): expected returns of each asset
cov_matrix (ndarray): covariance metrix of asset returns
risk_free_rate (np.float64): annualised risk free return

Returns:
tuple: annualised risk (ndarray), annualised total return (ndarray), Sharpe ratio (ndarray), weights (ndarray)
"""
rand_weights = np.random.random(size=(size*mean_returns.shape[0])).reshape(size, -1)
rand_weights /= np.sum(rand_weights, axis=1)[:, np.newaxis]
perf = portfolio_annualised_performance(rand_weights, mean_returns, cov_matrix)
return perf + ((perf[1] - risk_free_rate)/perf[0], rand_weights)


This has occurred to me as well when trying to simulate the efficient frontier. The conclusion I came up with was the as I increase the number of instruments I try to use, the frontier losses its shape. If you want to check your code correctness, simply leave 2-3 instruments, see the results that come up, I'm quite sure you'll see the usual shape come up.

• Hi Thanks for answering. What did you mean by instruments? is it the number of portfolios? – Hiru Mar 4 at 9:40
• Number of stocks/sectors/bonds you are using to get the efficient frontier. I think you're using 9 of them. – user1919071 Mar 4 at 9:48
• A wild guess - the Optimal portfolio usually outputs very concentrated results (gives weights <> 0 to only a few sectors), when you use a lot of sectors the random weights just don't assign this amount of 0's so the shape looks weird - but the code is totally correct, it's just a simulation. – user1919071 Mar 4 at 9:54
• ok... But the issue is in my real question I need to use, 48 industries. And find the optimal parameters. quant.stackexchange.com/questions/44405/… . I used regularizers to the equation. This is a question I posted. Do u have any idea about the steps I should try? – Hiru Mar 4 at 9:57