0
$\begingroup$

In an attempt to construct the efficient frontier curve for a portfolio made up of all the stocks in the s&p index, I have run into some strange error for the calculations of the volatility of this portfolio. Here's the piece of the python code with the error:

tickers = ['A'..........# all the tickers of the s&p listed companies]

adj_close = pd.DataFrame()  # an empty variable that will house

adj_close = web.get_data_yahoo(tickers, start='2017-5-25', end='2017-6-25')['Adj Close']

returns = (adj_close / adj_close.shift(1)) - 1

pf_returns = []
pf_volatility = []

num_assets = len(tickers)

cov_matrix = returns.cov() * 20

for x in range(100):
    weights = np.random.random(num_assets) #random set of floats that matches the number of assets
    weights /= np.sum(weights)             #sum of weights = 1
    pf_returns.append(np.sum(weights * returns.mean() * 20))
    pf_volatility.append(np.dot(weights.T, np.dot(cov_matrix, weights)) * 0.5)

pf_returns = np.array(pf_returns)
pf_volatility = np.array(pf_volatility)

port = pd.DataFrame({'Returns': pf_returns, 'Volatility': pf_volatility})
print(port)

Here's the output:

   Returns  Volatility
0   0.014793         NaN
1   0.013081         NaN
2   0.013250         NaN
3   0.010461         NaN
4   0.010808         NaN
5   0.011683         NaN
6   0.011159         NaN
7   0.012093         NaN
8   0.010168         NaN
9   0.013329         NaN
10  0.012290         NaN
11  0.011616         NaN
12  0.011767         NaN
13  0.011151         NaN
14  0.011238         NaN
15  0.010688         NaN
16  0.011548         NaN
17  0.010132         NaN
18  0.010191         NaN
19  0.012638         NaN
20  0.010190         NaN
21  0.013640         NaN
22  0.014704         NaN
23  0.012479         NaN
24  0.011054         NaN
25  0.011495         NaN
26  0.012228         NaN
27  0.008502         NaN
28  0.012055         NaN
29  0.013000         NaN
..       ...         ...
70  0.010414         NaN
71  0.012023         NaN
72  0.011955         NaN
73  0.012560         NaN
74  0.014966         NaN
75  0.011383         NaN
76  0.012452         NaN
77  0.010191         NaN
78  0.011512         NaN
79  0.014761         NaN
80  0.011228         NaN
81  0.013632         NaN
82  0.011403         NaN
83  0.011184         NaN
84  0.013233         NaN
85  0.015004         NaN
86  0.012597         NaN
87  0.012751         NaN
88  0.012068         NaN
89  0.009325         NaN
90  0.012346         NaN
91  0.010205         NaN
92  0.010199         NaN
93  0.014781         NaN
94  0.011009         NaN
95  0.011710         NaN
96  0.010670         NaN
97  0.013398         NaN
98  0.012641         NaN
99  0.011366         NaN

[100 rows x 2 columns]

How can correct my volatility code?. Thanks in advance.

$\endgroup$
3
  • $\begingroup$ Why don't you investigate the output of the following (that you append to the pf_volatility) in the debug mode: np.dot(weights.T, np.dot(cov_matrix, weights)) * 0.5 $\endgroup$ Jul 26, 2017 at 14:53
  • $\begingroup$ your call to adj_close.shift(1) inserts a NaN into the first entry since there was no data to shift into that index. These NaNs could be falling through your subsequent operations (since essentially every floating point operation on NaN yields a NaN). You can work around this by dropping the first row of returns. $\endgroup$ Jul 26, 2017 at 16:24
  • $\begingroup$ The cov method on returns has never given me any issues. I just tried with a smaller set and it works fine even with the first row has the NaN returns = (adj_close[['FB', 'WMT']] / adj_close[['FB', 'WMT']].shift(1)) - 1 $\endgroup$
    – Mysterio
    Jul 26, 2017 at 17:36

1 Answer 1

1
$\begingroup$

You probably have NaN values in your covariance matrix. NaN is a special floating point value that's often used to denote missing observations.

Example: If you have a dataframe $y$ with two series that don't overlap in time more than 1 period, the covariance between the two series is going to be undefined (which the cov function returns as NaN).

x = pd.DataFrame([[1, np.NaN],[2,3],[np.NaN,4]])
x.cov()

Dataframe x is

     0    1
0  1.0  NaN
1  2.0  3.0
2  NaN  4.0

and x.cov() is

     0    1
0  0.5  NaN
1  NaN  0.5

Notice the NaN value in the covariance matrix. Note that NaN times a number returns NaN and NaN plus a number returns NaN.

$\endgroup$
5
  • $\begingroup$ I will check the output of the cov matrix for missing observations then $\endgroup$
    – Mysterio
    Jul 26, 2017 at 14:50
  • 1
    $\begingroup$ @Mysterio Check your adj_close dataframe for NaNs $\endgroup$ Jul 26, 2017 at 14:54
  • $\begingroup$ @Mathew, some stocks have empty cells after exporting returns to a csv. Could it be the NaN? $\endgroup$
    – Mysterio
    Jul 26, 2017 at 17:46
  • $\begingroup$ @Mysterio Empty cells will get read into pandas as a NaN. NaN is now the conventional number used to denote a missing floating point value because it causes this type of error! NaN makes it harder to do something undefined/wrong. $\endgroup$ Jul 26, 2017 at 17:56
  • $\begingroup$ I thought there was an issue with the code. I now realise it's the dataset itself $\endgroup$
    – Mysterio
    Jul 26, 2017 at 18:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.