@chrisaycock already gave you a correct answer, but I thought I would add a more verbose version (and practice some MathJax by the way).
In fact when I began answering I thought it was going to be a straightforward answer, but having spent some more time with this question I see there are some potential traps you can fall in.
Especially since some of the steps you name are not 100% clear, I assumed the worst-case scenario (AKA everything wrong). I suppose some of them are just shorthand notions. Sorry if you already do it the right way and it's obvious it's wrong making my explanations ridiculous, but at least one of the steps is to blame as you are getting different results.
So, going through your task:
Say I'm given a set of monthly returns over 10 years on a monthly basis.
Let's call them
$$
r_{1_{jan}}, \ ...,\ r_{1_{dec}}, \ ...,\ r_{10_{jan}}, \ ...,\ r_{10_{dec}} \ [eq. 1]
$$
What you do is:
I found the cumulative returns of each year
Your cumulative return for a year is a product of monthly returns:
$$
R_{i} = (1+r_{i_{jan}}) * \ ... \ * (1+r_{i_{dec}}) - 1 \ [eq. 2]
$$
OK, straightforward. Not that many options here.
then found the geometric mean of the 10 years
if you mean that literally (I warned you I would take the worst approach possible, sorry), as in found the geometric mean of those 10 returns:
$$
R_{G} = \sqrt[10]{R_{1} * R_{2} * \ ... \ * R_{10}} \ [eq. 3]
$$
we have our first problem. While technically you can calculate anything (as long as it's not negative), it doesn't make sense. We are looking for a geometric average rate of return instead:
$$
R_{G} = \sqrt[10]{(1 + R_{1}) * (1 + R_{2}) * \ ... \ * (1 + R_{10})} - 1 \ [eq. 4]
$$
OK, done, should be the correct answer.
Your classmate's version:
He found the cumulative returns of the entire time period,
He calculated it either this way:
$$
AR = (1+r_{1_{jan}}) * \ ... \ * (1+r_{1_{dec}}) * \ ... \ * (1+r_{10_{jan}}) * \ ... \ * (1+r_{10_{dec}}) - 1 \ [eq. 5]
$$
or just used $\frac{P_{last}}{P_{first}} - 1$ which is the same. No problem here.
then took the (months in a year / total months) root of that data.
First assumption - I suppose you meant power here (or total months / months in a year
root), because otherwise it wouldn't make much sense.
Now, if we literally take the root out of our accumulated returns ($AR$):
$$
\sqrt[\frac{120}{12}]{AR} = \sqrt[10]{(1+r_{1_{jan}}) * \ ... \ * (1+r_{1_{dec}}) * \ ... \ * (1+r_{10_{jan}}) * \ ... \ * (1+r_{10_{dec}}) - 1} \ [eq. 6]
$$
using $[eq. 2]$ we get:
$$
= \sqrt[10]{(1+R_{1})*(1+R_{2})* \ ... \ * (1+R_{10}) - 1}
$$
Oops, seems similar to $[eq. 4]$, but it's not the same. We did something wrong.
In fact we wanted it this way (remembering that we're looking for annual returns):
$$
R_{G} = \sqrt[10]{1 + AR} - 1 \ [eq. 7]
$$
Now plugging $[eq. 5]$ and $[eq. 2]$:
$$
= \sqrt[10]{1 + (1+R_{1})*(1+R_{2})* \ ... \ * (1+R_{10}) - 1} -1
$$
$$
= \sqrt[10]{(1+R_{1})*(1+R_{2})* \ ... \ * (1+R_{10})} - 1
$$
and this is the same as $[eq. 4]$
This way you see that both methods should give equivalent results. If not, then either it's a calculation mistake/rounding issue or you're using different methods and someone is not calculating an actual geometric average rate of return.
I hope now you can find where the issue was.