Estimating conditional expectation using monte carlo and least squares regression

Question

I'm looking to understand the problem of least squares monte carlo that is used in valuation of bermudan options, but from a simpler context.

Say I have random variables $X$ and $Y$ which are uniform [0,1] and independent. Define $Z=X^2+Y^2+XY$. Let us say I want to evaluate the expectation $E(X|Z=a)$ using Monte Carlo. Can least squares MC help in this case? If so, can anyone outline the process?

I'm trying to understand the core of the algorithm without the tedious notation one has to go through while reading papers and other articles explaining MC least squares.

Answer 1

The link in your comments mention section 11.6 of Numerical Methods in Economics by Kenneth Judd. I recommend giving that a read as well. It's only a few pages. Below some code that implements least squares Monte Carlo for the problem you gave:

set.seed(42)
fun <- function(x, y) x^2 + y^2 + x * y
N <- 1e3L
X <- runif(N)
Y <- runif(N)
Z <- fun(X, Y)
plot(Z, X)

# We can create a function of Z that gives an estimate of X:
model <- lm(X ~ Z + sqrt(Z))
print(model)

# Call:
# lm(formula = X ~ Z + sqrt(Z))
#
# Coefficients:
# (Intercept)            Z      sqrt(Z)  
#     0.02860      0.07183      0.44900  

a <- seq(0, 3, by = 0.01)
x_hat <- 
  model$coefficients[[1L]] + 
    a * model$coefficients[[2L]] + sqrt(a) * model$coefficients[[3L]]
lines(a, x_hat, col = 'blue')

Answer 2

I am not sure how to outline that process without going through some notation. The definition of the conditional expectation $f(a)=E[X|Z=a]$ is that the function $f$ is such that the squared norm $E[(X-f(Z))^2]$ is minimized.

Least squares regression looks only for affine linear functions $f(a)=\beta\,a+\varepsilon$ (where $\beta,\varepsilon$ are the constants that are to be found). In your example $X$ is not a linear function of $Z$ (take a small $Y$ so see that it is more like $X\sim\sqrt{Z}$). Therefore I find it unlikely that an affine linear $f$ is a good candidate for $E[X|Z=a]\,.$ It is probably better to try Polynomial regression instead.

Python should have all the packages to simulate $X,Y,Z$ and try all sorts of regressions and compare.