Random variable minus Integral of Ito Generator is a Martingale under what conditions?

Question

I am reading about american option pricing and the variational inequality, and the book I am reading states, in the derivation of the variational inequality, the following is a martingale: $$M_s = U(s,X_s) - \int_t^s(\partial_tU(x,X_r) + \mathcal{L}U(x,X_r))dr$$ where $\mathcal{L}$ is the Ito/infinitesimal generator of the form:$$\sum_{i} b_{i} (t,x) \partial_i + \frac1{2} \sum_{i, j} \big( \sigma (t,x) \sigma (t,x)^{\top} \big)_{i, j} \partial_{i,j}$$

This is given without any context, but it's definitely using the usual conditions, such as a Martingale with respect to the natural filtration generated by the Brownian Motion under the risk neutral measure, etc... But I think there are some sort of constraints, and that is may not be true for any given stochastic process.

I was trying to find out an equality for the expression, and saw that it was very similar to Ito's lemma applied to a function $U$ and Ito process $X$.

So I wrote: $$dU(t,X_t) = [\partial_tU(t,X_t) + \mathcal{L}U(t,X_t)]dt + \nabla U (t,X_t) \sigma (t,X_t)dW$$ and integrating from $t$ to $s$, this gives: $$U(s,X_s) - U(t,X_t) = \int_t^s(\partial_tU(x,X_r) + \mathcal{L}U(x,X_r))dr + \\ \int_t^s\nabla U(r,X_r) \sigma (r,X_r)dW(r)$$

So the expression from the book evaluates to $$M_s = U(t,X_t) + \int_t^s\nabla U(r,X_r) \sigma (r,X_r)dW(r)$$

So the second term looks like the difference between two martingales with respect to the same filtration, but I am unsure about $U(t,X_t)$ and the overall expression being a martingale.

It may also be that the martingale is only defined on the filtration $(\mathcal{F}_s)_{t \leq s}$, since this would make the expression a martingale, but I am unsure if that is a valid rationale. I'm only a beginner at this stuff and have only seen local martingales and martingales with respect to filtrations beginning at $0$.

Thanks for the help!

EDIT: It's looking like there are stochastic processes $X_s$ in the book that are defined only beginning at some $s \geq t$ and are said to be true $\mathbb{Q}$ martingales with respect to the Filtration $(\mathcal{F}_s)_{s \geq 0}$. So now I am thinking the process $M_s$ is defined only for $s \geq t$, but I have tried searching up this kind of situation, but haven't found anything helpful.

I think for the stochastic process $(X_s)_{s \geq t}$, for $s < t$, the value would be 'undefined', so we couldn't really apply the martingale definition to it, until it was at time $s \geq t$, at which point it would be a martingale with respect to $(\mathcal{F}_s)_{s \geq t}$, so if that definition is valid, the above should all make sense. Hoping someone can confirm!