I assume you want to price a derivative product that pays $\int_0^T\ln S_tdt$ at maturity time $T$, from time $t=0$. I'll ignore generalization to time $t$ because it is trivial (split the integral in two, before and after $t$ as you did).
The first trick it to do an integration by part on $\ln S_t dt$:
$d(t\ln S_t) = \ln S_t dt + \frac{t}{S_t}dS_t + \frac{t}{2S_t^2} \sigma_t^2 S_t^2 dt$
Now let's integrate both sides between $0$ and $T$ and rearrange:
$\int_0^T \ln S_t dt = \int_0^T d(t\ln S_t) - \int_0^T \frac{t}{S_t}dS_t - \int_0^T \frac{t}{2} \sigma_t^2 dt = T \ln S_T - \int_0^T \frac{t}{S_t}dS_t - \int_0^T \frac{t}{2} \sigma_t^2 dt$
So your payoff $\int_0^T \ln S_t dt$ equals three terms:
- $T \ln S_T$: $T$-times the log-contract, which is statically replicable.
- $\int_0^T -\frac{t}{S_t}dS_t$: a dynamic delta-hedging strategy (to short a quantity $\frac{t}{S_t}$ of stock at any time $t$).
- $\int_0^T -\frac{t}{2} \sigma_t^2 dt$ a third dynamic term that is a function of $S_t$'s total variance.
We price the derivative that pays $\int_0^T \ln S_t dt$ at time $T$, so you know by heart that its price is the discounted expectation under the risk-neutral measure:
$Price_0 = e^{-rT}\mathbb{E}^\mathbb{Q}( \int_0^T \ln S_t dt ) = e^{-rT}\mathbb{E}^\mathbb{Q}( T \ln S_T ) - e^{-rT}\mathbb{E}^\mathbb{Q}( \int_0^T \frac{t}{S_t}dS_t ) - e^{-rT}\mathbb{E}^\mathbb{Q}( \int_0^T \frac{t}{2} \sigma_t^2 dt )$
The two first terms are straightforward.
- $e^{-rT}\mathbb{E}^\mathbb{Q}( T \ln S_T )$ is the price off $T$ times log-contract.
- $e^{-rT}\mathbb{E}^\mathbb{Q}( \int_0^T -\frac{t}{S_t}dS_t )$ is the present value of the expected financing cost of the dynamic delta-hedging. Note that here, delta is negative (you short the stock) so financing is a benefit: you are paid interest to be short the stock.
The tricky part here is the third term: how are you going to hedge this integral of $\sigma_t^2$?
- In basic Black-Scholes framework, you have $\sigma_t = \sigma$, so
$\int_0^T \frac{-t}{2} \sigma_t^2 dt = -\frac{\sigma^2}{2} \int_0^T tdt = -\frac{T^2\sigma^2}{4}$
Nothing to hedge dynamically; it just makes the derivative cheaper by $e^{-rT}\frac{T^2\sigma^2}{4}$ (it is just some deterministic cash amount you remove from the rest of the replication strategy).
If $\sigma_t = \sigma (t,S_t)$, i.e. $\sigma$ is a local volatility function, it becomes tricky. You would need to dynamically hedge that variance integral $\int_0^T -\frac{t}{2} \sigma_t^2 dt$, because it has some delta risk as well (it depends on $S_t$!). Maths become nasty because you have some recursivity there (hedging the variance integral will require a dynamic hedging strategy, likely to involve a variance integral as well, that would need to be delta-hedged..). In practice, all it means is that you would have to adjust your delta hedging $\int_0^T \frac{-t}{S_t}dS_t$ to account for the fact that volatility is not constant with time and spot! (different model, different delta)
If volatility is stochastic and has its own source of risk, then simple delta-hedging is not enough. You need something else to hedge that variance risk (the variance integral). Note that the integral ($\int_0^T \frac{-t}{2} \sigma_t^2 dt$) is similar to the payoff of a variance swap ($\int_0^T \sigma_t^2 dt$), but not exactly, as it has some time weighting. So there is probably some dynamic hedging through options to be done there.