I've come across the notes of the 2003 lecture "Advanced Lecture on Mathematical Science and Information Science I: Optimization in Finance" by Reha H. Tutuncu.
It describes on page 62 in section 5.2 a way to reformulate the tangency portfolio to the efficient frontier as a quadratic optimization problem:
$$ \min_{y,\kappa} y^T Q y \qquad \text{where} \quad (\mu-r_f)^T y = 1,\; \kappa > 0 $$
I'm wondering if anyone has seen an adaptation or similar work to incorporate a factor model. I believe an adaptation for the $y$ vector will need to take place but I'm unsure what that would be.