In the C++ version of QuantLib it is possible to pass a separate discount curve to the engine, but the functionality is not exported in Python (and therefore, as @Cornholio said, the risk-free curve is also used for discounting). If you need this feature in Python, please open an issue at https://github.com/lballabio/QuantLib-SWIG/issues.
If you have enough forward rates for a given observation date, you should be able to construct a full swap curve for that date. This would involve some curve fitting and some interpolation , so it’s not trivial. However once you’ve done that , you can observe any rate that you like from the curve so you can calculate your roll down.