It depends on how you fit your combined models
If you do sequential fitting ie. fit the ARFIMA(3,3) model first and then feed the residuals through a GARCH model, then all ARFIMA(3,3)-GARCH(P,Q) models will have redundant ar1 and ar3 variables. However, if you do joint estimation of ARFIMA(p,q)-GARCH(P.Q) you might end up with a model combination where ar1 and ar3 becomes statistically significant, depending on the fitted GARCH model.
In the end, it all boils down to the purpose of the chosen model: Will it act as an explanatory model or a predictive model?
If the goal of the model is to forecast the time-series, then statistical tests of the model variables aren't your main concern. Instead, you should be validating the model performance via out-of-sample test procedures.
If the goal of the model is to explain which variables contribute to the patterns in your time-series, then there is no real need of removing non-significant variables in your time-series. Presumably, you included the variables in your model because you thought that they might play a role in capturing some of the patterns in your time-series.
That the variables failed to reject the null (aka. became redundant) does not imply that the model will perform poorly, it just means that your sample did not detect an effect of the ar1 and ar3 variables. However, if there is a foundational basis to include the extra ar-terms (either from expert opinions or from past experience), then the variables might become statistically significant in the future and hence, they shouldn't be removed from the explanatory model.