What determines whether a swap should be discounted against a standard OIS curve VS a 'custom' CSA curve specific to the swap's counterparty? (such custom curves are marked as spreads to some base curve by the bank's trading desk)?
To trade a swap counterparties must have an ISDA Master Agreement drawn up and signed between themselves.
If collateral is to be exchanged that agreement will also contain a section called a CSA: a Credit Support Annex.
That documentation defines the types of collateral available to post as the liability holder: and the banks choice of discount curve will reflect these choices.