Almost always, the market convention is to use for yield the same frequency as the coupon payment frequency.
However in a few markets, the market convention is to convert this yield to the frequency of the local government bond. For example, if the local government bonds usually pay annually, as they do in Eurozone, and some corporate bond pays quarterly or semi-annually, then you annualize the latter bond's yield, so it is easier to compare with the rest of the universe. Conversely, U.S. treasury debt (notes and bonds) and U.K. gilts pay semi-annual coupons, so the yield of GBP bonds having other frequencies are often quoted as semi-annual, so that, e.g., spread over benchmark is more meaningful.
If you have access to Bloomberg terminal, look for the field 'conventional yield frequency', which contains this frequency (periodicity) conventionally used to quote this bond's yield.
However I have never seen any bond for which the convention would be convert the yield to continuous compounding.
I suggest you read this paper on gilts to get a good feeling for price-yield conventions.
Edit: Also in a few markets, the market conventions even for a yield of bonds that don't pay coupons is to use compounding anyway. For example, LTNs in Brazil have no coupons, have had maturities up to 5 years, their yield are conventionally quoted using annual compounding. There are non-coupon-paying bonds in Eurozone (example) whose yield is likewise quoted using annual compounding.
If $f_1$ and $f_2$ are frequencies (1 - annual, 2 - semiannual, 4 - quarterly, 12 - monthly...), and $y_{f_1}$ and $y_{f_2}$ denote the yields corresponding to these frequencies, then ${\left(1+\frac{y_{f_1}}{f_1}\right)}^{f_1}={\left(1+\frac{y_{f_2}}{f_2}\right)}^{f_2}$, so I think $y_{f_2} = \left(\left(1+\frac{y_{f_1}}{f_1} \right)^{\frac{f_1}{f_2}} -1 \right)\times {f_2} $ (check my algebra before using) - higher frequency quotes lower yield.