I think 5.16 applies, but you can only put in as much radius as you can fit. There are practical limitations that outweigh code provisions. So if code requires 2" diameter, and all you have room for is 3/8", then you do the 3/8" (or, I suppose, someone might want you to drill a larger hole at the end there, but let's not go there yet), you don't just say, "Oh, gosh, no room for 2", so all bets are off and I can have a miter finish to a perfect 90-degree cut." (And, of course, by "you" I don't mean you.)
Another provision that isn't exactly relevant but can be used for guidance is 2.3.4.2. It's talking about slots for fillet welds in slots, not quite the same as your application, but it's still a point of reference. 2.3.4.2 says the slot has to have a radius at least the thickness of the plate in which it's made. (Again, if your HSS wall thickness is less than twice the gusset thickness, you do what you can.) 2.3.5.2 says the same thing about slots made for slot welds. I think that's enough to take this as general slot practice, no?
Hg