Yes this kind of steel can get martensitic if cooling rates are high (8/5 cooling rate)
I would not agree that it cracks during welding, but rather after welding. Cracking during welding is generally caused by a poor width to depth ratio.
Cracking can happen up to 24-48 hours after welding, which is why a bakeout or go straight in to PWHT is a good idea.
Filler material storage and handeling can also be related to cracking.
NDT must be performed after the PWHT.
Grade P5 is not rocket science but it must be treated with some respect, also because it is usually "nasty" operating temperatures, which is the reason for using P5.
Good luck with your new job?
3.2