I believe you can qualify the welder for partial joint penetration groove welds by having them weld a partial joint penetration groove weld, performing a couple of macroetch samples to verify the weld size is obtained and then machine the samples to 3/8 inch thick and perform a "root bend" and a "face bend".
The test would only qualify the welder for the specific groove geometry tested. That is, he would not be qualified for fillet welds or other groove welds having different groove angles or CJP using back gouging or backing.
The basis of the qualification testing would be per AWS B2.1 as a "mock-up" or production weld. It would only be acceptable if the company is an OEM or if the EOR approves the testing regiment.
Check the specifics of B2.1. I don't have the standard with me while I travel, but I recollect that B2.1 allows for such testing. My advice is worth just what you pay for it, nothing until you check out the details and verify they apply to your situation.
In my opinion, you would be ahead of the game if you simply tested the welders using a groove weld with backing and any controversy would evaporate as the early morning mist on a sunny day.
Best regards - Al