IMHO, this is a nonconforming item if indeed the job specs or applicable building codes required the filler metal comply with a CVN toughness value.
Compliance with CVN is now a big issue with some building departments, especially on the west coast. Many building codes have adopted FEMA requirements for steel frame buildings constructed in regions of moderate to high seismic risk.
Your next step will depend on several factors, one being the attitude of the building department to quality assurance. Some will go along with what the EOR says while others are more vigilant when it comes to quality assurance.
Just make sure you document everything.