Removal by grinding is one way, but proper use of the arc gouge usually eliminates the presence of carbon deposits. 

If not removed, welding over remaining carbon deposits of only 1% can reduce weld metal strength and toughness. 

Karir

Specifically, D 1.1 does not require grinding to "Bright Metal" as D 1.5 does.  I maintain that just because tha D1.1 Code does not specifically require grinding for removal of the carbonized metal dust, the pools of carbon and graphite dust, and little spots of copper flashing, does not justify failing to grind.  Good welding practice is reason enought to grind or at least power brush.  The Codes do not say that you are required to turn on the welding machine, or hold a consistent arc length, or travel at the correct travel angle, or do many other things autonomically during welding, but you do these things anyway.  The Code does not require a person be trained to perform CAC, and I see that most welders cannot make a clean cut with hand held Air Arc torches.  Grinding or at least power brushing are necessary!

During the inspection of hand held back gouging operations, I usually see little flashes of copper remaining in the ripples of the groove, and molten copper can cause microvoids and microcracks.  This is another "Good Practice" reason to grind the arc gouged groove. 

The little small nodules you speak of, are likely as not Cementite Nodules. (C3I)  They are very hard.  These may not re-melt and remix mix in weld solution, and can be considered crack initiators.

In my experience I have run into only 1 (ONE) AISC certificated fabrication shop that does not grind the back gouges made by Air Carbon Arc Gouging. As a third party inspector in their facility, I have an issue with this.

The AWS manual on CAC, paragraph 5.8.2, says;
  "To avoid difficulties with carburized metal, users of the air carbon arc cutting process should be aware of the metallurgical  events that occur during gouging and cutting.  With DCEP and the corresponding half cycle of alternating current, the current flow carries the ionozed carbon atoms from the electrode to the base metal.  The free carbon particles are rapidly absorbed by the melted base metal.  Increased carbon can can lead to increased hardness and possible cracking.  Since this absorption cannot be avoided, it is important that all carburized molten metal be removed from the cut surface, preferrably by the air jet.

When the air carbon arc cutting process is used under improper conditions, the carburized molten metal left on the surface may usually be recognized by its dull gray-black color.   This is in contrast to to the bright blue color of the properly made groove.  Inadequate air flow may leave small pools of carburized metal in the bottom of the groove.  Irregular electrode travel, which is particularly true for manual gouging may produce ripples in the groove wall that tend to trap the carburized metal.  Finally, an improper electrode push angle may cause small beads of carburized metal to remain along the edge of the groove.

The effect of carburized metal on the cut surface during subsequent welding depends on many factors, including the welding process to be employed, the kind of base metal, and the weld quality required. Although it may seem that the filler metal deposited on the surface during welding should assimilate small pools or beads of of carburized metal, experience with steel base metals shows that traces of metal containing approximately 1% carbon may remain along the weld interface.  These imperfections become more significant with demands for increasing weld strength and toughness..

There is no evidence that the copper from copper coated electrodes is transferred to the cut surface in base metal, except when the process is improperly used.  Carburized metal on the cut surface may be removed by grinding, but it is more efficient to conduct air carbon arc gouging and cutting properly within prescribed conditions to completely avoid the retention of undesirable metal."...

So, this paragraph establishes that the carburized byproduct of CAC is detrimental, and that the detrimental by product is usually left by improper hand held gouging.  The real question that remains is; how "Properly" your welder performs the back gouging operations.   (Remember, 1% Carbon is 100 points of Carbon) Grinding is necessary.

Air-carbon arc gouging removes a lot of metal fast. In the hands of an experienced worker, metal can removed as if cut with a scalpel. On the other hand, the unskilled person can leave a trail of destruction.

I always marveled that fabricators wouldn't devise a test requiring a person to demonstrate their skill at using the air carbon arc torch before turning them loose on a valuable fabricated member or component.

AWS C5.3-91 Recommended Practice for Air Carbon Arc Gouging and Cutting essentially says, any carburized metal should be carried away by the jet of air used to blow away the molten metal. Problems caused by improper use of the manual air carbon gouging torch can be removed by grinding.

My experience has been that fabricators will grind the air carbon arc gouged surface when challenged by the inspector. That's not to say there isn't any "boneheaded" fabricator that will refuse to grind the back gouged surface that clearly needs to be. It is then up to the inspector to be sharp enough to refer to AWS D1.1 paragraphs 5.15.4 (including the subordinate paragraphs) and require corrective action based on surface roughness requirements that will surely be violated and the requirement that cut surface must be free of slag (refer to paragraph 5.15), etc. 

Best regards - Al