I think that sandpaper, or steel wool, would be best. Wear gloves and dip steel wool in one of the following chemicals. These are guesses for cleaning: denatured alcohol, acetone, methyl ethyl ketone, unleaded gasoline (EXTREME CAUTION: all of these are highly flammable). Toluol or xy

(VERY TOXIC).
Or:
Switch to gray tufy.
Switch to a polyurethane.
Did you mix the black tufy thoroughly??
Is the residue from uncured polysulphide component??
Is this the first casting out of the mold?? If so, try a few more castings. Sometimes the first one or two castings yield less than desirable results.
Paint.
Make another mold. Use polyurethane.