Terrain paint showing up incorrectly

If it did this uniformly I'd be fine as it'd give it a gravel look, however, just some spots will change.

If I go into the editor and apply paint anywhere it all goes back to normal.

Before:




After:



Is there anyway to fix this so that it doesn't happen?

EDIT:
Blending dirt/rock will keep rock walls from being green from grass and looks better all around. Still, the roadside gravel...