I've seen this technique used at various detailing places. I tried it out myself and it worked great. Spray down the windshield with some water, then go over it a few times in circular motions with some steel wool. Should take everything right off.

Razor blades, extra fine steel wool, and small amounts of paint thinner are my usual methods of cleaning the stuff that regular glass cleaner won't get off.

Use the least intrusive method possible, and then work your way to the more aggressive cutting stuff. It looks like you were following a cement truck or something. :p

Clay bar, lime away or deposit/acid wash, steel wool, blade, and if all else fails a random orbital buffer, glass pad, and some good cutting creme will take care of it. If not, a replacement windshield isn't crazy expensive. :p

You should pick up some glass polish - it gets rid of most of that stuff if used properly with an orbital or some time of machine polisher to break the compound up and fix it. Works similarly to paint polish... In fact, you could probably get away with using paint polish as well.

Glass cleaner or claybars typically don't get that stuff out. They look just like small little pits and stuff from small stones. An aggressive compound can usually take some of it off.