I've done ~25 track days and have seen a fair number of wrecks. Last year I was following behind someone in his BRZ when he lost the rear end exiting a fast corner. He hit the tire wall and totaled his car. A lot of spins from RWD cars (BRZ, S2000, MR2, NSX, E36 M3, Miata etc.) A couple of offs caused by fluid on the track or issues with brakes/suspension. No car-to-car incidents I can think of though.
Personally I haven't gone 4-wheels off yet (*knock on wood), but have gone 2-wheels off a few times, mostly trying to pass similar speed cars in corners or driving in the rain.
I've noticed that a lot of incidents happen in the first few laps of a session (cold tires), in the last session of the last day, or during open track time when people have been out driving over 30 minutes. Those are probably good times to take things down a notch. As others have said, if you use common sense and push your limits slowly, you should be fine. Just keep in mind the trophy at the end of your track day is the shiny car you get to drive home.