This issue is normal operation, it has to do with the design of the parking brake on most Subaru's. The Parking/E-Brake on most Subaru's are a "Drum In Disc" type design where there is a smaller drum brake inside the hub of your rear Disc brakes.
If you have ever worked on "Self-Adjusting Drum Brakes" then you know that the entire Mechanism inside the brakes floats around a bit as a function of the self adjustment part of the brakes. This is what causes the movement when you let off the floor pedal, because you are switching from the disc brake portion of the brakes to the drum brake portion and the drum brake portion of the brakes floats a little bit inside the drum as part of the self-adjustment function of the drum brakes.
Here is a useful animation on how drum brake adjusters work.
http://auto.howstuffworks.com/auto-p...rum-brake2.htm