I'm having a similar issue in my 13 FRS with 37.7k miles. Just bought it used at 37k miles 3 weeks ago. All was well until yesterday. At higher speeds/rpms (60mph+/4-5k) when downshifting from 6th to 5th there's a notchy feeling and a slight grind.
Some people say to live with it, or to get better with rev-matching and double-clutching. Some are getting new transmissions. We bought the FRS from Carmax and have the extended warranty but it's pretty worthless. I'm going to make an appointment with Toyota and let the tech drive and reproduce. Hopefully they will honor the 60/60 powertrain warranty and fix/replace the transmission.
Also, we switched to amsoil and it had no positive effect on the problem.
So until I can get it into the dealership I'm just driving feeder roads and staying under 60. Totally sucks because this car loves to be driven hard.
