Just picked up my BRZ. Quick Question
 

https://scontent-atl3-1.xx.fbcdn.net...24&oe=56FA9FFF

https://scontent-atl3-1.xx.fbcdn.net...64&oe=56B2BFBB

How easy is it to remove the diffuser piece under the exhaust? I'm not the biggest fan of it

I'd start with taking a look at the OEM Subaru rear diffuser installation instructions, assuming that is an OEM diffuser. Here is a link:
http://www.subaruonlineparts.com/docs/E5610CA000.pdf

And then you can ship it to me... :)

About 10 minutes, assuming you know how to use tools.

-alex

And another 10 minutes to pack into a box and ship to my doorstep!

And another 30 to take off the tail lights and ship them to me!

A follow-up; I just noticed you got a 2013 twin but have the shark fin-style antenna, so I suppose it is more likely that your diffuser is aftermarket (I only assume this because the price differential between OEM and aftermarket diffusers is ~$600, not judging or anything). If so, it is likely from Advan-Emotion which installs identically to the OEM diffuser, so reversing the installation instructions I linked to above should be sufficient to remove. You will have a couple holes left over in your bumper cover, but they shouldn't be visible except when under the car.