First off .......

...... for using an abrasive to get the glue off ......
Now that you've done it, I would suggest you use some warm soapy water to clean it up.
Followed by a bit of rubbing alcohol to make sure you got all the goo and wax off.
Then clay bar the area.
If it still looks rough, rub in some scratch remover (or light polishing compound).
If it looks good, wax it.
Finish your beer, you're done.
humfrz