I took the headers off and saw a decent amount of soot on the drivers side gasket. The flange on the header was also changing colors from the heat so it's pretty obvious that was the source. I'm sure the gaskets aren't the problem but put the OEM gaskets back on and retorqued everything.
A big thanks to @
moto-mike for the idea to use a crows foot to get those inner bolts. Definitely makes it easier to get them tight.
Overall, sounds much better and the exhaust smell is gone.