When you consistently exceed expectations and have proven yourself invaluable to the project, nobody questions what you're doing with your time
If they did, I'd probably start looking for a healthier/happier work environment.
Government employees apparently can't get fired so they can do whatever they want and worst that will happen is they might get promoted out of the position to prevent them f'ing things up anymore.