When I find it annoying, I typically switch over to ECO mode. It lowers the fake engine noise from the speakers. Leaving you with just the sound of your exhaust. It might help.
At least it matches the fake engine sound coming out of the speakers now.