I removed the throttle body and was able to get it out with some vise grips.
New regulator did not solve the problem. It looks like it's the fuel pump.