The AC switching to defrost is a vacuum leak. You have a broken hose, a bad vacuum solenoid, or something else causing a big vacuum leak when you engage the ESOF.

It's usually not a "hub going out". Check for broken vacuum lines from the vacuum solenoid on the passenger-side inner fender, going to a "T", and down to the front hubs. If you have one, use a hand-vacuum-pump to pull vacuum on the line going from the vacuum solenoid to the front hubs. If it holds vacuum, the vacuum solenoid needs to be replaced.

If the vacuum leak is intermittent, that sounds like the vacuum solenoid is hanging up.