When a U joint fails while underway it can be catastrofic. I would advise getting it fixed. You as the driver are responsible for what comes flying off your vehicle.
I have had a driveshaft come loose and watched it tumble down the road in my rearview all the time hoping it didn't skip through someones windshield.
Please don't take this lightly get it repaired.