Are you looking at pulling a camper/5th wheel or payload as in slide in camper.
Slide in campers are a big NO NO with a F150. A lot depends on how big a camper you want to get and if you are going to be full time RVing or not.
Driving a big 250/350 Screw 8' PSD can get old for a daily driver to/from work if you put any distance on. Loaded they ride fine but empty you'll feel every bump. If your like most of us, your truck will be 99% unloaded rolling up and down the road.
What type of off roading...Sand? Rocks? Mud? Dirt? and how often?

If you get a diesel a few years old, make sure you aren't buying someone elses troubles. 6.0L and 6.4L trucks I would personnally stay away from buying used ( I know, I had a '05 PSD from new till trade-in for the Eco). Newer diesels are ok but you have to fill with diesel exhaust fluid (DEF).