I wouldn't go too big too fast. Larger pipe means lower velocity. For a NA engine scavenging is important and if you go too big you don't scavenge well.

There's a practical limit to larger exhaust and I think for a 3.5 engine at normal (not 8000 rpm race) operating speed 4" might be beyond it.