Might be vacuum related if it gets a little worse when put into drive (or any other gear).

Try unplugging vac lines, one at a time, and see if the noise goes away ( especially anything on the firewall, under the hood or the dash). Also, if it was vac related, tap the gas pedal a few times to get the RPM's to jump to 2500-3000. The vac will drop pretty low then, so if something is ruptured, the noise should momentarily cease.

Dave