I wanted to make sure it wasn't the rear main seal before I put it all back together but that gasket does seem to be the real problem.
I am going to replace the pan too, its pretty beat up. I don't know much about pans so I am going to start a new thread about select one... after I search.