There are differences. Typically the Lariat trim level comes with the LCD information center in the gauge pod. Many XLT's do not. That feature alone would make my decision for me. There is a wealth of information available there, that you simply do not have without it.

Beyond that, I like the other Lariat features, the one you are looking at may or may not have:

Ford Sync - hands free phone connection
Power rear window?
Heated/cooled seats
climate control

15K miles on the 2011 is hardly broken in. If it were me, I would go with the higher trim level assuming everything else is equal.