Connection between library cache latches and statistics?

I'm encountering a strange problem. We're suffering from terrible library cache latches (30-50% of waits in statspack reports) when table's stats being older than two business days. I do not understand why, because the affected tables are only slowly changing and I can't imagine any possible connection between stats too old and library latch waits.

Any ideas?

Database: 9.2.0.7 EE 64bit Itanium

--
Dusan Bolek