Seeking advice with Wordpress and TTFB

I have a client who has 30 or so Wordpress sites..... Some have become slow. You might say "check plugins", and that was my first thought....

However, upon inspection:

1. TTFB takes seconds;
2. The admin areas are not affected at all, they are fast;

So, I'm not too sure what is up. The server currently uses mod_php + mod_ruid2, I am going to try php-fpm later tonight then try to optimise apache and mysql more than they are.

Has anyone experienced this with Wordpress sites?

PS. TTFB = Time to first byte