Help Me Out Here!

I have a legal (or is it theory?) question. The President has has asserted executive privilege with regard to staff or cabinet level advisers. This has been presented with the notion that if the President's advisers cannot speak freely, without their words being made public for political gain, then the President is unreasonably denied the frank advice which must be afforded the high office.

Ok, let's say we all (or at least I) buy that line of reasoning. Why is it that this sort of privilege should be extended to individuals who are no longer working with an advisory/executive status in the government? (e.g.: Karl Rove, Monica Goodling, etc.) It seems to me they no longer have to worry about ensuring frank exchanges from people who no longer work with the president, as they will no longer have the protected advisory role. This, would negate the supposed 'chilling effect' that the executive privilege doctrine proposes to protect.

Any thoughts?