Issue
Varnish is not caching as expected.
Resolution
Drupal has several caching mechanisms that can store rendered pages into the Varnish cache hierarchy for future recall. (For more detailed documentation regarding Varnish caching, see the Varnish cache documentation.)
Varnish caches everything from Drupal that contains the proper and default output headers — this means that it works optimally if the performance settings (at http://[site_URL]/admin/settings/performance) are left at their default settings, and aggregation is enabled. Whenever a user signs in, Drupal sets a cookie in the browser and from that point on, as long as the browser keeps sending that cookie, Varnish cache will not be used. This effectively means that Varnish is not used if the outdated page is displayed fresh when you sign in.
Varnish is not the only caching mechanism, so let's look at the entire stack:
Varnish: Varnish serves as a gatekeeper and will cache everything as long as a user is anonymous and not on an HTTPS connection. Non-anonymous and HTTPS traffic will directly go to the web servers.
Apache: When a request is not served by Varnish, the web server checks against the rules in .htaccess. If the requested URL exists as a file on disk, that file will be served and PHP/Drupal will not start. For example, the ImageCache and Boost modules work this way, writing images and HTML to the files directory, which saves expensive calls to PHP, Drupal and the database. Be aware that the Boost module is not recommended on Acquia Cloud.
Drupal fast page cache: Modules can choose to implement the so-called fast page cache, which means that a PHP process is already started but Drupal is still in its early bootstrap phase and has no connection to the database yet. A site that has this usually has special code in the settings.php file. The Fast 404 module works this way.
Drupal page caching: When Drupal has a connection to the database and anonymous page caching is enabled, it checks the requested page against the cache_pagetable in the database or the cache bin when Memcached is used. When it finds a record, it serves this back to Varnish (which will recache it for the next time) and stops the request.
Other cached elements
When a page is not served from one of the earlier caching techniques, this usually means that Drupal will fully load its subsystems and start pulling the page together. However, the following items are still being served from other Drupal caches to make this as inexpensive as possible:
- Full pages when the Authenticated user page caching (authcache) module is in use, designed for this purpose
- System components like menu trees, content types, and template metadata
- When block cache is enabled, blocks are returned from the block cache
- Views result sets or rendered views can be served when Views caching is configured
More troubleshooting
If you have edited content and have to save it more than once to make caching update, this generally indicates that the problem is at the application level. This can be caused either by poorly designed contributed modules that leverage caching or by other special systems at play. It's also important to note that Varnish is a completely loose component from the Drupal level of the stack and that cron normally has no influence on it. When a page is saved, a message should go out to Varnish saying "please clean me from your cache," and this can be achieved by using either the Acquia Purge module, another purging module, or a custom, yet more flexible, rule that has PHP code for this.