Logo Knowledge
  • Product Documentation
  • Insight
  • Developer Center
  • Training
  • Submit a Request
  • Product Documentation
  • Insight
  • Developer Center
  • Training
  • Submit a Request
  1. Acquia Support Knowledge Base
  2. Troubleshooting
  3. Common Problem

Analyzing Your Traffic

    See more
    • Created
    • December 07, 2018 22:53

    Issue

    Your site is experiencing a traffic spike, and you'd like to know more about where the traffic is coming from, or what is being targeted.

    Resolution

    If you’re noticing that your site is responding more slowly than normal, you can use the Acquia Log Streaming tool to see if you’re receiving more traffic than usual.

    But how do you determine if there are patterns to the traffic?  

    You will need the raw logs to analyze.  You can download these from the Log Streaming page, or you can SSH in to the server.  Accessing the logs via the Acquia Cloud log streaming interface is quick, but provides limited search customization. Accessing the logs via SSH will allow greater search flexibility. Here are some examples:

    Find the most frequent IP addresses

    awk '{print $1}' access.log| sort | uniq -c | sort -nr| head

    What can I do with this information?

    • If this is malicious traffic, you can block it.

    • If this is an aggressive bot that you do not want to block at this time, you can increase the crawl-delay in robots.txt.  Note that it is up to the bot whether it respects this directive.

    • If you need this traffic to continue, and your site’s performance is being impacted, you can see if these performance recommendations help.

    Find the most frequent User Agents

    awk -F\" '{print $6}' access.log | sort | uniq -c | sort -nr | head

    What can I do with this information?

    • If this is malicious traffic, you can block it.

    • If this is an aggressive bot that you do not want to block at this time, you can increase the crawl-delay in robots.txt.  Note that it is up to the bot whether it respects this directive.

    • If you need this traffic to continue, and your site’s performance is being impacted, you can see if these performance recommendations help.

    Find the most frequent URLs visited on your site

    awk -F\" '{print $2}' access.log| awk -F? '{print $1}' | sort | uniq -c | sort -nr | head

    This query strips the variables (anything after a question mark in your URL).

    What can I do with this information?

    • If the URLs that appear typically have a variable string at the end, there is a good chance that these are bypassing Varnish if they are varied enough.  These are often intentional so the site can process the requests appropriately. If these are required but the site’s performance is being impacted, you may want to see if these performance recommendations help, or if an upsize is needed.

    • If there is no variable string, confirm if Varnish is serving this page.

      • If it is, but the amount of traffic hitting the webs is impacting performance, increase your page cache maximum age until the traffic subsides.

      • If it is not, investigate why the site is bypassing Varnish cache.  If you need assistance determining why Varnish is not serving the page, contact Acquia Support.

    Performance Recommendations

    • Make sure that Drupal core and all modules are updated.  Many updates include performance improvements (and bug fixes) that can make a big difference in how your site responds.

    • Increase the page cache maximum age.  If you are concerned about stale content remaining on the site after an update, Acquia Purge can prevent this.

    • Make sure dblog is disabled.  Dblog writes log entries to the database, which can cause a performance impact.  We recommend syslog.

    • Do not include memory limit overrides in settings.php.  If you need a higher PHP memory limit for certain pages, we recommend increasing the limit just for these specific pages.

    • Enable memcaching.  

    • Check for opcache errors.

    Acquia Products

    • Acquia Cloud

    External Links

    • Restricting website access
    • Introduction to Varnish
    • Varnish: Bypassing the cache
    • Acquia Support
    • Using Varnish
    • Installing Acquia Purge
    • Conditionally increasing memory limits
    • Improving application performance
    • Using Memcached
    • Understanding and Resolving PHP OPcache and OPcache Interned Strings buffer errors
    Avatar
    Daphne Garrison
    • December 07, 2018 22:53
    • Facebook
    • Twitter
    • LinkedIn

    Was this article helpful?
    0 out of 0 found this helpful

    Return to top

    Related articles

    • Preparing for a high-traffic event
    • Block Access to Bad Bots coming from the Huawei Cloud
    • Upgrading from Drupal 8 to Drupal 9
    • Upgrading your database(s) to MySQL 5.7
    • Upgrading from PHP 7.2 to 7.3

    Support

    Acquia Support Knowledge Base
    • Submit a Request
    • Contact Support
    • Acquia Support Guide
    • Product Documentation
    • System Status

    About Acquia

    • About Us
    • Leadership
    • Board of directors
    • Newsroom
    • Careers
    • Customers
    • Contact Us
    53 State Street, 10th Floor
    Boston, MA 02109
    United States
    Phone: 888-922-7842
    Map: Google Maps
    View other locations
    • Feeds
    • Legal
    • Security Issue?

    Copyright © 2018 Acquia Inc. All Rights Reserved. Drupal is a registered trademark of Dries Buytaert.