How to investigate Memory Consumption problems

    See more
    • Updated

    Issue

    There are a number of reasons why your Drupal installation could run out of memory (oom). Insufficient allocation at install, additional modules, and large tasks are all causes for out-of-memory problems. These often manifest as PHP Fatal Error: Out of memory notices.

    Resolution

    Here are some techniques to help you figure out what is using significant amounts of memory:

    Logging and memory profiler

    Memory Profiler module is a lightweight module that can be used on a production site. However, we recommend testing memory consumption on staging or development servers.

    1. Increase your PHP Memory limit incrementally until the errors subside.
    2. Install and enable syslog (for the drupal-watchdog.log) and the Memory profiler module. Disable the dblog module.
    3. Repeat the steps that led to the initial high memory usage or oom error.
    4. Disable the Memory profiler module when testing is completed.

    Each page will then log an entry to the drupal-watchdog.log, with a tag of memory profiler and the amount of memory that page used. You can easily search and analyze this. You can then either create conditional memory increases for those paths as described in Conditionally increasing memory limits, or determine if an overall site memory boost is required.

    This is also a small enough footprint to use safely in production, but if you can replicate the issue on a development or staging environment, that is usually better.

    If you download your logs and have UNIX command line tools such as grep available, you can use this query to help parse the memory profiler output for today's log:

    # Run this from the shell in your /var/log/sites/[site.env]/logs/[servername] folder.
#
# This gets the latest 50 (see $num) memory-exhaustion errors from php-errors.log, 
# and then shows the corresponding lines found in $logfile (e.g. access.log, drupal-watchdog.log, etc)
# NOTE: Large values in $num can place load on your server while this runs--you should download files if you plan to do any heavy analysis.
#
num=50; logfile=access.log; grepstring=`grep "Allowed memory size" php-errors.log |tail -$num |grep -o 'request_id="[^"]*"' |awk -F'"' 'NR>1 { printf "|"; } { printf $2 }'`; egrep "$grepstring" $logfile

    XHProf

    XHProf can trace script execution times, as well as enable memory profiling. Memory profiling is not on by default, however, and must be enabled. The XHPROF_FLAGS_MEMORY flag will enable this feature. More information on both enabling xhprof and the xhprof contstants is on PHP.net. For specific information on enabling and using XHProf on your Drupal site, see Using XHProf to evaluate code performance.

    Other applications

    Many performance management software applications can also help you figure out if it's a memory, database, or other issue. Newrelic is able to be used on Acquia Cloud

    New Relic

    If you have a New Relic account, you can use that for determining where bottlenecks may be occurring for code, network or databases.

    For information on increasing memory limits, see the article on Conditionally increasing memory limits.

    Related articles