This week I encountered a problem when migrating a cron job to Cloud Run Job. I had experiences in this kind of task so many times before so I expected every thing to go smoothly as it used to be, until the cron run the 1st time and returned a 255 exit code.

The error message told that the process was lack of memory to run. It had a memory limit of 128M and I was wondering where was this limit because I set the memory limit on Cloud Run as much as 512Mi. After a short investigation, I realized that 128M was the memory limit of the php-fpm Docker image I built some months ago. Since I didn’t touch the memory limit, it is the default of the official php-fpm Docker image.

Next, I had to investigate how much memory this cron job needs so I could set an appropriate number of memory limit for it. I used the following PHP functions to check it.

I was surprised to find that while almost every cron processes consumed only a few MB to around some MB, this particular process consumed approximately 700MB of memory. That’s why every other cron processes are running very well with the default 128M memory limit.

Some lessons I’ve learnt

  1. The php-fpm container is limited to its memory_limit which is configured in php.ini as the default or via ini_set on demand.

  2. Memory limit in PHP is the limit for each individual PHP’s script, while the memory limit in Cloud Run is the limit for the entire container.

  3. If the cron job primarily executes a single PHP script, then it is meaningless to set the Cloud Run (Job) a memory limit that is too much higher than the limit of PHP.

  4. Increasing PHP’s memory_limit cannot speed up the process of the script. It just means that we give more room for the script to run. However, it is ideal to optimize the script so that it can use as less memory as possible.

Endings

This is a total new experience to me. I feel lucky when I have the chance to work with the cloud so I can get to know something that is hard to know in case of running the processes in a stand-alone virtual machine which is usually allocated a lot of memory.

Credits

Photo by Nils Stahl on Unsplash