I'm running a small server (f1-micro) on Google GCE. This server is acting as a reverse proxy, so only a small NGINX has been installed on it, which just proxies for a Google Cloud Function. Usage is very low, hardly getting traffic.
However, about once 1 month (might be even more often), the server crashes without apparent reason. Looking in the Console, CPU usage is through the roof, SSH access doesn't work, and the only solution is to kill it and restart the server. Often I also see high disk I/O.
Logging on to the server, I noticed quite high memory usage by the fluentd daemon, more specifically the ruby process, output of top below:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1123 root 20 0 40.7g 1.3g 1.3g S 0.7 233.1 0:40.79 ruby
867 ? Sl 0:00 \_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/aec49f162b96e0a58522a7ccdee023ad01f7abefbf996352bf4fe9e4fc976465 -address /run/containerd/containerd.sock -container 897 ? Ssl 0:05 | \_ /opt/google-fluentd/embedded/bin/ruby /usr/sbin/google-fluentd 1123 ? Sl 0:40 | \_ /opt/google-fluentd/embedded/bin/ruby -Eascii-8bit:ascii-8bit /usr/sbin/google-fluentd --under-supervisor
This is after running the server for about 12 hours. The memory usage is just a wild guess, but might explain disk I/O (swap usage). Any clues to why this is happening?