How to Reduce Bottlerocket's High Memory Usage on AWS

Select Language:

If you’re noticing a big jump in memory use after switching from Amazon Linux 2 to Bottlerocket, the cause is probably related to how the system manages resource limits rather than an issue with Bottlerocket itself.

Amazon Linux 2 uses cgroup v1 to control resource allocation, while Bottlerocket, like newer Amazon Linux versions, defaults to cgroup v2. This difference can impact how certain applications, especially Java programs, handle memory. If your Java applications are running on older JDK versions—specifically JDK 8 versions before update 372—they might not work well with cgroup v2.

The problem is that cgroup v2 stores memory information in different locations, and older JVM versions might not be able to detect container memory limits properly. When this happens, the JVM assumes the total server memory is available, rather than respecting the set limits for each container. This can cause the application to use much more memory than intended, leading to possible Out of Memory errors.

To fix this, you have a few options:

First, consider upgrading your Java version. Moving to JDK 8u372 or newer will improve compatibility with cgroup v2. Alternatively, you can set Bottlerocket to use cgroup v1 temporarily. This involves changing kernel parameters in your Bottlerocket configuration, allowing older JVM versions to recognize memory limits correctly, at least until a more permanent fix is put in place.

Lastly, check your Kubernetes resource settings. Ensure that your memory requests and limits are properly defined in your pod specifications. Since resource management differs slightly between cgroup versions, setting these correctly helps your applications behave as expected.

This increase in memory usage isn’t a problem with Bottlerocket itself but results from the interaction between your applications and the underlying resource management system. Adjusting your Java versions or system settings can help restore normal memory behavior and prevent unnecessary resource consumption.