AWS Feature Request: Enable Package-Level Logging Overrides in Managed Flink

Select Language:

If you’re using Amazon Managed Service for Apache Flink and need more control over your logging setup, you’re not alone. Currently, the service only allows setting a single global log level, such as INFO, DEBUG, WARN, or ERROR. This setup works fine for simple applications, but it can cause major issues when working with modern Kafka connectors or other source architectures based on the new FLIP-27 standards.

Many users find that enabling DEBUG logs for troubleshooting causes their environment to slow down or even become unusable. This happens because certain internal components, like the SplitFetcher, generate a huge volume of log messages when set to DEBUG. These messages flood CloudWatch logs, overwhelming the system and resulting in dropped log packets. As a result, critical application logs don’t make it to CloudWatch Logs Insights, making debugging nearly impossible.

Trying to solve this by modifying the logging configuration inside the Java application doesn’t work well because the Managed Flink runtime isolates or overrides the logging environment before your code runs. So, what can you do?

The best solution is to find a way to pass custom logging configurations directly to the cluster. Here are some practical options:

Use an AWS application property group where you can specify key-value pairs for logging. For example, you could set a property named Log4jOverrides and define specific logger names and levels, like logger.splitfetcher.name = org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher and logger.splitfetcher.level = INFO. This allows you to target specific components for different log levels.
Bundle your own custom log4j.properties or log4j2.xml file within your application JAR. When the cluster launches, it can read and apply this configuration, giving you fine-grained control over what gets logged and at what level.
Pass JVM arguments directly to the cluster deployment, such as setting -Dlog4j2.configurationFile=path-to-your-config.xml. This way, you can define the exact logging setup you want without relying solely on AWS configuration.

Open-source Flink community discussions highlight that using log4j.properties to filter out noisy logs like those from SplitFetcher is best practice. Unfortunately, Managed Flink currently lacks this flexibility, creating a blind spot when trying to troubleshoot modern connectors.

Implementing one of these options will give you the ability to fine-tune your logging, reduce unnecessary noise, and improve your debugging experience. This change would significantly help enterprise teams manage their environments more effectively when working with the latest Flink features.