Logging in Flink with Apache Beam

‹ Cutie Fish and Bear | What is slf4j? ›

Logging in Flink is implemented using the slf4j logging interface on top of log4j. This means that the most flexible approach you can take in your user pipeline code is to depend on slf4j instead of depending on log4j directly.

In the fat jar that you submit into the flink cluster, you include the slf4j-log4j12 log4j binding for slf4j. This causes the slf4j logging calls to be sent to log4j.

Then, in the appropriate .properties file, you can configure log4j to control log levels. For example, with a standalone cluster, and the default setup, you can set

log4j.rootLogger=DEBUG, file

to turn on DEBUG logging

Subscribe to All Posts - Wesley Tanaka