yarn mode

Yarn mode means Zeppelin will create a flink session cluster in yarn cluster. Then you can submit flink jobs to this flink session cluster.

Configuration

• Besides FLINK_HOME and flink.execution.mode, you also need to configure HADOOP_CONF_DIR, and make sure the zeppelin machine can access hadoop cluster.

• Make sure zeppelin machine install hadoop client and hadoop command on the PATH. Because flink is not shipped with hadoop by default (although you can download flink-shaded-hadoop and put it under flink lib, but it is not the recommended approach). Zeppelin will run command hadoop classpath underneath, and put them on the classpath of flink interpreter process.

After these configuration, you can run wordcount example to verify it. And you will see a flink session cluster launched in yarn as following.

Video Tutorial

Community

Join Zeppelin community to discuss with others

Community