Flink on Zeppelin

Setup Zeppelin
Main Features
Flink on Zeppelin Architecture
Execution mode
Languages
Hive Integration
Interpreter binding mode
Configure Flink Interpreter
Third party dependencies
UDF
Checkpoint & Savepoint
Recovery
Flink Sql Cookbook
FAQ

Powered by GitBook

On this page

Was this helpful?

Execution mode

yarn mode

Previousremote mode Nextyarn-application mode

Last updated 3 years ago

Was this helpful?

Yarn mode means Zeppelin will create a flink session cluster in yarn cluster. Then you can submit flink jobs to this flink session cluster.

Configuration

Besides FLINK_HOME and flink.execution.mode, you also need to configure HADOOP_CONF_DIR, and make sure the zeppelin machine can access hadoop cluster.
Make sure zeppelin machine install hadoop client and hadoop command on the PATH. Because flink is not shipped with hadoop by default (although you can download flink-shaded-hadoop and put it under flink lib, but it is not the recommended approach). Zeppelin will run command hadoop classpath underneath, and put them on the classpath of flink interpreter process.

After these configuration, you can run wordcount example to verify it. And you will see a flink session cluster launched in yarn as following.

Video Tutorial

Community

Join Zeppelin community to discuss with others

Configuration
Video Tutorial
Community