📋
Flink on Zeppelin
  • Setup Zeppelin
  • Main Features
  • Flink on Zeppelin Architecture
  • Execution mode
    • local mode
    • remote mode
    • yarn mode
    • yarn-application mode
  • Languages
    • Scala
    • PyFlink
    • SQL
  • Hive Integration
  • Interpreter binding mode
  • Configure Flink Interpreter
  • Third party dependencies
  • UDF
  • Checkpoint & Savepoint
  • Recovery
  • Flink Sql Cookbook
  • FAQ
Powered by GitBook
On this page
  • Architecture
  • Configuration
  • Community

Was this helpful?

  1. Execution mode

yarn-application mode

Previousyarn modeNextLanguages

Last updated 3 years ago

Was this helpful?

Application mode is a new execution mode which is introduced after Flink 1.11. The purpose of application mode is to mitigate the resource pressure on the client side. yarn-application mode would launch a Flink cluster on YARN, where the main() method of the application jar gets executed on the JobManager in YARN.

Architecture

The is the original yarn mode where Flink Interpreter would run in the same host with Zeppelin server host.

This is yarn-application mode where Flink Interpreter would run in JobManager.

Configuration

It is very to easy to configure yarn-application mode, very similar as yarn mode

  • Configure FLINK_HOME

  • Configure flink.execution.mode as yarn-application

  • Make sure hadoop client installed, and put hadoop in PATH. Because flink itself doesn't ship hadoop jars, 为Flink本身不带hadoop相关依赖（你可以把flink-shaded-hadoop 拷贝到flink lib下），when zeppelin run flink, it would calll command hadoop classpath，then put all the jars under CLASSPATH

Community

Join Zeppelin community to discuss with others

Community
Logo