Azkaban 系列文章

1、azkaban介绍、三种（solo-server、two-server和multiple-executor）部署方式及验证 2、azkaban-3.51.0 任务类型（Jobtypes，详细介绍command、java2种类型及示例） 3、azkaban-3.51.0 条件工作流flow和参数传递 4、azkaban-3.51.0 通过api进行操作azkaban界面功能

本文的前提是上一篇的azkaban使用介绍，因为本文的示例不会再说怎么上传、运行，而是直接给出源文件与运行结果。本文分为四个部分，即azkaban支持的类型、command类型示例、java类型示例和任务调度。

一、Azkaban支持的plugin类型

command：Linux shell命令行任务
gobblin：通用数据采集工具
hadoopJava：运行hadoopMR任务
java：原生java任务
hive：支持执行hiveSQL
pig：pig脚本任务
spark：spark任务
hdfsToTeradata：把数据从hdfs导入Teradata
teradataToHdfs：把数据从Teradata导入hdfs

二、command类型

1、 job配置（示例）示例一

test.job

type=command
command=echo "This is azkaban cmd ... "
command.1=whoami
#依赖前一个job
dependencies=cmd1

cmd1.job

type=command
command=echo "This is azkaban cmd1 ... "

运行日志 test.job

18-08-2022 11:10:12 CST cmd1 INFO - Starting job cmd1 at 1660792212782
18-08-2022 11:10:12 CST cmd1 INFO - job JVM args: -Dazkaban.flowid=test -Dazkaban.execid=36 -Dazkaban.jobid=cmd1
18-08-2022 11:10:12 CST cmd1 INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 11:10:12 CST cmd1 INFO - Building command job executor. 
18-08-2022 11:10:12 CST cmd1 INFO - Memory granted for job cmd1
18-08-2022 11:10:12 CST cmd1 INFO - 1 commands to execute.
18-08-2022 11:10:12 CST cmd1 INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36
18-08-2022 11:10:12 CST cmd1 INFO - effective user is: azkaban
18-08-2022 11:10:12 CST cmd1 INFO - Command: echo "This is azkaban cmd1 ... "
18-08-2022 11:10:12 CST cmd1 INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_output_7156127205290501411_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_props_4986904569339043450_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__test__cmd1__36__azkaban, JOB_NAME=cmd1}
18-08-2022 11:10:12 CST cmd1 INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36
18-08-2022 11:10:12 CST cmd1 INFO - This is azkaban cmd1 ... 
18-08-2022 11:10:12 CST cmd1 INFO - Process completed successfully in 0 seconds.
18-08-2022 11:10:12 CST cmd1 INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_output_7156127205290501411_tmp
18-08-2022 11:10:12 CST cmd1 INFO - Finishing job cmd1 at 1660792212797 with status SUCCEEDED

cmd1.job

18-08-2022 11:10:12 CST cmd1 INFO - Starting job cmd1 at 1660792212782
18-08-2022 11:10:12 CST cmd1 INFO - job JVM args: -Dazkaban.flowid=test -Dazkaban.execid=36 -Dazkaban.jobid=cmd1
18-08-2022 11:10:12 CST cmd1 INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 11:10:12 CST cmd1 INFO - Building command job executor. 
18-08-2022 11:10:12 CST cmd1 INFO - Memory granted for job cmd1
18-08-2022 11:10:12 CST cmd1 INFO - 1 commands to execute.
18-08-2022 11:10:12 CST cmd1 INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36
18-08-2022 11:10:12 CST cmd1 INFO - effective user is: azkaban
18-08-2022 11:10:12 CST cmd1 INFO - Command: echo "This is azkaban cmd1 ... "
18-08-2022 11:10:12 CST cmd1 INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_output_7156127205290501411_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_props_4986904569339043450_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__test__cmd1__36__azkaban, JOB_NAME=cmd1}
18-08-2022 11:10:12 CST cmd1 INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36
18-08-2022 11:10:12 CST cmd1 INFO - This is azkaban cmd1 ... 
18-08-2022 11:10:12 CST cmd1 INFO - Process completed successfully in 0 seconds.
18-08-2022 11:10:12 CST cmd1 INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_output_7156127205290501411_tmp
18-08-2022 11:10:12 CST cmd1 INFO - Finishing job cmd1 at 1660792212797 with status SUCCEEDED

2、 job配置（依赖）示例二

basic.flow

---
config:
  failure.emails: noreply@foo.com

nodes:
  - name: jobC
    type: noop
    # jobC depends on jobA and jobB
    dependsOn:
      - jobA
      - jobB

  - name: jobA
    type: command
    config:
      command: echo "This is an echoed text."

  - name: jobB
    type: command
    config:
      command: pwd

图形依赖示例
执行结果
运行日志 JobB

18-08-2022 13:51:03 CST jobB INFO - Starting job jobB at 1660801863329
18-08-2022 13:51:03 CST jobB INFO - job JVM args: -Dazkaban.flowid=basic -Dazkaban.execid=37 -Dazkaban.jobid=jobB
18-08-2022 13:51:03 CST jobB INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 13:51:03 CST jobB INFO - Building command job executor. 
18-08-2022 13:51:03 CST jobB INFO - Memory granted for job jobB
18-08-2022 13:51:03 CST jobB INFO - 1 commands to execute.
18-08-2022 13:51:03 CST jobB INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37
18-08-2022 13:51:03 CST jobB INFO - effective user is: azkaban
18-08-2022 13:51:03 CST jobB INFO - Command: pwd
18-08-2022 13:51:03 CST jobB INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobB_output_3128039260514846300_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobB_props_7802985401895047938_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__basic__jobB__37__azkaban, JOB_NAME=jobB}
18-08-2022 13:51:03 CST jobB INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37
18-08-2022 13:51:03 CST jobB INFO - /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37
18-08-2022 13:51:03 CST jobB INFO - Process completed successfully in 0 seconds.
18-08-2022 13:51:03 CST jobB INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobB_output_3128039260514846300_tmp
18-08-2022 13:51:03 CST jobB INFO - Finishing job jobB at 1660801863361 with status SUCCEEDED

JobA

18-08-2022 13:51:03 CST jobA INFO - Starting job jobA at 1660801863336
18-08-2022 13:51:03 CST jobA INFO - job JVM args: -Dazkaban.flowid=basic -Dazkaban.execid=37 -Dazkaban.jobid=jobA
18-08-2022 13:51:03 CST jobA INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 13:51:03 CST jobA INFO - Building command job executor. 
18-08-2022 13:51:03 CST jobA INFO - Memory granted for job jobA
18-08-2022 13:51:03 CST jobA INFO - 1 commands to execute.
18-08-2022 13:51:03 CST jobA INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37
18-08-2022 13:51:03 CST jobA INFO - effective user is: azkaban
18-08-2022 13:51:03 CST jobA INFO - Command: echo "This is an echoed text."
18-08-2022 13:51:03 CST jobA INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobA_output_1861938410828184844_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobA_props_7973309314170642567_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__basic__jobA__37__azkaban, JOB_NAME=jobA}
18-08-2022 13:51:03 CST jobA INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37
18-08-2022 13:51:03 CST jobA INFO - This is an echoed text.
18-08-2022 13:51:03 CST jobA INFO - Process completed successfully in 0 seconds.
18-08-2022 13:51:03 CST jobA INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobA_output_1861938410828184844_tmp
18-08-2022 13:51:03 CST jobA INFO - Finishing job jobA at 1660801863361 with status SUCCEEDED

JobC

18-08-2022 13:51:03 CST jobC INFO - Starting job jobC at 1660801863373
18-08-2022 13:51:03 CST jobC INFO - job JVM args: -Dazkaban.flowid=basic -Dazkaban.execid=37 -Dazkaban.jobid=jobC
18-08-2022 13:51:03 CST jobC INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 13:51:03 CST jobC INFO - Building noop job executor. 
18-08-2022 13:51:03 CST jobC INFO - Finishing job jobC at 1660801863383 with status SUCCEEDED

3、 job配置（嵌入式）示例三

embedded.flow

---
config:
  failure.emails: noreply@foo.com

nodes:
  - name: jobC
    type: noop
    dependsOn:
      - embedded_flow

  - name: embedded_flow
    type: flow
    config:
      prop: value
    nodes:
      - name: jobB
        type: noop
        dependsOn:
          - jobA

      - name: jobA
        type: command
        config:
          command: pwd

图示
运行结果
运行日志

18-08-2022 13:59:03 CST jobC INFO - Starting job jobC at 1660802343666
18-08-2022 13:59:03 CST jobC INFO - job JVM args: -Dazkaban.flowid=embedded -Dazkaban.execid=38 -Dazkaban.jobid=jobC
18-08-2022 13:59:03 CST jobC INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 13:59:03 CST jobC INFO - Building noop job executor. 
18-08-2022 13:59:03 CST jobC INFO - Finishing job jobC at 1660802343677 with status SUCCEEDED

4、job配置示（hdfs）例四

本示例是hdfs创建目录

创建job

# fs.job
type=command
command=/usr/local/bigdata/hadoop-3.1.4/bin/hadoop fs -mkdir /azkatest

将job资源文件打包成zip文件
通过azkaban的web管理平台创建project并上传job压缩包
启动执行该job

5、job配置（mapreduce）示例五

执行mapreduce任务，执行示例为wordcount。

创建job

# mrwc.job
type=command
command=/usr/local/bigdata/hadoop-3.1.4/bin/hadoop  jar hadoop-mapreduce-examples-2.6.1.jar wordcount /wordcount/input /wordcount/azout

将job资源文件打包成zip文件
通过azkaban的web管理平台创建project并上传job压缩包
启动执行该job

6、job配置（hive）示例六

执行hive的sql

Hive脚本， test.sql 假设数据文件都是准备好的，hadoop、hive环境都是好用的

use default;
drop table aztest;
create table aztest(id int,name string) row format delimited fields terminated by ',';
load data inpath '/aztest/hiveinput' into table aztest;
create table azres as select * from aztest;
insert overwrite directory '/aztest/hiveoutput' select count(1) from aztest;

创建job

# hivef.job
type=command
command=/usr/local/bigdata/hive/bin/hive -f 'test.sql'

将job资源文件打包成zip文件
通过azkaban的web管理平台创建project并上传job压缩包
启动执行该job

三、java类型

1、 job配置（普通java）示例一

1）、创建job

testjavajob.job

type=javaprocess
classpath=./lib/*,${azkaban.home}/lib/*
java.class=com.win.TestAzkabanType param1 param2

2）、TestAzkabanType.java

package com.win;

public class TestAzkabanType {
	public static void main(String[] args) {
		System.out.println("this is a java class. java type,package:com.win!  参数1="+args[0]+"  参数2= "+args[1]);
	}
}

3）、打包

将testjavajob.job和TestAzkabanType.jar打包成一个zip文件，并上传至azkaban中

4）、运行

2、azkaban-3.51.0 任务类型（Jobtypes，详细介绍command、java2种类型及示例）

5）、运行日志

22-08-2022 16:32:51 CST testjavajob INFO - Starting job testjavajob at 1661157171968
22-08-2022 16:32:51 CST testjavajob INFO - job JVM args: -Dazkaban.flowid=testjavajob -Dazkaban.execid=94 -Dazkaban.jobid=testjavajob
22-08-2022 16:32:51 CST testjavajob INFO - user.to.proxy property was not set, defaulting to submit user azkaban
22-08-2022 16:32:51 CST testjavajob INFO - Building javaprocess job executor. 
22-08-2022 16:32:51 CST testjavajob INFO - Memory granted for job testjavajob
22-08-2022 16:32:51 CST testjavajob INFO - No classpath specified. Trying to load classes from /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94
22-08-2022 16:32:51 CST testjavajob INFO - 1 commands to execute.
22-08-2022 16:32:51 CST testjavajob INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94
22-08-2022 16:32:51 CST testjavajob INFO - effective user is: azkaban
22-08-2022 16:32:51 CST testjavajob INFO - Command: java -Dazkaban.flowid=testjavajob -Dazkaban.execid=94 -Dazkaban.jobid=testjavajob -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType param1 param2 
22-08-2022 16:32:51 CST testjavajob INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94/testjavajob_output_7286382865992072360_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94/testjavajob_props_8373462308281870001_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__testjavajob__testjavajob__94__azkaban, JOB_NAME=testjavajob}
22-08-2022 16:32:51 CST testjavajob INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94
22-08-2022 16:32:52 CST testjavajob INFO - this is a java class. java type,package:com.win!  参数1=param1  参数2= param2
22-08-2022 16:32:52 CST testjavajob INFO - Process completed successfully in 0 seconds.
22-08-2022 16:32:52 CST testjavajob INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94/testjavajob_output_7286382865992072360_tmp
22-08-2022 16:32:52 CST testjavajob INFO - Finishing job testjavajob at 1661157172037 with status SUCCEEDED

其日志中有执行命令，实际情况可以根据该命令确定执行的main-class，看执行命令好像打包的时候不需要指定具体的main-class

22-08-2022 16:32:51 CST testjavajob INFO - Command: java -Dazkaban.flowid=testjavajob -Dazkaban.execid=94 -Dazkaban.jobid=testjavajob -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType param1 param2

如果涉及到引用jdk外的包，需要一并打包进jar即可（或指定具体的目录结构，形如：classpath=./lib/*,${azkaban.home}/lib/*）

2、job配置（依赖）示例三

1）、创建java文件并打包成jar

参见示例1

2）、创建job

innerFlow.job

type=javaprocess
java.class=com.win.TestAzkabanType innerFlow dependencies=innerJobB,innerJobC
seconds=1
fail=false
dependencies=innerJobB,innerJobC

innerJobC.job

type=javaprocess
java.class=com.win.TestAzkabanType innerJobC innerJobA
seconds=1
fail=false
dependencies=innerJobA

innerJobA.job

type=javaprocess
java.class=com.win.TestAzkabanType innerJobA nothing
seconds=1
fail=false

innerJobB.job

type=javaprocess
java.class=com.win.TestAzkabanType innerJobB innerJobA
seconds=1
fail=false
dependencies=innerJobA

3）、打包

4个job和一个jar文件一起打包，并上传即可

4）、运行

2、azkaban-3.51.0 任务类型（Jobtypes，详细介绍command、java2种类型及示例）

5）、运行日志

整体运行日志

22-08-2022 16:44:59 CST innerFlow INFO - Assigned executor : localhost:12321
22-08-2022 16:44:59 CST innerFlow INFO - Running execid:95 flow:innerFlow project:6 version:56
22-08-2022 16:44:59 CST innerFlow INFO - Updating initial flow directory.
22-08-2022 16:44:59 CST innerFlow INFO - Fetching job and shared properties.
22-08-2022 16:44:59 CST innerFlow INFO - Starting flows
22-08-2022 16:44:59 CST innerFlow INFO - Running flow 'innerFlow'.
22-08-2022 16:44:59 CST innerFlow INFO - Configuring Azkaban metrics tracking for jobrunner object
22-08-2022 16:44:59 CST innerFlow INFO - Submitting job 'innerJobA' to run.
22-08-2022 16:44:59 CST innerFlow INFO - Created file appender for job innerJobA
22-08-2022 16:44:59 CST innerFlow INFO - Attached file appender for job innerJobA
22-08-2022 16:44:59 CST innerFlow INFO - Job Started: innerJobA
22-08-2022 16:45:00 CST innerFlow INFO - No attachment file for job innerJobA written.
22-08-2022 16:45:00 CST innerFlow INFO - Job innerJobA finished with status SUCCEEDED in 0 seconds
22-08-2022 16:45:00 CST innerFlow INFO - Configuring Azkaban metrics tracking for jobrunner object
22-08-2022 16:45:00 CST innerFlow INFO - Submitting job 'innerJobB' to run.
22-08-2022 16:45:00 CST innerFlow INFO - Created file appender for job innerJobB
22-08-2022 16:45:00 CST innerFlow INFO - Attached file appender for job innerJobB
22-08-2022 16:45:00 CST innerFlow INFO - Configuring Azkaban metrics tracking for jobrunner object
22-08-2022 16:45:00 CST innerFlow INFO - Submitting job 'innerJobC' to run.
22-08-2022 16:45:00 CST innerFlow INFO - Job Started: innerJobB
22-08-2022 16:45:00 CST innerFlow INFO - Created file appender for job innerJobC
22-08-2022 16:45:00 CST innerFlow INFO - Attached file appender for job innerJobC
22-08-2022 16:45:00 CST innerFlow INFO - Job Started: innerJobC
22-08-2022 16:45:00 CST innerFlow INFO - No attachment file for job innerJobB written.
22-08-2022 16:45:00 CST innerFlow INFO - Job innerJobB finished with status SUCCEEDED in 0 seconds
22-08-2022 16:45:00 CST innerFlow INFO - No attachment file for job innerJobC written.
22-08-2022 16:45:00 CST innerFlow INFO - Job innerJobC finished with status SUCCEEDED in 0 seconds
22-08-2022 16:45:00 CST innerFlow INFO - Configuring Azkaban metrics tracking for jobrunner object
22-08-2022 16:45:00 CST innerFlow INFO - Submitting job 'innerFlow' to run.
22-08-2022 16:45:00 CST innerFlow INFO - Created file appender for job innerFlow
22-08-2022 16:45:00 CST innerFlow INFO - Attached file appender for job innerFlow
22-08-2022 16:45:00 CST innerFlow INFO - Job Started: innerFlow
22-08-2022 16:45:00 CST innerFlow INFO - No attachment file for job innerFlow written.
22-08-2022 16:45:00 CST innerFlow INFO - Job innerFlow finished with status SUCCEEDED in 0 seconds
22-08-2022 16:45:00 CST innerFlow INFO - Flow '' is set to SUCCEEDED in 0 seconds
22-08-2022 16:45:00 CST innerFlow INFO - Finishing up flow. Awaiting Termination
22-08-2022 16:45:00 CST innerFlow INFO - Finished Flow
22-08-2022 16:45:00 CST innerFlow INFO - Setting end time for flow 95 to 1661157900207

innerFlow.job

22-08-2022 16:45:00 CST innerFlow INFO - Starting job innerFlow at 1661157900121
22-08-2022 16:45:00 CST innerFlow INFO - job JVM args: -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerFlow
22-08-2022 16:45:00 CST innerFlow INFO - user.to.proxy property was not set, defaulting to submit user azkaban
22-08-2022 16:45:00 CST innerFlow INFO - Building javaprocess job executor. 
22-08-2022 16:45:00 CST innerFlow INFO - Memory granted for job innerFlow
22-08-2022 16:45:00 CST innerFlow INFO - No classpath specified. Trying to load classes from /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerFlow INFO - 1 commands to execute.
22-08-2022 16:45:00 CST innerFlow INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerFlow INFO - effective user is: azkaban
22-08-2022 16:45:00 CST innerFlow INFO - Command: java -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerFlow -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType innerFlow dependencies=innerJobB,innerJobC 
22-08-2022 16:45:00 CST innerFlow INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerFlow_output_7020976528385638433_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerFlow_props_2379673642283223464_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__innerFlow__innerFlow__95__azkaban, JOB_NAME=innerFlow}
22-08-2022 16:45:00 CST innerFlow INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerFlow INFO - this is a java class. java type,package:com.win!  参数1=innerFlow  参数2= dependencies=innerJobB,innerJobC
22-08-2022 16:45:00 CST innerFlow INFO - Process completed successfully in 0 seconds.
22-08-2022 16:45:00 CST innerFlow INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerFlow_output_7020976528385638433_tmp
22-08-2022 16:45:00 CST innerFlow INFO - Finishing job innerFlow at 1661157900196 with status SUCCEEDED

innerJobA.job

22-08-2022 16:44:59 CST innerJobA INFO - Starting job innerJobA at 1661157899947
22-08-2022 16:44:59 CST innerJobA INFO - job JVM args: -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobA
22-08-2022 16:44:59 CST innerJobA INFO - user.to.proxy property was not set, defaulting to submit user azkaban
22-08-2022 16:44:59 CST innerJobA INFO - Building javaprocess job executor. 
22-08-2022 16:44:59 CST innerJobA INFO - Memory granted for job innerJobA
22-08-2022 16:44:59 CST innerJobA INFO - No classpath specified. Trying to load classes from /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:44:59 CST innerJobA INFO - 1 commands to execute.
22-08-2022 16:44:59 CST innerJobA INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:44:59 CST innerJobA INFO - effective user is: azkaban
22-08-2022 16:44:59 CST innerJobA INFO - Command: java -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobA -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType innerJobA nothing 
22-08-2022 16:44:59 CST innerJobA INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobA_output_2048172206987154574_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobA_props_8361574289772420710_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__innerFlow__innerJobA__95__azkaban, JOB_NAME=innerJobA}
22-08-2022 16:44:59 CST innerJobA INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobA INFO - this is a java class. java type,package:com.win!  参数1=innerJobA  参数2= nothing
22-08-2022 16:45:00 CST innerJobA INFO - Process completed successfully in 0 seconds.
22-08-2022 16:45:00 CST innerJobA INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobA_output_2048172206987154574_tmp
22-08-2022 16:45:00 CST innerJobA INFO - Finishing job innerJobA at 1661157900021 with status SUCCEEDED

innerJobB.job

22-08-2022 16:45:00 CST innerJobB INFO - Starting job innerJobB at 1661157900029
22-08-2022 16:45:00 CST innerJobB INFO - job JVM args: -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobB
22-08-2022 16:45:00 CST innerJobB INFO - user.to.proxy property was not set, defaulting to submit user azkaban
22-08-2022 16:45:00 CST innerJobB INFO - Building javaprocess job executor. 
22-08-2022 16:45:00 CST innerJobB INFO - Memory granted for job innerJobB
22-08-2022 16:45:00 CST innerJobB INFO - No classpath specified. Trying to load classes from /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobB INFO - 1 commands to execute.
22-08-2022 16:45:00 CST innerJobB INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobB INFO - effective user is: azkaban
22-08-2022 16:45:00 CST innerJobB INFO - Command: java -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobB -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType innerJobB innerJobA 
22-08-2022 16:45:00 CST innerJobB INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobB_output_1437460109394890875_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobB_props_6395251793150318646_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__innerFlow__innerJobB__95__azkaban, JOB_NAME=innerJobB}
22-08-2022 16:45:00 CST innerJobB INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobB INFO - this is a java class. java type,package:com.win!  参数1=innerJobB  参数2= innerJobA
22-08-2022 16:45:00 CST innerJobB INFO - Process completed successfully in 0 seconds.
22-08-2022 16:45:00 CST innerJobB INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobB_output_1437460109394890875_tmp
22-08-2022 16:45:00 CST innerJobB INFO - Finishing job innerJobB at 1661157900102 with status SUCCEEDED

innerJobC.job

22-08-2022 16:45:00 CST innerJobC INFO - Starting job innerJobC at 1661157900033
22-08-2022 16:45:00 CST innerJobC INFO - job JVM args: -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobC
22-08-2022 16:45:00 CST innerJobC INFO - user.to.proxy property was not set, defaulting to submit user azkaban
22-08-2022 16:45:00 CST innerJobC INFO - Building javaprocess job executor. 
22-08-2022 16:45:00 CST innerJobC INFO - Memory granted for job innerJobC
22-08-2022 16:45:00 CST innerJobC INFO - No classpath specified. Trying to load classes from /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobC INFO - 1 commands to execute.
22-08-2022 16:45:00 CST innerJobC INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobC INFO - effective user is: azkaban
22-08-2022 16:45:00 CST innerJobC INFO - Command: java -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobC -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType innerJobC innerJobA 
22-08-2022 16:45:00 CST innerJobC INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobC_output_3166311652967193814_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobC_props_1741757618178495101_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__innerFlow__innerJobC__95__azkaban, JOB_NAME=innerJobC}
22-08-2022 16:45:00 CST innerJobC INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobC INFO - this is a java class. java type,package:com.win!  参数1=innerJobC  参数2= innerJobA
22-08-2022 16:45:00 CST innerJobC INFO - Process completed successfully in 0 seconds.
22-08-2022 16:45:00 CST innerJobC INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobC_output_3166311652967193814_tmp
22-08-2022 16:45:00 CST innerJobC INFO - Finishing job innerJobC at 1661157900114 with status SUCCEEDED

四、azkaban定时任务调度

配置如下

首页选择待处理的project
设置好执行计划后
执行结果以上完成了azkaban定时任务配置。

活动

应用商城

合作伙伴

开发者

支持与服务

了解天翼云

azkaban-3.51.0 任务类型（Jobtypes，详细介绍command、java2种类型及示例）

azkaban-3.51.0 任务类型（Jobtypes，详细介绍command、java2种类型及示例）

Azkaban 系列文章

一、Azkaban支持的plugin类型

二、command类型

1、 job配置（示例）示例一

2、 job配置（依赖）示例二

3、 job配置（嵌入式）示例三

4、job配置示（hdfs）例四

5、job配置（mapreduce）示例五

6、job配置（hive）示例六

三、java类型

1、 job配置（普通java）示例一

1）、创建job

2）、TestAzkabanType.java

3）、打包

4）、运行

5）、运行日志

2、job配置（依赖）示例三

1）、创建java文件并打包成jar

2）、创建job

3）、打包

4）、运行

5）、运行日志

四、azkaban定时任务调度

相关文章

dolphinSchedule和azkaban两者调度区别

StreamSets简介和入门案例

作者介绍

最新文章

dolphinSchedule和azkaban两者调度区别

热门文章

dolphinSchedule和azkaban两者调度区别

热门标签

相关产品

弹性云主机

天翼云电脑（公众版）

对象存储

云硬盘

随机文章

dolphinSchedule和azkaban两者调度区别