searchusermenu
  • 发布文章
  • 消息中心
点赞
收藏
评论
分享
原创

yarn常用命令

2024-05-30 08:45:32
15
0

1、 yarn application查看任务

1列出所有Application

[hadoop102 hadoop-3.1.3]$ yarn application -list

2024-02-06 10:21:19,238 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):0

                Application-Id     Application-Name     Application-Type       User      Queue              State        Final-State        Progress                        Tracking-URL

2根据Application状态过滤yarn application -list -appStates (所有状态:ALLNEWNEW_SAVINGSUBMITTEDACCEPTEDRUNNINGFINISHEDFAILEDKILLED

[hadoop102 hadoop-3.1.3]$ yarn application -list -appStates FINISHED

2024-02-06 10:22:20,029 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Total number of applications (application-types: [], states: [FINISHED] and tags: []):1

                Application-Id     Application-Name     Application-Type       User      Queue              State        Final-State        Progress                        Tracking-URL

application_1612577921195_0001           word count            MAPREDUCE    root    default           FINISHED          SUCCEEDED            100% hadoop102:19888/jobhistory/job/job_1612577921195_0001

3KillApplication

[hadoop102 hadoop-3.1.3]$ yarn application -kill application_1612577921195_0001

2024-02-06 10:23:48,530 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Application application_1612577921195_0001 has already finished

 

2、 yarn logs查看日志

1查询Application日志yarn logs -applicationId <ApplicationId>

[hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1612577921195_0001

2查询Container日志yarn logs -applicationId <ApplicationId> -containerId <ContainerId>

[hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1612577921195_0001 -containerId container_1612577921195_0001_01_000001

 

3、 yarn applicationattempt查看尝试运行的任务

1列出所有Application尝试的列表yarn applicationattempt -list <ApplicationId>

[hadoop102 hadoop-3.1.3]$ yarn applicationattempt -list application_1612577921195_0001

2024-02-06 10:26:54,195 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Total number of application attempts :1

         ApplicationAttempt-Id                State                     AM-Container-Id                        Tracking-URL

appattempt_1612577921195_0001_000001             FINISHED container_1612577921195_0001_01_000001hadoop103:8088/proxy/application_1612577921195_0001/

2打印ApplicationAttemp状态yarn applicationattempt -status <ApplicationAttemptId>

[hadoop102 hadoop-3.1.3]$ yarn applicationattempt -status appattempt_1612577921195_0001_000001

2024-02-06 10:27:55,896 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Application Attempt Report :

ApplicationAttempt-Id : appattempt_1612577921195_0001_000001

State : FINISHED

AMContainer : container_1612577921195_0001_01_000001

Tracking-URL : hadoop103:8088/proxy/application_1612577921195_0001/

RPC Port : 34756

AM Host : hadoop104

Diagnostics 

 

4、 yarn container查看容器

1列出所有Containeryarn container -list <ApplicationAttemptId>

[hadoop102 hadoop-3.1.3]$ yarn container -list appattempt_1612577921195_0001_000001

2024-02-06 10:28:41,396 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Total number of containers :0

                  Container-Id           Start Time          Finish Time                State                 Host    Node Address

2打印Container状态 yarn container -status <ContainerId>

[hadoop102 hadoop-3.1.3]$ yarn container -status container_1612577921195_0001_01_000001

2021-02-06 10:29:58,554 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Container with id 'container_1612577921195_0001_01_000001' doesn't exist in RM or Timeline Server.

    注:只有在任务跑的途中才能看到container的状态

 

5、 yarn node查看节点状态

列出所有节点yarn node -list -all

[hadoop102 hadoop-3.1.3]$ yarn node -list -all

2024-02-06 10:31:36,962 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Total Nodes:3

         Node-Id      Node-State Node-Address Number-of-Running-Containers

 hadoop103:38168         RUNNING    hadoop103:8042                            0

 hadoop102:42012         RUNNING    hadoop102:8042                            0

 hadoop104:39702         RUNNING    hadoop104:8042                            0

6、 yarn rmadmin更新配置

加载队列配置yarn rmadmin -refreshQueues

[hadoop102 hadoop-3.1.3]$ yarn rmadmin -refreshQueues

2024-02-06 10:32:03,331 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8033

7、 yarn queue查看队列

打印队列信息yarn queue -status <QueueName>

[hadoop102 hadoop-3.1.3]$ yarn queue -status default

2024-02-06 10:32:33,403 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Queue Information :

Queue Name : default

State : RUNNING

Capacity : 100.0%

Current Capacity : .0%

Maximum Capacity : 100.0%

Default Node Label expression : <DEFAULT_PARTITION>

Accessible Node Labels : *

Preemption : disabled

Intra-queue Preemption : disabled

0条评论
0 / 1000
唐****汉
2文章数
0粉丝数
唐****汉
2 文章 | 0 粉丝
唐****汉
2文章数
0粉丝数
唐****汉
2 文章 | 0 粉丝
原创

yarn常用命令

2024-05-30 08:45:32
15
0

1、 yarn application查看任务

1列出所有Application

[hadoop102 hadoop-3.1.3]$ yarn application -list

2024-02-06 10:21:19,238 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):0

                Application-Id     Application-Name     Application-Type       User      Queue              State        Final-State        Progress                        Tracking-URL

2根据Application状态过滤yarn application -list -appStates (所有状态:ALLNEWNEW_SAVINGSUBMITTEDACCEPTEDRUNNINGFINISHEDFAILEDKILLED

[hadoop102 hadoop-3.1.3]$ yarn application -list -appStates FINISHED

2024-02-06 10:22:20,029 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Total number of applications (application-types: [], states: [FINISHED] and tags: []):1

                Application-Id     Application-Name     Application-Type       User      Queue              State        Final-State        Progress                        Tracking-URL

application_1612577921195_0001           word count            MAPREDUCE    root    default           FINISHED          SUCCEEDED            100% hadoop102:19888/jobhistory/job/job_1612577921195_0001

3KillApplication

[hadoop102 hadoop-3.1.3]$ yarn application -kill application_1612577921195_0001

2024-02-06 10:23:48,530 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Application application_1612577921195_0001 has already finished

 

2、 yarn logs查看日志

1查询Application日志yarn logs -applicationId <ApplicationId>

[hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1612577921195_0001

2查询Container日志yarn logs -applicationId <ApplicationId> -containerId <ContainerId>

[hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1612577921195_0001 -containerId container_1612577921195_0001_01_000001

 

3、 yarn applicationattempt查看尝试运行的任务

1列出所有Application尝试的列表yarn applicationattempt -list <ApplicationId>

[hadoop102 hadoop-3.1.3]$ yarn applicationattempt -list application_1612577921195_0001

2024-02-06 10:26:54,195 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Total number of application attempts :1

         ApplicationAttempt-Id                State                     AM-Container-Id                        Tracking-URL

appattempt_1612577921195_0001_000001             FINISHED container_1612577921195_0001_01_000001hadoop103:8088/proxy/application_1612577921195_0001/

2打印ApplicationAttemp状态yarn applicationattempt -status <ApplicationAttemptId>

[hadoop102 hadoop-3.1.3]$ yarn applicationattempt -status appattempt_1612577921195_0001_000001

2024-02-06 10:27:55,896 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Application Attempt Report :

ApplicationAttempt-Id : appattempt_1612577921195_0001_000001

State : FINISHED

AMContainer : container_1612577921195_0001_01_000001

Tracking-URL : hadoop103:8088/proxy/application_1612577921195_0001/

RPC Port : 34756

AM Host : hadoop104

Diagnostics 

 

4、 yarn container查看容器

1列出所有Containeryarn container -list <ApplicationAttemptId>

[hadoop102 hadoop-3.1.3]$ yarn container -list appattempt_1612577921195_0001_000001

2024-02-06 10:28:41,396 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Total number of containers :0

                  Container-Id           Start Time          Finish Time                State                 Host    Node Address

2打印Container状态 yarn container -status <ContainerId>

[hadoop102 hadoop-3.1.3]$ yarn container -status container_1612577921195_0001_01_000001

2021-02-06 10:29:58,554 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Container with id 'container_1612577921195_0001_01_000001' doesn't exist in RM or Timeline Server.

    注:只有在任务跑的途中才能看到container的状态

 

5、 yarn node查看节点状态

列出所有节点yarn node -list -all

[hadoop102 hadoop-3.1.3]$ yarn node -list -all

2024-02-06 10:31:36,962 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Total Nodes:3

         Node-Id      Node-State Node-Address Number-of-Running-Containers

 hadoop103:38168         RUNNING    hadoop103:8042                            0

 hadoop102:42012         RUNNING    hadoop102:8042                            0

 hadoop104:39702         RUNNING    hadoop104:8042                            0

6、 yarn rmadmin更新配置

加载队列配置yarn rmadmin -refreshQueues

[hadoop102 hadoop-3.1.3]$ yarn rmadmin -refreshQueues

2024-02-06 10:32:03,331 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8033

7、 yarn queue查看队列

打印队列信息yarn queue -status <QueueName>

[hadoop102 hadoop-3.1.3]$ yarn queue -status default

2024-02-06 10:32:33,403 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032

Queue Information :

Queue Name : default

State : RUNNING

Capacity : 100.0%

Current Capacity : .0%

Maximum Capacity : 100.0%

Default Node Label expression : <DEFAULT_PARTITION>

Accessible Node Labels : *

Preemption : disabled

Intra-queue Preemption : disabled

文章来自个人专栏
大数据-数据血缘
2 文章 | 1 订阅
0条评论
0 / 1000
请输入你的评论
0
0