Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> get job data in command line in MRv2


Copy link to this message
-
RE: get job data in command line in MRv2
Perhaps some of the data you require can be obtained with the mapred tool
(here I’m using 2.2.0)

 

e.g.

 

 

htf@ivygerman:~/hadoop/bin$ ./mapred job

Usage: CLI <command> <args>

        [-submit <job-file>]

        [-status <job-id>]

        [-counter <job-id> <group-name> <counter-name>]

        [-kill <job-id>]

        [-set-priority <job-id> <priority>]. Valid values for priorities
are: VERY_HIGH HIGH NORMAL LOW VERY_LOW

        [-events <job-id> <from-event-#> <#-of-events>]

        [-history <jobHistoryFile>]

        [-list [all]]

        [-list-active-trackers]

        [-list-blacklisted-trackers]

        [-list-attempt-ids <job-id> <task-type> <task-state>]. Valid values
for <task-type> are REDUCE MAP. Valid values for <task-state> are running,
completed

        [-kill-task <task-attempt-id>]

        [-fail-task <task-attempt-id>]

        [-logs <job-id> <task-attempt-id>]

 

Generic options supported are

-conf <configuration file>     specify an application configuration file

-D <property=value>            use value for given property

-fs <local|namenode:port>      specify a namenode

-jt <local|jobtracker:port>    specify a job tracker

-files <comma separated list of files>    specify comma separated files to
be copied to the map reduce cluster

-libjars <comma separated list of jars>    specify comma separated jar files
to include in the classpath.

-archives <comma separated list of archives>    specify comma separated
archives to be unarchived on the compute machines.

 

The general command line syntax is

bin/hadoop command [genericOptions] [commandOptions]

 

htf@ivygerman:~/hadoop/bin$ ./mapred job -list

14/01/02 11:29:54 INFO client.RMProxy: Connecting to ResourceManager at
ivy02.spa.sarc.sas/172.31.204.11:8032

Total jobs:1

                  JobId      State           StartTime      UserName
Queue      Priority       UsedContainers  RsvdContainers  UsedMem
RsvdMem         NeededMem         AM info

job_1388162829887_0050    RUNNING       1388678456812           htf
root.htf        NORMAL                  300              20  307712M
20480M           328192M
ivygerman-192-168-10-12.htfcluster.org:8088/proxy/application_1388162829887_
0050/

htf@ivygerman-192-168-10-12:~/hadoop/bin$

 

 

 

From: Azuryy Yu [mailto:[EMAIL PROTECTED]]
Sent: Monday, December 30, 2013 10:42 PM
To: [EMAIL PROTECTED]
Subject: Re: get job data in command line in MRv2

 

Generally, MRv2 indicates Yarn. you can try:

 

yarn application

 

then there are full help lists.

 

On Tue, Dec 31, 2013 at 12:32 PM, 小网客 <[EMAIL PROTECTED]>
wrote:

ui or hadoop job command like:hadoop job -list

 

------------------

---------------------------------------------

BestWishes!

小网客

Blog: <http://snv.iteye.com/> http://snv.iteye.com/

Email:[EMAIL PROTECTED] <mailto:Email%[EMAIL PROTECTED]>

 

 

 

------------------ Original ------------------

From:  "xeon";<[EMAIL PROTECTED]>;

Date:  Tue, Dec 31, 2013 03:19 AM

To:  "user"<[EMAIL PROTECTED]>;

Subject:  get job data in command line in MRv2

 

Hi,

I would like to know if the MRv2 provide the following commands through
the bash command line:
- get the number of jobs running?
- get the percentage of job completion of jobs?
- get the number of jobs that are waiting to be submitted?

--
Thanks,

.

 

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB