Skip Ribbon Commands
Skip to main content
Navigate Up
Sign In

Quick Launch

Average Rating:

facebook Twitter
Email
Print Bookmark Alert me when this article is updated

Feedback

HOW TO: Monitor the MDM BDRM "mapReduce" jobs using the command line
Solution
​To monitor the list of mapReduce jobs, run the following command:

[prompt]>mapred job -list
15/12/27 00:59:21 INFO client.RMProxy: Connecting to ResourceManager at <domain>/<hostip>:<port>
Total jobs:1
 JobId      State     StartTime      UserName    Queue      Priority       UsedContainers  RsvdContainers  UsedMem         RsvdMem         NeededMem         AM info
 job_1416639675736_0228       PREP       1417067955247          hdfs       root.hdfs        NORMAL                    1               0   10240M              0M            10240M      <URL>

The above command lists all the MapReduce jobs running along with the JobId. To check the Job Status of individual job, run the following command:
[prompt]>mapred job -status job_1416639675736_0228
15/12/27 01:01:23 INFO client.RMProxy: Connecting to ResourceManager at <domain>/<hostip>:<port> 
Job: job_1416639675736_0228
Job File: hdfs://<domain>.<hostip>:<port>/user/hdfs/.staging/job_1416639675736_0228/job.xml
Job Tracking URL : http://<domain>.<hostip>:<port>/proxy/application_1416639675736_0228/
Uber job : false
Number of maps: 18
Number of reduces: 16
map() completion: 0.8004539
reduce() completion: 0.0
Job state: RUNNING
retired: false
reason for failure:
Counters: 34
        File System Counters
                FILE: Number of bytes read=2441639711
                FILE: Number of bytes written=6696060596
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=1000278362
                HDFS: Number of bytes written=0
                HDFS: Number of read operations=49
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=3
        Job Counters
                Launched map tasks=21
                Data-local map tasks=21
                Total time spent by all maps in occupied slots (ms)=1154251
                Total time spent by all map tasks (ms)=1154251
                Total vcore-seconds taken by all map tasks=1154251
                Total megabyte-seconds taken by all map tasks=4727812096
        Map-Reduce Framework
                Map input records=8539745
                Map output records=223023876
                Map output bytes=31000318780
                Map output materialized bytes=2428809344
                Input split bytes=2394
                Combine input records=0
                Spilled Records=278058986
                Failed Shuffles=0
                Merged Map outputs=0
                GC time elapsed (ms)=9377
                CPU time spent (ms)=1927670
                Physical memory (bytes) snapshot=36778799104
                Virtual memory (bytes) snapshot=73599758336
                Total committed heap usage (bytes)=41813016576
        com.informatica.mdmbde.group.MDMBDEGroupMapper$GroupCounter
                NULLKEY_SKIP=4
                NULLRANGE_SKIP=4
        com.informatica.mdmbde.utils.MDMBDEConstants$MATCH_COUNTER
                NUM_DATA_PARTITIONS=18
        File Input Format Counters
                Bytes Read=1000275968 
​​​
More Information
Reference
Applies To
Product: Relate 360
Problem Type:
User Type:
Project Phase:
Product Version:
Database:
Operating System:
Other Software:
Attachments
Last Modified Date:7/24/2018 9:53 PMID:438804
People who viewed this also viewed

Feedback

Did this KB document help you?



What can we do to improve this information (2000 or fewer characters)