19/03/18 04:42:53 INFO client.RMProxy: Connecting to ResourceManager at cluster-55f7-m/10.138.0.2:8032 19/03/18 04:42:54 INFO client.AHSProxy: Connecting to Application History server at cluster-55f7-m/10.138.0.2:10200 19/03/18 04:42:54 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 19/03/18 04:42:55 INFO input.FileInputFormat: Total input files to process : 74 19/03/18 04:42:55 INFO mapreduce.JobSubmitter: number of splits:74 19/03/18 04:42:55 INFO Configuration.deprecation: yarn.resourcemanager.system-metrics-publisher.enabled is deprecated. Instead, use yarn.system-metrics-publisher.enabled 19/03/18 04:42:56 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1552881060889_0004 19/03/18 04:42:56 INFO impl.YarnClientImpl: Submitted application application_1552881060889_0004 19/03/18 04:42:56 INFO mapreduce.Job: The url to track the job: http://cluster-55f7-m:8088/proxy/application_1552881060889_0004/ 19/03/18 04:42:56 INFO mapreduce.Job: Running job: job_1552881060889_0004 19/03/18 04:43:03 INFO mapreduce.Job: Job job_1552881060889_0004 running in uber mode : false 19/03/18 04:43:03 INFO mapreduce.Job: map 0% reduce 0% 19/03/18 04:43:14 INFO mapreduce.Job: map 1% reduce 0% 19/03/18 04:43:16 INFO mapreduce.Job: map 3% reduce 0% 19/03/18 04:43:17 INFO mapreduce.Job: map 5% reduce 0% 19/03/18 04:43:18 INFO mapreduce.Job: map 8% reduce 0% 19/03/18 04:43:19 INFO mapreduce.Job: map 9% reduce 0% 19/03/18 04:43:20 INFO mapreduce.Job: map 12% reduce 0% 19/03/18 04:43:22 INFO mapreduce.Job: map 13% reduce 0% 19/03/18 04:43:23 INFO mapreduce.Job: map 14% reduce 0% 19/03/18 04:43:25 INFO mapreduce.Job: map 15% reduce 0% 19/03/18 04:43:28 INFO mapreduce.Job: map 16% reduce 0% 19/03/18 04:43:30 INFO mapreduce.Job: map 18% reduce 0% 19/03/18 04:43:31 INFO mapreduce.Job: map 20% reduce 0% 19/03/18 04:43:32 INFO mapreduce.Job: map 22% reduce 0% 19/03/18 04:43:33 INFO mapreduce.Job: map 24% reduce 0% 19/03/18 04:43:35 INFO mapreduce.Job: map 25% reduce 0% 19/03/18 04:43:36 INFO mapreduce.Job: map 27% reduce 0% 19/03/18 04:43:37 INFO mapreduce.Job: map 28% reduce 0% 19/03/18 04:43:39 INFO mapreduce.Job: map 29% reduce 0% 19/03/18 04:43:40 INFO mapreduce.Job: map 31% reduce 0% 19/03/18 04:43:42 INFO mapreduce.Job: map 32% reduce 0% 19/03/18 04:43:43 INFO mapreduce.Job: map 35% reduce 0% 19/03/18 04:43:45 INFO mapreduce.Job: map 37% reduce 0% 19/03/18 04:43:48 INFO mapreduce.Job: map 41% reduce 0% 19/03/18 04:43:50 INFO mapreduce.Job: map 42% reduce 0% 19/03/18 04:43:52 INFO mapreduce.Job: map 43% reduce 0% 19/03/18 04:43:54 INFO mapreduce.Job: map 45% reduce 0% 19/03/18 04:43:55 INFO mapreduce.Job: map 46% reduce 0% 19/03/18 04:43:57 INFO mapreduce.Job: map 50% reduce 0% 19/03/18 04:44:01 INFO mapreduce.Job: map 53% reduce 0% 19/03/18 04:44:03 INFO mapreduce.Job: map 54% reduce 0% 19/03/18 04:44:05 INFO mapreduce.Job: map 55% reduce 0% 19/03/18 04:44:06 INFO mapreduce.Job: map 57% reduce 0% 19/03/18 04:44:07 INFO mapreduce.Job: map 58% reduce 0% 19/03/18 04:44:08 INFO mapreduce.Job: map 61% reduce 0% 19/03/18 04:44:14 INFO mapreduce.Job: map 64% reduce 0% 19/03/18 04:44:16 INFO mapreduce.Job: map 67% reduce 0% 19/03/18 04:44:17 INFO mapreduce.Job: map 69% reduce 0% 19/03/18 04:44:18 INFO mapreduce.Job: map 74% reduce 0% 19/03/18 04:44:19 INFO mapreduce.Job: map 76% reduce 0% 19/03/18 04:44:28 INFO mapreduce.Job: map 78% reduce 0% 19/03/18 04:44:29 INFO mapreduce.Job: map 82% reduce 0% 19/03/18 04:44:31 INFO mapreduce.Job: map 84% reduce 0% 19/03/18 04:44:32 INFO mapreduce.Job: map 86% reduce 0% 19/03/18 04:44:33 INFO mapreduce.Job: map 88% reduce 0% 19/03/18 04:44:34 INFO mapreduce.Job: map 89% reduce 0% 19/03/18 04:44:36 INFO mapreduce.Job: map 90% reduce 0% 19/03/18 04:44:38 INFO mapreduce.Job: map 93% reduce 0% 19/03/18 04:44:39 INFO mapreduce.Job: map 95% reduce 0% 19/03/18 04:44:41 INFO mapreduce.Job: map 96% reduce 0% 19/03/18 04:44:43 INFO mapreduce.Job: map 100% reduce 0% 19/03/18 04:44:57 INFO mapreduce.Job: map 100% reduce 9% 19/03/18 04:44:59 INFO mapreduce.Job: map 100% reduce 18% 19/03/18 04:45:00 INFO mapreduce.Job: map 100% reduce 27% 19/03/18 04:45:01 INFO mapreduce.Job: map 100% reduce 53% 19/03/18 04:45:03 INFO mapreduce.Job: map 100% reduce 71% 19/03/18 04:45:04 INFO mapreduce.Job: map 100% reduce 100% 19/03/18 04:45:06 INFO mapreduce.Job: Job job_1552881060889_0004 completed successfully 19/03/18 04:45:06 INFO mapreduce.Job: Counters: 56 File System Counters FILE: Number of bytes read=859401679 FILE: Number of bytes written=1736577306 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 GS: Number of bytes read=3358558058 GS: Number of bytes written=20481506 GS: Number of read operations=0 GS: Number of large read operations=0 GS: Number of write operations=0 HDFS: Number of bytes read=10508 HDFS: Number of bytes written=0 HDFS: Number of read operations=74 HDFS: Number of large read operations=0 HDFS: Number of write operations=0 Job Counters Killed map tasks=1 Killed reduce tasks=1 Launched map tasks=75 Launched reduce tasks=12 Rack-local map tasks=75 Total time spent by all maps in occupied slots (ms)=2849532 Total time spent by all reduces in occupied slots (ms)=605559 Total time spent by all map tasks (ms)=949844 Total time spent by all reduce tasks (ms)=201853 Total vcore-milliseconds taken by all map tasks=949844 Total vcore-milliseconds taken by all reduce tasks=201853 Total megabyte-milliseconds taken by all map tasks=2917920768 Total megabyte-milliseconds taken by all reduce tasks=620092416 Map-Reduce Framework Map input records=74 Map output records=45368000 Map output bytes=768665613 Map output materialized bytes=859406497 Input split bytes=10508 Combine input records=0 Combine output records=0 Reduce input groups=270330 Reduce shuffle bytes=859406497 Reduce input records=45368000 Reduce output records=270330 Spilled Records=90736000 Shuffled Maps =814 Failed Shuffles=0 Merged Map outputs=814 GC time elapsed (ms)=64326 CPU time spent (ms)=576270 Physical memory (bytes) snapshot=80650977280 Virtual memory (bytes) snapshot=374682116096 Total committed heap usage (bytes)=85880995840 Shuffle Errors BAD_ID=0 CONNECTION=0 IO_ERROR=0 WRONG_LENGTH=0 WRONG_MAP=0 WRONG_REDUCE=0 File Input Format Counters Bytes Read=3358558058 File Output Format Counters Bytes Written=20481506