Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> hive mapred problem


+
Ajit Kumar Shreevastava 2012-10-08, 10:43
+
Nitin Pawar 2012-10-08, 10:49
+
Ajit Kumar Shreevastava 2012-10-08, 12:00
Copy link to this message
-
Re: hive mapred problem
from the error looks like you have some incorrect hive settings which
are failing the job initialization.

this is the error
>java.io.IOException: Number of maps in JobConf doesn't match number of
> recieved splits for job job_201210051717_0015! numMapTasks=10

can you tell us if you are setting in hive variables before firing up
the query?  something like split size or # no maps etc

On Mon, Oct 8, 2012 at 5:30 PM, Ajit Kumar Shreevastava
<[EMAIL PROTECTED]> wrote:
> Hi Nitin,
>
>
>
> Job tracker log:-->
>
>
>
> 2012-10-08 15:43:29,331 WARN org.apache.hadoop.conf.Configuration:
> /usr/local/hadoopstorage/mapred/local/jobTracker/job_201210051717_0015.xml:a
> attempt to override fina
>
> l parameter: mapred.reduce.tasks;  Ignoring.
>
> 2012-10-08 15:43:29,331 WARN org.apache.hadoop.conf.Configuration:
> /usr/local/hadoopstorage/mapred/local/jobTracker/job_201210051717_0015.xml:a
> attempt to override fina
>
> l parameter: mapred.map.tasks;  Ignoring.
>
> 2012-10-08 15:43:29,332 INFO org.apache.hadoop.mapred.JobInProgress:
> job_201210051717_0015: nMaps=10 nReduces=2 max=-1
>
> 2012-10-08 15:43:29,332 INFO org.apache.hadoop.mapred.JobTracker: Job
> job_201210051717_0015 added successfully for user 'hadoop' to queue
> 'default'
>
> 2012-10-08 15:43:29,332 INFO org.apache.hadoop.mapred.AuditLogger:
> USER=hadoop  IP=10.99.42.9   OPERATION=SUBMIT_JOB
> TARGET=job_201210051717_0015    RESULT=SUCCESS
>
> 2012-10-08 15:43:29,332 INFO org.apache.hadoop.mapred.JobTracker:
> Initializing job_201210051717_0015
>
> 2012-10-08 15:43:29,332 INFO org.apache.hadoop.mapred.JobInProgress:
> Initializing job_201210051717_0015
>
> 2012-10-08 15:43:29,350 INFO org.apache.hadoop.mapred.JobInProgress:
> jobToken generated and stored with users keys in
> /usr/local/hadoopstorage/mapred/system/job_2012100
>
> 51717_0015/jobToken
>
> 2012-10-08 15:43:29,352 ERROR org.apache.hadoop.mapred.JobTracker: Job
> initialization failed:
>
> java.io.IOException: Number of maps in JobConf doesn't match number of
> recieved splits for job job_201210051717_0015! numMapTasks=10, #splits=1
>
>         at
> org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:703)
>
>         at org.apache.hadoop.mapred.JobTracker.initJob(JobTracker.java:4207)
>
>         at
> org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:79)
>
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>         at java.lang.Thread.run(Thread.java:662)
>
>
>
> 2012-10-08 15:43:29,352 INFO org.apache.hadoop.mapred.JobTracker: Failing
> job job_201210051717_0015
>
> 2012-10-08 15:43:29,352 INFO
> org.apache.hadoop.mapred.JobInProgress$JobSummary:
> jobId=job_201210051717_0015,submitTime=1349691209316,launchTime=0,,finishTime=1349691209
>
> 352,numMaps=0,numSlotsPerMap=1,numReduces=0,numSlotsPerReduce=1,user=hadoop,queue=default,status=FAILED,mapSlotSeconds=0,reduceSlotsSeconds=0,clusterMapCapacity=14,clus
>
> terReduceCapacity=14,jobName=select count(1) from pokes(Stage-1)
>
> 2012-10-08 15:43:29,353 INFO org.apache.hadoop.mapred.JobHistory: Moving
> file:/var/log/hadoop/history/job_201210051717_0015_1349691209316_hadoop_select+count%281%29+fro
>
> m+pokes%28Stage-1%29 to
> file:/var/log/hadoop/history/done/version-1/NHCLT-PC44-2_1349437647390_/2012/10/08/000000
>
> 2012-10-08 15:43:29,356 INFO org.apache.hadoop.mapred.JobHistory: Moving
> file:/var/log/hadoop/history/job_201210051717_0015_conf.xml to
> file:/var/log/hadoop/history/don
>
> e/version-1/NHCLT-PC44-2_1349437647390_/2012/10/08/000000
>
> 2012-10-08 15:43:30,362 INFO org.apache.hadoop.mapred.JobTracker: Killing
> job job_201210051717_0015
>
> 2012-10-08 17:17:35,641 INFO
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
> Updating the current master key for generating delegation
>
> tokens
>

Nitin Pawar
+
Ajit Kumar Shreevastava 2012-10-09, 09:12
+
Nitin Pawar 2012-10-09, 09:15
+
Ajit Kumar Shreevastava 2012-10-09, 10:24
+
Nitin Pawar 2012-10-09, 10:32
+
Ajit Kumar Shreevastava 2012-10-09, 11:29
+
Bejoy KS 2012-10-09, 12:27
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB