Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - hive mapred problem


Copy link to this message
-
Re: hive mapred problem
Nitin Pawar 2012-10-08, 12:21
from the error looks like you have some incorrect hive settings which
are failing the job initialization.

this is the error
>java.io.IOException: Number of maps in JobConf doesn't match number of
> recieved splits for job job_201210051717_0015! numMapTasks=10

can you tell us if you are setting in hive variables before firing up
the query?  something like split size or # no maps etc

On Mon, Oct 8, 2012 at 5:30 PM, Ajit Kumar Shreevastava
<[EMAIL PROTECTED]> wrote:
> Hi Nitin,
>
>
>
> Job tracker log:-->
>
>
>
> 2012-10-08 15:43:29,331 WARN org.apache.hadoop.conf.Configuration:
> /usr/local/hadoopstorage/mapred/local/jobTracker/job_201210051717_0015.xml:a
> attempt to override fina
>
> l parameter: mapred.reduce.tasks;  Ignoring.
>
> 2012-10-08 15:43:29,331 WARN org.apache.hadoop.conf.Configuration:
> /usr/local/hadoopstorage/mapred/local/jobTracker/job_201210051717_0015.xml:a
> attempt to override fina
>
> l parameter: mapred.map.tasks;  Ignoring.
>
> 2012-10-08 15:43:29,332 INFO org.apache.hadoop.mapred.JobInProgress:
> job_201210051717_0015: nMaps=10 nReduces=2 max=-1
>
> 2012-10-08 15:43:29,332 INFO org.apache.hadoop.mapred.JobTracker: Job
> job_201210051717_0015 added successfully for user 'hadoop' to queue
> 'default'
>
> 2012-10-08 15:43:29,332 INFO org.apache.hadoop.mapred.AuditLogger:
> USER=hadoop  IP=10.99.42.9   OPERATION=SUBMIT_JOB
> TARGET=job_201210051717_0015    RESULT=SUCCESS
>
> 2012-10-08 15:43:29,332 INFO org.apache.hadoop.mapred.JobTracker:
> Initializing job_201210051717_0015
>
> 2012-10-08 15:43:29,332 INFO org.apache.hadoop.mapred.JobInProgress:
> Initializing job_201210051717_0015
>
> 2012-10-08 15:43:29,350 INFO org.apache.hadoop.mapred.JobInProgress:
> jobToken generated and stored with users keys in
> /usr/local/hadoopstorage/mapred/system/job_2012100
>
> 51717_0015/jobToken
>
> 2012-10-08 15:43:29,352 ERROR org.apache.hadoop.mapred.JobTracker: Job
> initialization failed:
>
> java.io.IOException: Number of maps in JobConf doesn't match number of
> recieved splits for job job_201210051717_0015! numMapTasks=10, #splits=1
>
>         at
> org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:703)
>
>         at org.apache.hadoop.mapred.JobTracker.initJob(JobTracker.java:4207)
>
>         at
> org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:79)
>
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>         at java.lang.Thread.run(Thread.java:662)
>
>
>
> 2012-10-08 15:43:29,352 INFO org.apache.hadoop.mapred.JobTracker: Failing
> job job_201210051717_0015
>
> 2012-10-08 15:43:29,352 INFO
> org.apache.hadoop.mapred.JobInProgress$JobSummary:
> jobId=job_201210051717_0015,submitTime=1349691209316,launchTime=0,,finishTime=1349691209
>
> 352,numMaps=0,numSlotsPerMap=1,numReduces=0,numSlotsPerReduce=1,user=hadoop,queue=default,status=FAILED,mapSlotSeconds=0,reduceSlotsSeconds=0,clusterMapCapacity=14,clus
>
> terReduceCapacity=14,jobName=select count(1) from pokes(Stage-1)
>
> 2012-10-08 15:43:29,353 INFO org.apache.hadoop.mapred.JobHistory: Moving
> file:/var/log/hadoop/history/job_201210051717_0015_1349691209316_hadoop_select+count%281%29+fro
>
> m+pokes%28Stage-1%29 to
> file:/var/log/hadoop/history/done/version-1/NHCLT-PC44-2_1349437647390_/2012/10/08/000000
>
> 2012-10-08 15:43:29,356 INFO org.apache.hadoop.mapred.JobHistory: Moving
> file:/var/log/hadoop/history/job_201210051717_0015_conf.xml to
> file:/var/log/hadoop/history/don
>
> e/version-1/NHCLT-PC44-2_1349437647390_/2012/10/08/000000
>
> 2012-10-08 15:43:30,362 INFO org.apache.hadoop.mapred.JobTracker: Killing
> job job_201210051717_0015
>
> 2012-10-08 17:17:35,641 INFO
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
> Updating the current master key for generating delegation
>
> tokens
>

Nitin Pawar