Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Re: YARN hangs while computing pig?


Copy link to this message
-
Re: YARN hangs while computing pig?
Redirecting to Apache pig user list

On Thu, Nov 29, 2012 at 1:01 AM, Johnny Kowalski
<[EMAIL PROTECTED]>wrote:

> Another hint? Is is some permission issue?
>
> MY EXAMPLE:
> in = LOAD '/user/myUser/aTest' USING PigStorage();
> DUMP in;
>
> When I try to run above pig in grunt shell with "pig -x local" it
> successfully prints output.
> *
> And when I do it in mapreduce pig grundshell it hangs... on this line:"
>
> 2012-11-28 15:24:27,709 [main] INFO  org.apache.pig.backend.hadoop.
> *
> ***executionengine.****mapReduceLayer.****MapReduceLauncher - 0% complete"
> *
>
>
> W dniu środa, 28 listopada 2012 15:44:22 UTC+1 użytkownik Johnny Kowalski
> napisał:
>
>> Hi, after configuring whole stack I've got another issue. My pig jobs
>> hangs.
>> I've created completly new user added a dir for him
>> sudo -u hdfs hadoop fs -mkdir /user/newUser
>> sudo -u hdfs hadoop fs -chown newUser:newUser /user/newUser
>>
>> and wanted to run pig that runs fine on another yarn configuration
>> cluster. And got something like this:
>>
>>
>> PIG CONSOLE OUTPUT
>>
>> 2012-11-28 15:24:26,759 [Thread-4] INFO  org.apache.hadoop.mapreduce.**lib.input.FileInputFormat
>> - Total input paths to process : 1
>> 2012-11-28 15:24:26,760 [Thread-4] INFO  org.apache.pig.backend.hadoop.**
>> executionengine.util.**MapRedUtil - Total input paths to process : 1
>> 2012-11-28 15:24:26,783 [Thread-4] INFO  org.apache.pig.backend.hadoop.**
>> executionengine.util.**MapRedUtil - Total input paths (combined) to
>> process : 1
>> 2012-11-28 15:24:26,872 [Thread-4] INFO  org.apache.hadoop.mapreduce.**JobSubmitter
>> - number of splits:1
>> 2012-11-28 15:24:26,889 [Thread-4] WARN  org.apache.hadoop.conf.**Configuration
>> - fs.default.name is deprecated. Instead, use fs.defaultFS
>> 2012-11-28 15:24:26,890 [Thread-4] WARN  org.apache.hadoop.conf.**Configuration
>> - mapreduce.job.counters.limit is deprecated. Instead, use
>> mapreduce.job.counters.max
>> 2012-11-28 15:24:26,891 [Thread-4] WARN  org.apache.hadoop.conf.**Configuration
>> - mapred.job.tracker is deprecated. Instead, use
>> mapreduce.jobtracker.address
>> 2012-11-28 15:24:26,892 [Thread-4] WARN  org.apache.hadoop.conf.**Configuration
>> - dfs.https.address is deprecated. Instead, use dfs.namenode.https-address
>> 2012-11-28 15:24:26,893 [Thread-4] WARN  org.apache.hadoop.conf.**Configuration
>> - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
>> 2012-11-28 15:24:26,894 [Thread-4] WARN  org.apache.hadoop.conf.**Configuration
>> - mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
>> 2012-11-28 15:24:27,563 [Thread-4] INFO  org.apache.hadoop.mapred.**ResourceMgrDelegate
>> - Submitted application application_1354100898821_0011 to ResourceManager
>> at userver/192.168.56.101:8032
>> 2012-11-28 15:24:27,629 [Thread-4] INFO  org.apache.hadoop.mapreduce.**Job
>> - The url to track the job: http://userver:8088/proxy/**
>> application_1354100898821_**0011/<http://userver:8088/proxy/application_1354100898821_0011/>
>> 2012-11-28 15:24:27,709 [main] INFO  org.apache.pig.backend.hadoop.**
>> executionengine.**mapReduceLayer.**MapReduceLauncher - 0% complete
>>
>> Here I've noticed logs keep growing while mine job is in status RUNNING
>>
>> /var/log/hadoop-yarn# less hadoop-cmf-yarn1-NODEMANAGER-**userver.log.out
>>
>> 2012-11-28 15:39:16,227 INFO org.apache.hadoop.yarn.server.**nodemanager.
>> **NodeStatusUpdaterImpl: Sending out status for container: container_id
>> {, app_attempt_id {, application_id {, id: 11, cluster_timestamp:
>> 1354100898821, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics:
>> "", exit_status: -1000,
>> 2012-11-28 15:39:17,230 INFO org.apache.hadoop.yarn.server.**nodemanager.
>> **NodeStatusUpdaterImpl: Sending out status for container: container_id
>> {, app_attempt_id {, application_id {, id: 11, cluster_timestamp:
>> 1354100898821, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics:
>> "", exit_status: -1000,
>> 2012-11-28 15:39:18,233 INFO org.apache.hadoop.yarn.server.**nodemanager.
+
Johnny Kowalski 2012-11-30, 08:43
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB