Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Map Reduce jobs taking a long time at the end


Copy link to this message
-
Re: Map Reduce jobs taking a long time at the end
Hi jay..
  Are you trying to do M-R on HBase Table ?
Thanks and regards
Syed Abdul Kather
            Thanks and Regards,
        S SYED ABDUL KATHER

On Wed, Dec 5, 2012 at 9:53 PM, Jay Whittaker [via Lucene] <
ml-node+[EMAIL PROTECTED]> wrote:

> Hey Ac,
>
> The logs I copied were from the .out files while a job was running. I
> thought that would be the best way to get a good idea of what was
> happening.
>
> Cheers,
>
> Jay
>
> On Tue, 2012-12-04 at 21:59 +0800, [hidden email]<http://user/SendEmail.jtp?type=node&node=4024496&i=0>wrote:
>
> > Hi,
> >
> > Have you also checked .out file of the tasktracker in logs? It could
> contain some useful information for the issue.
> >
> > Thanks
> > ac
> >
> >
> > On 4 Dec 2012, at 8:27 PM, Jay Whittaker wrote:
> >
> > > Hey,
> > >
> > > We are running Map reduce jobs against a 12 machine hbase cluster and
> > > for a long time they took approx 30 mins to return a result against
> ~95
> > > million rows. Without any major changes to the data or any upgrade of
> > > hbase/hadoop they now seem to be taking about 4 hours. and the logs
> are
> > > full of
> > >
> > > 2012-12-04 13:33:15,602 INFO org.apache.hadoop.mapred.TaskTracker:
> > > attempt_201211210952_0293_m_000031_0 0.0% row: 63 6f 6d 2e 70 72 6f 75
> > > 67 68 74
> > > ...
> > > 2012-12-04 13:45:17,134 INFO org.apache.hadoop.mapred.TaskTracker:
> > > attempt_201211210952_0293_m_000031_0 0.0% row: 63 6f 6d 2e 70 75 72 70
> > > 6c 65 64 65 73 69 67 6e 73 65 72 76 69 63 65 73
> > > ...
> > > 2012-12-04 13:46:11,515 INFO org.apache.hadoop.mapred.TaskTracker:
> > > attempt_201211210952_0293_m_000031_0 0.0% row: 63 6f 6d 2e 70 75 73 68
> > > 74 6f 74 61 6c 6b 2d 6f 6e 6c 69 6e 65
> > >
> > > I presume the 0% is percent complete but I'm not sure as to why the
> time
> > > to complete has now jumped massively. Ganglia shows no major load on
> the
> > > nodes in question so I don't think it's that.
> > >
> > > What steps should I be taking to try troubleshoot the problem?
> > >
> > > Regards,
> > >
> > > Jay
> >
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/Map-Reduce-jobs-taking-a-long-time-at-the-end-tp4024231p4024496.html
>  To start a new topic under Hadoop lucene-users, email
> ml-node+[EMAIL PROTECTED]
> To unsubscribe from Lucene, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=472066&code=aW4uYWJkdWxAZ21haWwuY29tfDQ3MjA2NnwxMDczOTUyNDEw>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
-----
THANKS AND REGARDS,
SYED ABDUL KATHER
--
View this message in context: http://lucene.472066.n3.nabble.com/Map-Reduce-jobs-taking-a-long-time-at-the-end-tp4024231p4024515.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB