Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Map Reduce jobs taking a long time at the end


+
Jay Whittaker 2012-12-04, 12:27
+
ac@...) 2012-12-04, 13:59
+
Jay Whittaker 2012-12-05, 15:47
+
in.abdul 2012-12-05, 16:46
Copy link to this message
-
Re: Map Reduce jobs taking a long time at the end
Yeah, it's against a ~95million row table in hbase.

It takes about 30 mins to get to 90% then about 3+ hours to get from 90%
to 100%

On Wed, 2012-12-05 at 08:46 -0800, in.abdul wrote:
> Hi jay..
>   Are you trying to do M-R on HBase Table ?
>
>
> Thanks and regards
> Syed Abdul Kather
>
>
>             Thanks and Regards,
>         S SYED ABDUL KATHER
>
>
>
> On Wed, Dec 5, 2012 at 9:53 PM, Jay Whittaker [via Lucene] <
> ml-node+[EMAIL PROTECTED]> wrote:
>
> > Hey Ac,
> >
> > The logs I copied were from the .out files while a job was running. I
> > thought that would be the best way to get a good idea of what was
> > happening.
> >
> > Cheers,
> >
> > Jay
> >
> > On Tue, 2012-12-04 at 21:59 +0800, [hidden email]<http://user/SendEmail.jtp?type=node&node=4024496&i=0>wrote:
> >
> > > Hi,
> > >
> > > Have you also checked .out file of the tasktracker in logs? It could
> > contain some useful information for the issue.
> > >
> > > Thanks
> > > ac
> > >
> > >
> > > On 4 Dec 2012, at 8:27 PM, Jay Whittaker wrote:
> > >
> > > > Hey,
> > > >
> > > > We are running Map reduce jobs against a 12 machine hbase cluster and
> > > > for a long time they took approx 30 mins to return a result against
> > ~95
> > > > million rows. Without any major changes to the data or any upgrade of
> > > > hbase/hadoop they now seem to be taking about 4 hours. and the logs
> > are
> > > > full of
> > > >
> > > > 2012-12-04 13:33:15,602 INFO org.apache.hadoop.mapred.TaskTracker:
> > > > attempt_201211210952_0293_m_000031_0 0.0% row: 63 6f 6d 2e 70 72 6f 75
> > > > 67 68 74
> > > > ...
> > > > 2012-12-04 13:45:17,134 INFO org.apache.hadoop.mapred.TaskTracker:
> > > > attempt_201211210952_0293_m_000031_0 0.0% row: 63 6f 6d 2e 70 75 72 70
> > > > 6c 65 64 65 73 69 67 6e 73 65 72 76 69 63 65 73
> > > > ...
> > > > 2012-12-04 13:46:11,515 INFO org.apache.hadoop.mapred.TaskTracker:
> > > > attempt_201211210952_0293_m_000031_0 0.0% row: 63 6f 6d 2e 70 75 73 68
> > > > 74 6f 74 61 6c 6b 2d 6f 6e 6c 69 6e 65
> > > >
> > > > I presume the 0% is percent complete but I'm not sure as to why the
> > time
> > > > to complete has now jumped massively. Ganglia shows no major load on
> > the
> > > > nodes in question so I don't think it's that.
> > > >
> > > > What steps should I be taking to try troubleshoot the problem?
> > > >
> > > > Regards,
> > > >
> > > > Jay
> > >
> >
> >
> > ------------------------------
> >  If you reply to this email, your message will be added to the discussion
> > below:
> >
> > http://lucene.472066.n3.nabble.com/Map-Reduce-jobs-taking-a-long-time-at-the-end-tp4024231p4024496.html
> >  To start a new topic under Hadoop lucene-users, email
> > ml-node+[EMAIL PROTECTED]
> > To unsubscribe from Lucene, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=472066&code=aW4uYWJkdWxAZ21haWwuY29tfDQ3MjA2NnwxMDczOTUyNDEw>
> > .
> > NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> >
>
>
>
>
> -----
> THANKS AND REGARDS,
> SYED ABDUL KATHER
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Map-Reduce-jobs-taking-a-long-time-at-the-end-tp4024231p4024515.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB