Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Map Reduce jobs taking a long time at the end


+
Jay Whittaker 2012-12-04, 12:27
+
ac@...) 2012-12-04, 13:59
+
Jay Whittaker 2012-12-05, 15:47
Copy link to this message
-
Re: Map Reduce jobs taking a long time at the end
in.abdul 2012-12-05, 16:46
Hi jay..
  Are you trying to do M-R on HBase Table ?
Thanks and regards
Syed Abdul Kather
            Thanks and Regards,
        S SYED ABDUL KATHER

On Wed, Dec 5, 2012 at 9:53 PM, Jay Whittaker [via Lucene] <
ml-node+[EMAIL PROTECTED]> wrote:

> Hey Ac,
>
> The logs I copied were from the .out files while a job was running. I
> thought that would be the best way to get a good idea of what was
> happening.
>
> Cheers,
>
> Jay
>
> On Tue, 2012-12-04 at 21:59 +0800, [hidden email]<http://user/SendEmail.jtp?type=node&node=4024496&i=0>wrote:
>
> > Hi,
> >
> > Have you also checked .out file of the tasktracker in logs? It could
> contain some useful information for the issue.
> >
> > Thanks
> > ac
> >
> >
> > On 4 Dec 2012, at 8:27 PM, Jay Whittaker wrote:
> >
> > > Hey,
> > >
> > > We are running Map reduce jobs against a 12 machine hbase cluster and
> > > for a long time they took approx 30 mins to return a result against
> ~95
> > > million rows. Without any major changes to the data or any upgrade of
> > > hbase/hadoop they now seem to be taking about 4 hours. and the logs
> are
> > > full of
> > >
> > > 2012-12-04 13:33:15,602 INFO org.apache.hadoop.mapred.TaskTracker:
> > > attempt_201211210952_0293_m_000031_0 0.0% row: 63 6f 6d 2e 70 72 6f 75
> > > 67 68 74
> > > ...
> > > 2012-12-04 13:45:17,134 INFO org.apache.hadoop.mapred.TaskTracker:
> > > attempt_201211210952_0293_m_000031_0 0.0% row: 63 6f 6d 2e 70 75 72 70
> > > 6c 65 64 65 73 69 67 6e 73 65 72 76 69 63 65 73
> > > ...
> > > 2012-12-04 13:46:11,515 INFO org.apache.hadoop.mapred.TaskTracker:
> > > attempt_201211210952_0293_m_000031_0 0.0% row: 63 6f 6d 2e 70 75 73 68
> > > 74 6f 74 61 6c 6b 2d 6f 6e 6c 69 6e 65
> > >
> > > I presume the 0% is percent complete but I'm not sure as to why the
> time
> > > to complete has now jumped massively. Ganglia shows no major load on
> the
> > > nodes in question so I don't think it's that.
> > >
> > > What steps should I be taking to try troubleshoot the problem?
> > >
> > > Regards,
> > >
> > > Jay
> >
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/Map-Reduce-jobs-taking-a-long-time-at-the-end-tp4024231p4024496.html
>  To start a new topic under Hadoop lucene-users, email
> ml-node+[EMAIL PROTECTED]
> To unsubscribe from Lucene, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=472066&code=aW4uYWJkdWxAZ21haWwuY29tfDQ3MjA2NnwxMDczOTUyNDEw>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
-----
THANKS AND REGARDS,
SYED ABDUL KATHER
--
View this message in context: http://lucene.472066.n3.nabble.com/Map-Reduce-jobs-taking-a-long-time-at-the-end-tp4024231p4024515.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
+
Jay Whittaker 2012-12-06, 12:42