Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Reduce java.lang.OutOfMemoryError


+
Kelly Burkhart 2011-02-16, 15:00
+
real great.. 2011-02-16, 15:02
+
Kelly Burkhart 2011-02-16, 15:11
+
James Seigel 2011-02-16, 15:16
+
real great.. 2011-02-16, 15:18
+
Jim Falgout 2011-02-16, 15:43
+
Kelly Burkhart 2011-02-16, 16:09
+
James Seigel 2011-02-16, 16:15
+
Kelly Burkhart 2011-02-16, 16:20
+
James Seigel 2011-02-16, 16:30
+
Kelly Burkhart 2011-02-16, 18:11
+
James Seigel 2011-02-16, 18:36
Copy link to this message
-
Re: Reduce java.lang.OutOfMemoryError
If you google for such memory failures, you'll find the mapreduce tunable
that'll help you:

mapred.job.shuffle.input.buffer.percent ; it is well known that the default
values in hadoop config

don't work well for large data systems

-Rahul
On Wed, Feb 16, 2011 at 10:36 AM, James Seigel <[EMAIL PROTECTED]> wrote:

> Good luck.
>
> Let me know how it goes.
>
> James
>
> Sent from my mobile. Please excuse the typos.
>
> On 2011-02-16, at 11:11 AM, Kelly Burkhart <[EMAIL PROTECTED]>
> wrote:
>
> > OK, the job was preferring the config file on my local machine which
> > is not part of the cluster over the cluster config files.  That seems
> > completely broken to me; my config was basically empty other than
> > containing the location of the cluster and my job apparently used
> > defaults rather than the cluster config.  It doesn't make sense to me
> > to keep configuration files synchronized on every machine that may
> > access the cluster.
> >
> > I'm running again; we'll see if it completes this time.
> >
> > -K
> >
> > On Wed, Feb 16, 2011 at 10:30 AM, James Seigel <[EMAIL PROTECTED]> wrote:
> >> Hrmmm. Well as you've pointed out. 200m is quite small and is probably
> >> the cause.
> >>
> >> Now thEre might be some overriding settings in something you are using
> >> to launch or something.
> >>
> >> You could set those values in the config to not be overridden in the
> >> main conf then see what tries to override it in the logs
> >>
> >> Cheers
> >> James
> >>
> >> Sent from my mobile. Please excuse the typos.
> >>
> >> On 2011-02-16, at 9:21 AM, Kelly Burkhart <[EMAIL PROTECTED]>
> wrote:
> >>
> >>> I should have mentioned this in my last email: I thought of that so I
> >>> logged into every machine in the cluster; each machine's
> >>> mapred-site.xml has the same md5sum.
> >>>
> >>> On Wed, Feb 16, 2011 at 10:15 AM, James Seigel <[EMAIL PROTECTED]> wrote:
> >>>> He might not have that conf distributed out to each machine
> >>>>
> >>>>
> >>>> Sent from my mobile. Please excuse the typos.
> >>>>
> >>>> On 2011-02-16, at 9:10 AM, Kelly Burkhart <[EMAIL PROTECTED]>
> wrote:
> >>>>
> >>>>> Our clust admin (who's out of town today) has mapred.child.java.opts
> >>>>> set to -Xmx1280 in mapred-site.xml.  However, if I go to the job
> >>>>> configuration page for a job I'm running right now, it claims this
> >>>>> option is set to -Xmx200m.  There are other settings in
> >>>>> mapred-site.xml that are different too.  Why would map/reduce jobs
> not
> >>>>> respect the mapred-site.xml file?
> >>>>>
> >>>>> -K
> >>>>>
> >>>>> On Wed, Feb 16, 2011 at 9:43 AM, Jim Falgout <
> [EMAIL PROTECTED]> wrote:
> >>>>>> You can set the amount of memory used by the reducer using the
> mapreduce.reduce.java.opts property. Set it in mapred-site.xml or override
> it in your job. You can set it to something like: -Xm512M to increase the
> amount of memory used by the JVM spawned for the reducer task.
> >>>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Kelly Burkhart [mailto:[EMAIL PROTECTED]]
> >>>>>> Sent: Wednesday, February 16, 2011 9:12 AM
> >>>>>> To: [EMAIL PROTECTED]
> >>>>>> Subject: Re: Reduce java.lang.OutOfMemoryError
> >>>>>>
> >>>>>> I have had it fail with a single reducer and with 100 reducers.
> >>>>>> Ultimately it needs to be funneled to a single reducer though.
> >>>>>>
> >>>>>> -K
> >>>>>>
> >>>>>> On Wed, Feb 16, 2011 at 9:02 AM, real great..
> >>>>>> <[EMAIL PROTECTED]> wrote:
> >>>>>>> Hi,
> >>>>>>> How many reducers are you using currently?
> >>>>>>> Try increasing the number or reducers.
> >>>>>>> Let me know if it helps.
> >>>>>>>
> >>>>>>> On Wed, Feb 16, 2011 at 8:30 PM, Kelly Burkhart <
> [EMAIL PROTECTED]>wrote:
> >>>>>>>
> >>>>>>>> Hello, I'm seeing frequent fails in reduce jobs with errors
> similar
> >>>>>>>> to
> >>>>>>>> this:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 2011-02-15 15:21:10,163 INFO org.apache.hadoop.mapred.ReduceTask:
> >>>>>>>> header: attempt_201102081823_0175_m_002153_0, compressed len:
+
Kelly Burkhart 2011-02-16, 20:40
+
Harsh J 2011-02-17, 03:22
+
James Seigel 2011-02-16, 15:17
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB