Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Memory exception in the mapper
Mark Kerzner
2012-05-23, 19:16
Arun C Murthy
2012-05-24, 02:29
Mark Kerzner
2012-05-24, 02:40
Mark Kerzner
2012-05-24, 02:48
Joey Krabacher
2012-05-23, 19:47
Mark Kerzner
2012-05-24, 02:35
Joey Krabacher
2012-05-24, 02:43
Mark Kerzner
2012-05-24, 03:21
Joey Krabacher
2012-05-24, 03:39
+
Mark Kerzner 2012-05-23, 19:16
+
Arun C Murthy 2012-05-24, 02:29
+
Mark Kerzner 2012-05-24, 02:40
+
Mark Kerzner 2012-05-24, 02:48
+
Joey Krabacher 2012-05-23, 19:47
+
Mark Kerzner 2012-05-24, 02:35
+
Joey Krabacher 2012-05-24, 02:43
+
Mark Kerzner 2012-05-24, 03:21
+
Joey Krabacher 2012-05-24, 03:39
Copy link to this message
-
Re: Memory exception in the mapper
Mark Kerzner 2012-05-24, 03:44
Thanks, Joey,

we are in beta, and I kinda need these for debugging. But as soon as we go
to production, your word is well taken. (I hope we will replace the current
primitive logging with good one (log4j is I think preferred with Hadoop),
and then we can change the log level.

Mark

On Wed, May 23, 2012 at 10:39 PM, Joey Krabacher <[EMAIL PROTECTED]>wrote:

> No problem, glad I could help.
>
> In our test environment I have lots of output and logging turned on, but as
> soon as it is on production all output and logging is reduced to the bare
> minimum.
> Basically, in production we only log caught exceptions.
>
> I would take it out unless you absolutely need it. IMHO.
> If your jobs are not mission critical and do not need to run as smooth as
> possible then it's not as important to remove those.
>
> /* Joey */
>
> On Wed, May 23, 2012 at 10:21 PM, Mark Kerzner <[EMAIL PROTECTED]
> >wrote:
>
> > Joey,
> >
> > that did the trick!
> >
> > Actually, I am writing to the log with System.out.println() statements,
> and
> > I write about 12,000 lines, would that be a problem? I don't really need
> > this output, so if you think it's inadvisable, I will remove that.
> >
> > Also, I hope that if I have not 6,000 maps but 12,000 or even 30,000, it
> > will still work.
> >
> > Well, I will see pretty soon, I guess, with more data.
> >
> > Again, thank you.
> >
> > Sincerely,
> > Mark
> >
> > On Wed, May 23, 2012 at 9:43 PM, Joey Krabacher <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Mark,
> > >
> > > Have you tried tweaking the mapred.child.java.opts property in your
> > > mapred-site.xml?
> > >
> > > <property>
> > >    <name>mapred.child.java.opts</name>
> > >    <value>-Xmx2048m</value>
> > >  </property>
> > >
> > > This might help.
> > > It looks like the fatal error came right after the log truncater fired
> > off.
> > > Are you outputting anything to the logs manually, or have you looked at
> > the
> > > user logs to see if there is anything taking up lots of room?
> > >
> > > / * Joey */
> > >
> > >
> > > On Wed, May 23, 2012 at 9:35 PM, Mark Kerzner <
> [EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Joey,
> > > >
> > > > my errors closely resembles this
> > > > one<
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-user/201006.mbox/%[EMAIL PROTECTED]%3E
> > > > >in
> > > > the archives. I can now be much more specific with the errors
> message,
> > > > and it is quoted below. I tried -Xmx3096. But I got the same error.
> > > >
> > > > Thank you,
> > > > Mark
> > > >
> > > >
> > > > syslog logs
> > > > 2012-05-23 20:04:52,349 WARN org.apache.hadoop.util.NativeCodeLoader:
> > > > Unable to load native-hadoop library for your platform... using
> > > > builtin-java classes where applicable
> > > > 2012-05-23 20:04:52,519 INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics:
> > > > Initializing JVM Metrics with processName=MAP, sessionId> > > > 2012-05-23 20:04:52,695 INFO org.apache.hadoop.util.ProcessTree:
> setsid
> > > > exited with exit code 0
> > > > 2012-05-23 20:04:52,699 INFO org.apache.hadoop.mapred.Task:  Using
> > > > ResourceCalculatorPlugin :
> > > > org.apache.hadoop.util.LinuxResourceCalculatorPlugin@d56b37
> > > > 2012-05-23 20:04:52,813 INFO org.apache.hadoop.mapred.MapTask:
> > > io.sort.mb > > > > 100
> > > > 2012-05-23 20:04:52,998 INFO org.apache.hadoop.mapred.MapTask: data
> > > buffer
> > > > = 79691776/99614720
> > > > 2012-05-23 20:04:52,998 INFO org.apache.hadoop.mapred.MapTask: record
> > > > buffer = 262144/327680
> > > > 2012-05-23 20:04:53,010 WARN
> > > > org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native
> library
> > > not
> > > > loaded
> > > > 2012-05-23 20:12:29,120 INFO org.apache.hadoop.mapred.MapTask:
> Spilling
> > > map
> > > > output: buffer full= true
> > > > 2012-05-23 20:12:29,134 INFO org.apache.hadoop.mapred.MapTask:
> > bufstart > > > > 0; bufend = 79542629; bufvoid = 99614720
> > > > 2012-05-23 20:12:29,134 INFO org.apache.hadoop.mapred.MapTask: