Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Issues starting TaskTracker


Copy link to this message
-
Re: Issues starting TaskTracker
Hi Shreya
       I'm not well verse with the Cygwin setup in windows. Even I tried cygwin  for hadoop but was running into multiple issues,So dropped it then n there. Also when you want work on multiple hadoop tools like pig,hive,hbase etc maintaining everything becomes a hassle.(That is what I think about it). My suggestion would be to try out a linux box which would be much better. Get a linux Ubuntu image follow the steps on cloudera site in setting up hadoop in pseudo distributed mode on top of the same.(You can go in for the apache url as well). Just set up hadoop first and once it is success, go in for the required sub projects. It would help you to keep things rolling till we get a 32 bit VM image released from Cloudera.
      Also when you go for the VM image go for Ubuntu 10.10 or previous versions 32 Bit.(Since your host OS is 32 bit). BeTter not to go in for Ubuntu 11 cos the CDH distribution don't list it as a supporting OS for their release and may need to go in for installation from the tarball and so on. To keep installation simple and straight forward,check the list of  supporting OS from Cloudera site before you go in for a VM image download.

Hope it helps.
Regards
Bejoy K S

-----Original Message-----
From: <[EMAIL PROTECTED]>
Date: Wed, 14 Sep 2011 18:16:21
To: <[EMAIL PROTECTED]>
Subject: RE: Issues starting TaskTracker
 

Hi Bejoy

 

I am still facing the same issue with task tracker not starting

This was the mail I had sent

Hi,

 

I am trying to setup Hadoop on windows using Cygwin.

Facing issues while starting task tracker:

 

2011-09-12 15:20:16,890 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics
system started

2011-09-12 15:20:17,375 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
ugi registered.

2011-09-12 15:20:17,390 WARN
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi
already exists!

2011-09-12 15:20:17,562 INFO org.mortbay.log: Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog

2011-09-12 15:20:17,671 INFO org.apache.hadoop.http.HttpServer: Added
global filtersafety
(class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)

2011-09-12 15:20:17,718 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
Initializing logs' truncater with mapRetainSize=-1 and
reduceRetainSize=-1

2011-09-12 15:20:17,750 INFO org.apache.hadoop.mapred.TaskTracker:
Starting tasktracker with owner as SYSTEM

2011-09-12 15:20:17,750 ERROR org.apache.hadoop.mapred.TaskTracker: Can
not start task tracker because java.io.IOException: Failed to set
permissions of path: /usr/local/mapredLocalDir/ttprivate to 0700

      at
org.apache.hadoop.fs.RawLocalFileSystem.checkReturnValue(RawLocalFileSys
tem.java:525)

      at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem
.java:499)

      at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:3
18)

      at
org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)

      at
org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:635)

      at
org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1328)

      at
org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3430)

 

2011-09-12 15:20:17,765 INFO org.apache.hadoop.mapred.TaskTracker:
SHUTDOWN_MSG:

/************************************************************

SHUTDOWN_MSG: Shutting down TaskTracker

Have tried manually setting permissions to 0700 but that doesnt help...
Still getting the same issue.

 

Regards,

Shreya

 

 

The owner is showing as system whereas

I was getting the message owner SYSTEM when I was using default
locations for "mapred.system.dir" and "mapred.local.dir" in
mapred-site.xml After giving my own locations in mapred-site.xml as
below:

<property><name>mapred.local.dir</name><value>/usr/local/mapredLocalDir<
/value></property>

<property><name>mapred.system.dir</name><value>/usr/local/mapredSystemDi
r</value></property>

 

I was getting the message - running as TITUN, but the same error

 

Following is the output of ls -al c:/usr/local/mapredLocalDir $ ls -al
c:/usr/local/mapredLocalDir total 0

drwx------+ 4 Titun None 0 Sep 12 15:20 .

drwx------+ 5 Titun None 0 Sep 12 15:20 ..

drwx------+ 2 Titun None 0 Sep 12 15:20 taskTracker

drwx------+ 2 Titun None 0 Sep 12 15:20 ttprivate

 

 

 

Do you know what can be the issue? I am assuming owner shd be the
problem but how can I change process owner to Titun?

Any pointers would help

 

Regards,

Shreya

 

 

From: Bejoy KS [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, September 14, 2011 5:53 PM
To: [EMAIL PROTECTED]
Subject: Re: Issues starting TaskTracker

 

Shreya
       You are right. Even I ran into the same hurdle recently and just
raised our concern with Cloudera team. You can join the discussion
https://groups.google.com/a/cloudera.org/group/cdh-user/browse_thread/th
read/56bc678ff473a15f#
<https://groups.google.com/a/cloudera.org/group/cdh-user/browse_thread/t
hread/56bc678ff473a15f>

       It is just with the recent release, the new VM released is out of
64 bit Cent OS. Where as the previous versions were based out of 32 bit
Ubuntu OS.(I do have the 32 bit one but size is too large to be send
across by any means). You can check with the Cloudera archives. (not
sure where you can find the link to older versions, need to reach out
Cloudera team for  help).

Regards
Bejoy.K.S

On Wed, Sep 14, 2011 at 9:33 AM, <[EMAIL PROTECTED]> wrote:

Hi

 

I downloaded cloudera VM
(https://ccp.cloudera.com/display/SUPPORT/Cloudera's+Hadoop+Demo+VM#Clou
dera%27sHadoopDemoVM-DemoVMWareImage
<https://ccp.cloudera.com/display/SUPPORT/Cloudera%27s+Hadoop+Demo+VM#Cl
oudera%27sHadoopDemoVM-DemoVMWareImage> )  for VMware and vmware player.

The VM is 64 bit but my OS is 32 bit.

What can be the solution?

 

 

Regards,

Shreya

 

From: Bejoy KS [mailto:[EMAIL PROTECTED]