Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Not able to start JobTracker in cygwin environment

Copy link to this message
RE: Not able to start JobTracker in cygwin environment
Brad Sarsfield 2013-02-20, 14:59
I'd recommend picking up branch-trunk-win; (a call for vote for merge into trunk is going to happen soon after the work is complete and precommit built is clean; then these changes will be in trunk (!) ).  

This removes the Cygwin dependency for running Hadoop on Windows. This work is being done under https://issues.apache.org/jira/browse/HADOOP-8562

It includes
1. Necessary changes in Hadoop to run natively on Windows. These changes handle differences in platforms related to path names, process/task management etc.
2. Addition of winutils tools for managing file permissions and ownership, user group mapping, hardlinks, symbolic links, chmod, disk utilization, and process/task management.
3. Added cmd scripts equivalent to existing shell scripts hadoop-daemon.sh, start and stop scripts.
4. Addition of block placement policy implementation to support cloud environment, more specifically Azure.


From: Amit [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, February 20, 2013 5:18 AM
Subject: Not able to start JobTracker in cygwin environment

I have configured cygwin with Windows 7 64bit and trying to run hadoop on it.

Was successful to run namenode, datanode, secondarynamenode and jobtracker, but facing problem with tasktracker.
Here is the log - 

13/02/20 18:37:04 INFO mapred.TaskTracker: STARTUP_MSG:
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG:   host = Franklin/
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 1.0.4
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct  3 05:13:58 UTC 2012
13/02/20 18:37:04 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: TaskTracker metrics system started
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source ugi registered.
13/02/20 18:37:04 WARN impl.MetricsSystemImpl: Source name ugi already exists!
13/02/20 18:37:05 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
13/02/20 18:37:05 INFO http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
13/02/20 18:37:05 INFO mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
13/02/20 18:37:05 INFO mapred.TaskTracker: Starting tasktracker with owner as Admin
13/02/20 18:37:05 INFO mapred.TaskTracker: Good mapred local directories are: /app/hadoop/tmp/mapred/local
13/02/20 18:37:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/02/20 18:37:05 ERROR mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \app\hadoop\tmp\mapred\local\ttprivate to 0700
        at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
        at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
        at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
        at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
        at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:728)
        at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1459)
        at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)

13/02/20 18:37:05 INFO mapred.TaskTracker: SHUTDOWN_MSG:
SHUTDOWN_MSG: Shutting down TaskTracker at Franklin/
Tried after setting permission of \app\hadoop\tmp\mapred\local\ttprivate to 0700 using cygwin shell but getting the same exception. Can you help me?