Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Not able to start JobTracker in cygwin environment


Copy link to this message
-
RE: Not able to start JobTracker in cygwin environment
I'd recommend picking up branch-trunk-win; (a call for vote for merge into trunk is going to happen soon after the work is complete and precommit built is clean; then these changes will be in trunk (!) ).  

This removes the Cygwin dependency for running Hadoop on Windows. This work is being done under https://issues.apache.org/jira/browse/HADOOP-8562

It includes
1. Necessary changes in Hadoop to run natively on Windows. These changes handle differences in platforms related to path names, process/task management etc.
2. Addition of winutils tools for managing file permissions and ownership, user group mapping, hardlinks, symbolic links, chmod, disk utilization, and process/task management.
3. Added cmd scripts equivalent to existing shell scripts hadoop-daemon.sh, start and stop scripts.
4. Addition of block placement policy implementation to support cloud environment, more specifically Azure.

~Brad

From: Amit [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, February 20, 2013 5:18 AM
To: [EMAIL PROTECTED]
Subject: Not able to start JobTracker in cygwin environment

I have configured cygwin with Windows 7 64bit and trying to run hadoop on it.

Was successful to run namenode, datanode, secondarynamenode and jobtracker, but facing problem with tasktracker.
Here is the log - 

13/02/20 18:37:04 INFO mapred.TaskTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG:   host = Franklin/192.168.33.241
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 1.0.4
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct  3 05:13:58 UTC 2012
************************************************************/
13/02/20 18:37:04 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
13/02/20 18:37:04 INFO impl.MetricsSystemImpl: TaskTracker metrics system started
13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source ugi registered.
13/02/20 18:37:04 WARN impl.MetricsSystemImpl: Source name ugi already exists!
13/02/20 18:37:05 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
13/02/20 18:37:05 INFO http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
13/02/20 18:37:05 INFO mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
13/02/20 18:37:05 INFO mapred.TaskTracker: Starting tasktracker with owner as Admin
13/02/20 18:37:05 INFO mapred.TaskTracker: Good mapred local directories are: /app/hadoop/tmp/mapred/local
13/02/20 18:37:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/02/20 18:37:05 ERROR mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \app\hadoop\tmp\mapred\local\ttprivate to 0700
        at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
        at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
        at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
        at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
        at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:728)
        at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1459)
        at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)

13/02/20 18:37:05 INFO mapred.TaskTracker: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down TaskTracker at Franklin/192.168.33.241
************************************************************/
Tried after setting permission of \app\hadoop\tmp\mapred\local\ttprivate to 0700 using cygwin shell but getting the same exception. Can you help me?

Thanks,
Amit
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB