|
|
-
RE: Not able to start JobTracker in cygwin environmentBrad Sarsfield 2013-02-20, 14:59
I'd recommend picking up branch-trunk-win; (a call for vote for merge into trunk is going to happen soon after the work is complete and precommit built is clean; then these changes will be in trunk (!) ).
This removes the Cygwin dependency for running Hadoop on Windows. This work is being done under https://issues.apache.org/jira/browse/HADOOP-8562 It includes 1. Necessary changes in Hadoop to run natively on Windows. These changes handle differences in platforms related to path names, process/task management etc. 2. Addition of winutils tools for managing file permissions and ownership, user group mapping, hardlinks, symbolic links, chmod, disk utilization, and process/task management. 3. Added cmd scripts equivalent to existing shell scripts hadoop-daemon.sh, start and stop scripts. 4. Addition of block placement policy implementation to support cloud environment, more specifically Azure. ~Brad From: Amit [mailto:[EMAIL PROTECTED]] Sent: Wednesday, February 20, 2013 5:18 AM To: [EMAIL PROTECTED] Subject: Not able to start JobTracker in cygwin environment I have configured cygwin with Windows 7 64bit and trying to run hadoop on it. Was successful to run namenode, datanode, secondarynamenode and jobtracker, but facing problem with tasktracker. Here is the log - 13/02/20 18:37:04 INFO mapred.TaskTracker: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting TaskTracker STARTUP_MSG: host = Franklin/192.168.33.241 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.0.4 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012 ************************************************************/ 13/02/20 18:37:04 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 13/02/20 18:37:04 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 13/02/20 18:37:04 INFO impl.MetricsSystemImpl: TaskTracker metrics system started 13/02/20 18:37:04 INFO impl.MetricsSourceAdapter: MBean for source ugi registered. 13/02/20 18:37:04 WARN impl.MetricsSystemImpl: Source name ugi already exists! 13/02/20 18:37:05 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 13/02/20 18:37:05 INFO http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 13/02/20 18:37:05 INFO mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1 13/02/20 18:37:05 INFO mapred.TaskTracker: Starting tasktracker with owner as Admin 13/02/20 18:37:05 INFO mapred.TaskTracker: Good mapred local directories are: /app/hadoop/tmp/mapred/local 13/02/20 18:37:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 13/02/20 18:37:05 ERROR mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \app\hadoop\tmp\mapred\local\ttprivate to 0700 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689) at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189) at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:728) at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1459) at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742) 13/02/20 18:37:05 INFO mapred.TaskTracker: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down TaskTracker at Franklin/192.168.33.241 ************************************************************/ Tried after setting permission of \app\hadoop\tmp\mapred\local\ttprivate to 0700 using cygwin shell but getting the same exception. Can you help me? Thanks, Amit |