Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - running local hadoop job in windows


Copy link to this message
-
Re: running local hadoop job in windows
Mark Kerzner 2011-04-05, 02:13
I understand now, I need to install cygwin correctly, asking it for all the
right options.

Thank you,
Mark

On Mon, Apr 4, 2011 at 9:06 PM, Lance Norskog <[EMAIL PROTECTED]> wrote:

> You're stuck with cygwin! Hadoop insists on running the 'chmod'
> program. You have to have a binary in your search path.
>
> Lance
>
> On Sat, Mar 19, 2011 at 9:15 PM, Mark Kerzner <[EMAIL PROTECTED]>
> wrote:
> > Now I AM running under cygwin, and I get the same error, as you can see
> from
> > the attached screenshot.
> > Thank you,
> > Mark
> >
> > On Sat, Mar 19, 2011 at 9:16 PM, Simon <[EMAIL PROTECTED]> wrote:
> >>
> >> As far as I know, currently hadoop can only run under *nix like systems.
> >> Correct me if I am wrong.
> >> And if you want to run it under windows, you can try cygwin as the
> >> environment.
> >>
> >> Thanks
> >> Simon
> >>
> >> On Fri, Mar 18, 2011 at 7:11 PM, Mark Kerzner <[EMAIL PROTECTED]>
> >> wrote:
> >>
> >> > No, I hoped that it is not absolutely necessary for that kind of use.
> I
> >> > am
> >> > not even issuing the "hadoop -jar" command, but it is pure "java
> -jar".
> >> > It
> >> > is true though that my Ubuntu has a Hadoop set up, so maybe it is
> doing
> >> > a
> >> > lot of magic behind my back.
> >> >
> >> > I did not want to have my inexperienced Windows users to have to
> install
> >> > cygwin for just trying the package.
> >> >
> >> > Thank you,
> >> > Mark
> >> >
> >> > On Fri, Mar 18, 2011 at 6:06 PM, Stephen Boesch <[EMAIL PROTECTED]>
> >> > wrote:
> >> >
> >> > > presumably you ran this under cygwin?
> >> > >
> >> > > 2011/3/18 Mark Kerzner <[EMAIL PROTECTED]>
> >> > >
> >> > > > Hi, guys,
> >> > > >
> >> > > > I want to give my users a sense of what my hadoop application can
> >> > > > do,
> >> > and
> >> > > I
> >> > > > am trying to make it run in Windows, with this command
> >> > > >
> >> > > > java -jar dist\FreeEed.jar
> >> > > >
> >> > > > This command runs my hadoop job locally, and it works in Linux.
> >> > However,
> >> > > in
> >> > > > Windows I get the error listed below. Since I am running
> completely
> >> > > > locally,
> >> > > > I don't see why it is trying to do what it does. Is there a
> >> > > > workaround?
> >> > > >
> >> > > > Thank you,
> >> > > > Mark
> >> > > >
> >> > > > Error:
> >> > > >
> >> > > > 11/03/18 17:57:43 INFO jvm.JvmMetrics: Initializing JVM Metrics
> with
> >> > > > processName
> >> > > > =JobTracker, sessionId> >> > > > java.io.IOException: Failed to set permissions of path:
> >> > > > file:/tmp/hadoop-Mark/ma
> >> > > > pred/staging/Mark-1397630897/.staging to 0700
> >> > > >        at
> >> > > >
> >> > > >
> org.apache.hadoop.fs.RawLocalFileSystem.checkReturnValue(RawLocalFile
> >> > > > System.java:526)
> >> > > >        at
> >> > > >
> >> > > >
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSys
> >> > > > tem.java:500)
> >> > > >        at
> >> > > >
> >> > > >
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.jav
> >> > > > a:310)
> >> > > >        at
> >> > > >
> >> > > >
> org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:18
> >> > > > 9)
> >> > > >        at
> >> > > >
> >> > > >
> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmi
> >> > > > ssionFiles.java:116)
> >> > > >        at
> >> > > > org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:799)
> >> > > >        at
> >> > > > org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
> >> > > >        at java.security.AccessController.doPrivileged(Native
> Method)
> >> > > >        at javax.security.auth.Subject.doAs(Unknown Source)
> >> > > >        at
> >> > > >
> >> > > >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
> >> > > > tion.java:1063)
> >> > > >        at
> >> > > >
> >> > > >
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:7
> >> > > > 93)
> >> > > >        at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
> >> > > >        at
> >>