Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> running local hadoop job in windows


Copy link to this message
-
Re: running local hadoop job in windows
Guys,

I really appreciate all your answers. I am sure that I could have made
Hadoop run under Windows - all of you have done it - but I may go with
Stephen's advice after all. Since I am doing this for my open source
eDiscovery project, FreeEed <https://github.com/markkerzner/FreeEed>, then I
need a lot of Linux utilities anyway, such as readpst, which only works on
Linux.

Thank you,
Mark

On Tue, Apr 5, 2011 at 2:51 PM, Stephen Boesch <[EMAIL PROTECTED]> wrote:

> another approach you may well have already considered.. but may
> reconsider..
> use (free version of ..) vmware player running on your winXXX env ("host")
> and install linux distro of your choice as the guest o/s.  you can spin up
> essentially any number of instances that way.
>
> .. and not be concerned about the configuration/behavorial discrepancies
> between cygwin and native linux.
>
> If you wish there are pre-packaged distros for cloudera, apache, yahoo! for
> vmware.
>
>
>
> 2011/4/5 Tish Heyssel <[EMAIL PROTECTED]>
>
> > Mark,
> >
> > Make sure you add the cygwin/bin to your global PATH variable in Windows
> > too... and echo the PATH, if you're running in a command window to make
> > sure
> > it shows up there...  When running through eclipse, it should pick up the
> > PATH variable.
> >
> > Good luck.  its worth the trouble.  this does work.
> >
> > tish
> >
> > On Mon, Apr 4, 2011 at 10:13 PM, Mark Kerzner <[EMAIL PROTECTED]>
> > wrote:
> >
> > > I understand now, I need to install cygwin correctly, asking it for all
> > the
> > > right options.
> > >
> > > Thank you,
> > > Mark
> > >
> > > On Mon, Apr 4, 2011 at 9:06 PM, Lance Norskog <[EMAIL PROTECTED]>
> wrote:
> > >
> > > > You're stuck with cygwin! Hadoop insists on running the 'chmod'
> > > > program. You have to have a binary in your search path.
> > > >
> > > > Lance
> > > >
> > > > On Sat, Mar 19, 2011 at 9:15 PM, Mark Kerzner <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > > > Now I AM running under cygwin, and I get the same error, as you can
> > see
> > > > from
> > > > > the attached screenshot.
> > > > > Thank you,
> > > > > Mark
> > > > >
> > > > > On Sat, Mar 19, 2011 at 9:16 PM, Simon <[EMAIL PROTECTED]> wrote:
> > > > >>
> > > > >> As far as I know, currently hadoop can only run under *nix like
> > > systems.
> > > > >> Correct me if I am wrong.
> > > > >> And if you want to run it under windows, you can try cygwin as the
> > > > >> environment.
> > > > >>
> > > > >> Thanks
> > > > >> Simon
> > > > >>
> > > > >> On Fri, Mar 18, 2011 at 7:11 PM, Mark Kerzner <
> > [EMAIL PROTECTED]>
> > > > >> wrote:
> > > > >>
> > > > >> > No, I hoped that it is not absolutely necessary for that kind of
> > > use.
> > > > I
> > > > >> > am
> > > > >> > not even issuing the "hadoop -jar" command, but it is pure "java
> > > > -jar".
> > > > >> > It
> > > > >> > is true though that my Ubuntu has a Hadoop set up, so maybe it
> is
> > > > doing
> > > > >> > a
> > > > >> > lot of magic behind my back.
> > > > >> >
> > > > >> > I did not want to have my inexperienced Windows users to have to
> > > > install
> > > > >> > cygwin for just trying the package.
> > > > >> >
> > > > >> > Thank you,
> > > > >> > Mark
> > > > >> >
> > > > >> > On Fri, Mar 18, 2011 at 6:06 PM, Stephen Boesch <
> > [EMAIL PROTECTED]>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > presumably you ran this under cygwin?
> > > > >> > >
> > > > >> > > 2011/3/18 Mark Kerzner <[EMAIL PROTECTED]>
> > > > >> > >
> > > > >> > > > Hi, guys,
> > > > >> > > >
> > > > >> > > > I want to give my users a sense of what my hadoop
> application
> > > can
> > > > >> > > > do,
> > > > >> > and
> > > > >> > > I
> > > > >> > > > am trying to make it run in Windows, with this command
> > > > >> > > >
> > > > >> > > > java -jar dist\FreeEed.jar
> > > > >> > > >
> > > > >> > > > This command runs my hadoop job locally, and it works in
> > Linux.
> > > > >> > However,
> > > > >> > > in
> > > > >> > > > Windows I get the error listed below. Since I am running