Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> windows support in trunk?


+
Andrey Klochkov 2013-09-24, 22:24
+
Arun C Murthy 2013-09-24, 22:31
+
Andrey Klochkov 2013-09-25, 00:10
+
Bikas Saha 2013-09-25, 05:31
+
Arpit Agarwal 2013-09-25, 18:12
+
Andrey Klochkov 2013-09-25, 18:30
Copy link to this message
-
Re: windows support in trunk?
I use native versions of sh.exe and tar.exe from GnuWin32 tools (
http://gnuwin32.sourceforge.net/). You don't need to build from an sh
command prompt, it just has to be in the path. cygwin is not supported any
more.

You are right about the missing mr-jobhistory-daemon.cmd.
mr-jobhistory-daemon.sh looks like a wrapper to start the job history
daemon using mapred.sh.

    nohup nice -n $HADOOP_MAPRED_NICENESS "$HADOOP_MAPRED_HOME"/bin/mapred
--config $HADOOP_CONF_DIR $command "$@" > "$log" 2>&1 < /dev/null &

For testing can you try something equivalent like 'mapred.cmd -config ..
start historyserver'? Feel free to file a Jira if you come across missing
functionality on Windows.

Thanks.
On Wed, Sep 25, 2013 at 11:30 AM, Andrey Klochkov <
[EMAIL PROTECTED]> wrote:

> I don't have problems with building. I have issues with running it. Usually
> I do this with the following steps:
> 1. build using "dist" maven profile to generate the distro under
> hadoop-dist/target
> 2. set HADOOP_HOME pointing to hadoop-dist/target/hadoop-x.x.x-SNAPSHOT
> 3. fix configs under $HADOOP_HOME/etc
> 4. use start-dfs.sh, start-yarh.sh and mr-jobhistory-daemon.sh to start the
> daemons
>
> The "dist" profile doesn't work on Windows without Cygwin as it depends on
> shell and tar utilities. How can I overcome this?
>
> Actually I installed Cygwin and managed to do step #1. Then I succeed with
> steps 2-4 and started all daemons using *cmd scripts. The only issue I have
> is with the jobhistory daemon, as there's no mr-jobhistory-daemon.cmd
> script. Is there an alternative way to start it?
>
>
>
> On Wed, Sep 25, 2013 at 11:12 AM, Arpit Agarwal <[EMAIL PROTECTED]
> >wrote:
>
> > Andrey, thanks for testing your patch on Windows.
> >
> > cygwin is no longer required. Instead, please install the free Windows
> SDK
> > and build from an SDK command prompt. This is documented in BUILDING.txt.
> > Please feel free to email the dev list if you get stuck at any step.
> >
> >
> > On Tue, Sep 24, 2013 at 10:31 PM, Bikas Saha <[EMAIL PROTECTED]>
> > wrote:
> >
> > > BUILDING.txt in the source code has a section for building on Windows.
> > >
> > > Bikas
> > >
> > > -----Original Message-----
> > > From: Andrey Klochkov [mailto:[EMAIL PROTECTED]]
> > > Sent: Tuesday, September 24, 2013 5:11 PM
> > > To: common-dev
> > > Subject: Re: windows support in trunk?
> > >
> > > Great, thanks!
> > >
> > > One more question on Windows support: how do I create a distribution
> and
> > > run
> > > it? It's for dev purposes - testing a patch to work on Windows, so I
> > can't
> > > really use HDP. On Linux/OSX I just do a build with -Pdist and then
> just
> > > use
> > > startXXX scripts from hadoop-project-dist/target to run all the
> daemons.
> > > Are
> > > there any docs on how to do it on Windows? E.g. do I need cygwin? I can
> > > build sources without cygwin, but hadoop-project-dist requires shell
> > > executable to build the tarball.
> > >
> > >
> > > On Tue, Sep 24, 2013 at 3:31 PM, Arun C Murthy <[EMAIL PROTECTED]>
> > > wrote:
> > >
> > > >
> > > > On Sep 24, 2013, at 3:24 PM, Andrey Klochkov
> > > > <[EMAIL PROTECTED]>
> > > > wrote:
> > > >
> > > > > Is Windows support in trunk currently? Or should I still use
> > > > > trunk-win to experiment with Hadoop on Windows? I've seen number of
> > > > > windows related patches going into trunk that's why I'm asking.
> > Thanks!
> > > > >
> > > > > I know I just need to ask Chris Nauroth, but sending here just in
> > > > > case
> > > > it's
> > > > > useful for others.
> > > >
> > > > *smile*
> > > >
> > > > Yes, FTR, Windows support is in trunk, branch-2 and branch-2.1.
> > > >
> > > > Arun
> > > >
> > > >
> > > > >
> > > > > --
> > > > > Andrey Klochkov
> > > >
> > > > --
> > > > Arun C. Murthy
> > > > Hortonworks Inc.
> > > > http://hortonworks.com/
> > > >
> > > >
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or

CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
+
Bikas Saha 2013-09-25, 22:00