Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> Hadoop Tools Layout (was Re: DistCpV2 in 0.23)


+
Amareshwari Sri Ramadasu 2011-08-29, 08:43
+
Allen Wittenauer 2011-08-29, 18:40
+
Amareshwari Sri Ramadasu 2011-08-30, 08:01
+
Vinod Kumar Vavilapalli 2011-08-30, 12:43
+
Mithun Radhakrishnan 2011-09-06, 05:28
+
Amareshwari Sri Ramadasu 2011-09-06, 07:13
+
Arun C Murthy 2011-09-06, 07:19
+
Vinod Kumar Vavilapalli 2011-09-06, 16:30
+
Allen Wittenauer 2011-09-06, 17:11
+
Eli Collins 2011-09-06, 23:32
+
Allen Wittenauer 2011-09-06, 23:46
+
Eric Yang 2011-09-07, 01:38
+
Alejandro Abdelnur 2011-09-07, 01:55
+
Vinod Kumar Vavilapalli 2011-09-07, 13:32
+
Eric Yang 2011-09-07, 17:50
+
Alejandro Abdelnur 2011-09-07, 18:18
+
Mahadev Konar 2011-09-07, 18:27
+
Milind.Bhandarkar@... 2011-09-07, 18:32
+
Alejandro Abdelnur 2011-09-07, 18:35
+
Rottinghuis, Joep 2011-09-08, 03:43
+
Amareshwari Sri Ramadasu 2011-09-08, 04:33
+
Rottinghuis, Joep 2011-09-09, 05:25
+
Vinod Kumar Vavilapalli 2011-09-12, 13:47
Copy link to this message
-
Re: Hadoop Tools Layout (was Re: DistCpV2 in 0.23)
Following up on this one, the hadoop-tools/ module is already in trunk,
distcp v2 addition could start.

Thanks.

Alejandro

On Mon, Sep 12, 2011 at 6:47 AM, Vinod Kumar Vavilapalli <
[EMAIL PROTECTED]> wrote:

> Alright, I think we've discussed enough on this and everybody seems to
> agree
> about a top level hadoop-tools module.
>
> Time to get into the action. I've filed HADOOP-7624. Amareshwari we can
> track the rest of the implementation related details and questions for your
> specific answers there.
>
> Thanks everyone for putting in your thoughts here.
> +Vinod
>
>
> On Fri, Sep 9, 2011 at 10:55 AM, Rottinghuis, Joep <[EMAIL PROTECTED]
> >wrote:
>
> > If hadoop-tools will be built as part of hadoop-common, then none of
> these
> > tools should be allowed to have a dependency on hdfs or mapreduce.
> > Conversely is also true, when tools do have any such dependency, they
> > cannot be bult as part of hadoop-common.
> > We cannot have circular dependencies like that.
> >
> > That is probably obvious, but I'm just saying...
> >
> > Joep
> > ________________________________________
> > From: Amareshwari Sri Ramadasu [[EMAIL PROTECTED]]
> > Sent: Wednesday, September 07, 2011 9:33 PM
> > To: [EMAIL PROTECTED]
> > Cc: [EMAIL PROTECTED]
> > Subject: Re: Hadoop Tools Layout (was Re: DistCpV2 in 0.23)
> >
> > It is good to have hadoop-tools module separately. But as I asked before
> we
> > need to answer some questions here. I'm trying to answer them myself.
> > Comments are welcome.
> >
> > > > 1.  Should the patches for tools be created against Hadoop Common?
> > Here, I meant should Hadoop common mailing list be used Or should we have
> a
> > separate mailing list for Tools? I agree with Vinod  here, that we can
> tie
> > it Hadoop-common jira/mailing lists.
> >
> > > > 2.  What will happen to the tools test automation? Will it run as
> part
> > of Hadoop Common tests?
> > Jenkins nightly/patch builds for Hadoop tools can run as part of Hadoop
> > common if use Hadoop common mailing list for this.
> > Also, I propose every patch build of HDFS and MAPREDUCE should also run
> > tools tests to make sure nothing is broken. That would ease the
> maintenance
> > of hadoop-tools module. I presume tools test should not take much time
> (some
> > thing like not more than 30 minutes).
> >
> > > > 3.  Will it introduce a dependency from MapReduce to Common? Or is
> this
> > > taken care in Mavenization?
> > I'm not sure about this whether Mavenization can take care of it.
> >
> > Thanks
> > Amareshwari
> >
> > On 9/8/11 9:13 AM, "Rottinghuis, Joep" <[EMAIL PROTECTED]> wrote:
> >
> > Does a separate hadoop-tools module imply that there will be a separate
> > Jenkins build as well?
> >
> > Thanks,
> >
> > Joep
> > ________________________________________
> > From: Alejandro Abdelnur [[EMAIL PROTECTED]]
> > Sent: Wednesday, September 07, 2011 11:35 AM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Hadoop Tools Layout (was Re: DistCpV2 in 0.23)
> >
> > Makes sense
> >
> > On Wed, Sep 7, 2011 at 11:32 AM, <[EMAIL PROTECTED]> wrote:
> >
> > > +1 for separate hadoop-tools module. However, if a tool is broken at
> > > release time, and no one comes forward to fix it, it should be removed.
> > > (i.e. Unlike contrib modules, where build and test failures were
> > > tolerated.)
> > >
> > > - milind
> > >
> > > On 9/7/11 11:27 AM, "Mahadev Konar" <[EMAIL PROTECTED]> wrote:
> > >
> > > >I like the idea of having tools as a seperate module and I dont think
> > > >that it will be a dumping ground unless we choose to make one of it.
> > > >
> > > >+1 for hadoop tools module under trunk.
> > > >
> > > >thanks
> > > >mahadev
> > > >
> > > >On Wed, Sep 7, 2011 at 11:18 AM, Alejandro Abdelnur <
> [EMAIL PROTECTED]>
> > > >wrote:
> > > >> Agreed, we should not have a dumping ground. IMO, what it would go
> > into
> > > >> hadoop-tools (i.e. distcp, streaming and someone could argue for
> > > >>FsShell as