Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> Hadoop Tools Layout (was Re: DistCpV2 in 0.23)


+
Amareshwari Sri Ramadasu 2011-08-29, 08:43
+
Allen Wittenauer 2011-08-29, 18:40
+
Amareshwari Sri Ramadasu 2011-08-30, 08:01
+
Vinod Kumar Vavilapalli 2011-08-30, 12:43
+
Mithun Radhakrishnan 2011-09-06, 05:28
+
Amareshwari Sri Ramadasu 2011-09-06, 07:13
+
Arun C Murthy 2011-09-06, 07:19
+
Vinod Kumar Vavilapalli 2011-09-06, 16:30
+
Allen Wittenauer 2011-09-06, 17:11
+
Eli Collins 2011-09-06, 23:32
+
Allen Wittenauer 2011-09-06, 23:46
+
Eric Yang 2011-09-07, 01:38
+
Alejandro Abdelnur 2011-09-07, 01:55
+
Vinod Kumar Vavilapalli 2011-09-07, 13:32
+
Eric Yang 2011-09-07, 17:50
+
Alejandro Abdelnur 2011-09-07, 18:18
+
Mahadev Konar 2011-09-07, 18:27
+
Milind.Bhandarkar@... 2011-09-07, 18:32
+
Alejandro Abdelnur 2011-09-07, 18:35
+
Rottinghuis, Joep 2011-09-08, 03:43
+
Amareshwari Sri Ramadasu 2011-09-08, 04:33
+
Rottinghuis, Joep 2011-09-09, 05:25
+
Vinod Kumar Vavilapalli 2011-09-12, 13:47
Copy link to this message
-
Re: Hadoop Tools Layout (was Re: DistCpV2 in 0.23)
Following up on this one, the hadoop-tools/ module is already in trunk,
distcp v2 addition could start.

Thanks.

Alejandro

On Mon, Sep 12, 2011 at 6:47 AM, Vinod Kumar Vavilapalli <
[EMAIL PROTECTED]> wrote:

> Alright, I think we've discussed enough on this and everybody seems to
> agree
> about a top level hadoop-tools module.
>
> Time to get into the action. I've filed HADOOP-7624. Amareshwari we can
> track the rest of the implementation related details and questions for your
> specific answers there.
>
> Thanks everyone for putting in your thoughts here.
> +Vinod
>
>
> On Fri, Sep 9, 2011 at 10:55 AM, Rottinghuis, Joep <[EMAIL PROTECTED]
> >wrote:
>
> > If hadoop-tools will be built as part of hadoop-common, then none of
> these
> > tools should be allowed to have a dependency on hdfs or mapreduce.
> > Conversely is also true, when tools do have any such dependency, they
> > cannot be bult as part of hadoop-common.
> > We cannot have circular dependencies like that.
> >
> > That is probably obvious, but I'm just saying...
> >
> > Joep
> > ________________________________________
> > From: Amareshwari Sri Ramadasu [[EMAIL PROTECTED]]
> > Sent: Wednesday, September 07, 2011 9:33 PM
> > To: [EMAIL PROTECTED]
> > Cc: [EMAIL PROTECTED]
> > Subject: Re: Hadoop Tools Layout (was Re: DistCpV2 in 0.23)
> >
> > It is good to have hadoop-tools module separately. But as I asked before
> we
> > need to answer some questions here. I'm trying to answer them myself.
> > Comments are welcome.
> >
> > > > 1.  Should the patches for tools be created against Hadoop Common?
> > Here, I meant should Hadoop common mailing list be used Or should we have
> a
> > separate mailing list for Tools? I agree with Vinod  here, that we can
> tie
> > it Hadoop-common jira/mailing lists.
> >
> > > > 2.  What will happen to the tools test automation? Will it run as
> part
> > of Hadoop Common tests?
> > Jenkins nightly/patch builds for Hadoop tools can run as part of Hadoop
> > common if use Hadoop common mailing list for this.
> > Also, I propose every patch build of HDFS and MAPREDUCE should also run
> > tools tests to make sure nothing is broken. That would ease the
> maintenance
> > of hadoop-tools module. I presume tools test should not take much time
> (some
> > thing like not more than 30 minutes).
> >
> > > > 3.  Will it introduce a dependency from MapReduce to Common? Or is
> this
> > > taken care in Mavenization?
> > I'm not sure about this whether Mavenization can take care of it.
> >
> > Thanks
> > Amareshwari
> >
> > On 9/8/11 9:13 AM, "Rottinghuis, Joep" <[EMAIL PROTECTED]> wrote:
> >
> > Does a separate hadoop-tools module imply that there will be a separate
> > Jenkins build as well?
> >
> > Thanks,
> >
> > Joep
> > ________________________________________
> > From: Alejandro Abdelnur [[EMAIL PROTECTED]]
> > Sent: Wednesday, September 07, 2011 11:35 AM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Hadoop Tools Layout (was Re: DistCpV2 in 0.23)
> >
> > Makes sense
> >
> > On Wed, Sep 7, 2011 at 11:32 AM, <[EMAIL PROTECTED]> wrote:
> >
> > > +1 for separate hadoop-tools module. However, if a tool is broken at
> > > release time, and no one comes forward to fix it, it should be removed.
> > > (i.e. Unlike contrib modules, where build and test failures were
> > > tolerated.)
> > >
> > > - milind
> > >
> > > On 9/7/11 11:27 AM, "Mahadev Konar" <[EMAIL PROTECTED]> wrote:
> > >
> > > >I like the idea of having tools as a seperate module and I dont think
> > > >that it will be a dumping ground unless we choose to make one of it.
> > > >
> > > >+1 for hadoop tools module under trunk.
> > > >
> > > >thanks
> > > >mahadev
> > > >
> > > >On Wed, Sep 7, 2011 at 11:18 AM, Alejandro Abdelnur <
> [EMAIL PROTECTED]>
> > > >wrote:
> > > >> Agreed, we should not have a dumping ground. IMO, what it would go
> > into
> > > >> hadoop-tools (i.e. distcp, streaming and someone could argue for
> > > >>FsShell as
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB