Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Bigtop >> mail # dev >> don't support hadoop1 in bigtop after 0.4


Copy link to this message
-
Re: don't support hadoop1 in bigtop after 0.4
1) regarding smoke tests and backwards compatibility:

Not sure what components of bigtop your referring to , but FYI the  smoke
tests on head, or at least a solid subset  of them, work enough to test a
cluster against mahout, mapreduce, sqoop on hadoop 1.x  .

- For pig, hive, we are just running shell scripts in the test-execution
for those.
- for mahout, we run all tests but BayesNewsgroupClassifier
- for mapreduce, we run the terasort/teragen, wordcount, grep, and sleep
tests.

2) like sean sais.. I assume that the bigtop head dudes would agree with me
that, even though alot of bigtop work is driven by vendor needs, but since
its an apache project, those of us in the broader community can continue
adding patches to it to support our needs as long as they are synergistic
with the overall project goals ; do feel free to ping me or create a jira
for 1.x wish lists maybe i might share some of those with you and maybe
could even help on a patch

On Thu, Nov 14, 2013 at 11:21 AM, Sean Mackrory <[EMAIL PROTECTED]>wrote:

> Supporting both hadoop1 and hadoop2 in the same distribution of all the
> other components is a rather massive undertaking. As a rule, Bigtop limits
> itself to being a downstream distribution point (we do not apply our own
> patches to the upstream releases) - this would make it even more difficult
> to achieve compatibility across all the projects. There was an effort to
> release an updated hadoop1 distribution based on Bigtop 0.3 (0.3.1),
> however that was not finished. There's nothing I know if to stop a
> developer from completing that effort if they wanted to.
>
>
> On Thu, Nov 14, 2013 at 1:39 AM, pengwenwu2008 <[EMAIL PROTECTED]
> >wrote:
>
> > Hi All,
> >
> > Anyone can help answer "are there any reason don't support hadoop1 in
> > bigtop after 0.4"?
> > Thanks a lot for any comment !!
> >
> > Regards,
> > Wenwu,Peng
>

--
Jay Vyas
http://jayunit100.blogspot.com