Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Bigtop >> mail # dev >> Spark's compute-classpath.sh


Copy link to this message
-
Re: Spark's compute-classpath.sh
Ah, I see. Thanks Cos!
On Tue, Jan 21, 2014 at 5:48 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:

> Yup, the compute class is different because the original one was doing done
> some funny things around the presence of examples source code and didn't
> work
> without them. Hence, I made the change to make sure that that cluster can
> be
> started, etc. Keep in mind, that runtime has been significantly changes
> recently (e.g. around 0.8.1) and you can see quite a bit of differences
> coming
> from it along.
>
> Contributing needed changes back to the open source will be a great idea
> indeed, especially considering that past 0.9.0 there likely to be much less
> fluctuations in the runtime environment.
>
> Cos
>
> On Tue, Jan 21, 2014 at 04:54PM, Mark Grover wrote:
> > Hi all,
> > I was poking around Bigtop's integration with Apache Spark and came
> across
> > our compute-classpath.sh (see
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/spark/compute-classpath.sh
> )
> > and noted it is significantly different from the compute-classpath.sh
> > bundled upstream (
> >
> https://github.com/apache/incubator-spark/blob/branch-0.8/bin/compute-classpath.sh
> > ).
> >
> > Does anyone (maybe, Cos?) know what the rationale was having a separate
> > copy of compute-classpath.sh in Bigtop? Also, if you have any ideas on
> how
> > we could contribute this upstream so we can decrease our divergence here,
> > I'd love to talk more.
> >
> > Thanks in advance!
> > Mark
>