Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Bigtop >> mail # user >> "bigpetstore" another idea that i forgot to mention which might fit into bigtop.


+
Jay Vyas 2013-09-19, 21:19
+
Roman Shaposhnik 2013-09-19, 21:28
+
Jay Vyas 2013-09-19, 21:41
+
Jay Vyas 2013-09-23, 18:47
Copy link to this message
-
Re: "bigpetstore" another idea that i forgot to mention which might fit into bigtop.
This is a great idea and this proposal looks good to me.

My only feedback would be:
Would "samples" be more obvious than "blueprints"?

On 09/23/2013 11:47 AM, Jay Vyas wrote:
> bumtp^^ ... Any thoughts on where these blueprints should go and how to
> organize them?  At that point ill roll it into  a jira
>
>
> On Thu, Sep 19, 2013 at 5:41 PM, Jay Vyas <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     Okay that makes sense.  Now time for my all-to-often asked bigtop
>     question:
>
>     Where would this project go?
>
>     My proposal:
>
>     My initial thoughts are a
>
>     1) location: Simply a new submodule, under top level bigtop, called
>     blueprints/ with a single java application under bigpetstore/ as the
>     submodule.
>
>     2) extensibility: Then others could add their own submodules easily
>     by just creating a new folder.
>
>     3) deliverable: The artifact created by this submodule would simply
>     be a jar file, with a shell script for executing the whole pipeline.
>
>     4) bootstrap / input data: We could put CSV delimited input data
>     somewhere on a public s3 instance , and have small input csv text
>     files as a failsafe inside the repo so people can always run it from
>     just the git repo alone.
>
>
>
>
>
>
>
>
>
>
>
>
>     On Thu, Sep 19, 2013 at 5:28 PM, Roman Shaposhnik <[EMAIL PROTECTED]
>     <mailto:[EMAIL PROTECTED]>> wrote:
>
>         On Thu, Sep 19, 2013 at 2:19 PM, Jay Vyas <[EMAIL PROTECTED]
>         <mailto:[EMAIL PROTECTED]>> wrote:
>          > Hey bigtop:
>          >
>          > Another idea, which i have been toying with for some time -
>         is the idea of
>          > implementing the old hibernate/ibatis app "jpetstore" for hadoop.
>
>         I think providing example would be very nice. I honestly think that
>         perhaps the best place to start would be in Hue, though. Hue already
>         comes with simple toy example for things like Hive/Pig
>         workflows, etc.
>
>         Take a look at those.
>
>          > I think bigtop might be a good template for this, but not
>         sure if it should
>          > go in bigtop itself : i.e.  put an entire bigdata workflow
>         into bigtop as an
>          > example/template for people to better comprehend how
>         mapreduce ETL plays
>          > with adhoc analytics (HIVE/PIG) , and how machine learning
>         (mahout etc)
>          > finally interact with end sinks (hbase). etc...
>
>         Ah! That actually goes beyond examples and would also be quite
>         appreciated.
>         I'd call those 'Bigdata pipelines blueprints'. There I would
>         encourage folks
>         to approach it from the Oozie perspective. That's what most of the
>         heavyweight Hadoop users seems to be doing -- they've got those
>         complex
>         pipelines with ingest coming from the Flume side of things,
>         batch managed
>         by Oozie and analytic being provided by Hive/Pig/Spark and most
>         recently Solr.
>
>          > Not sure if this is in the scope of bigtop but i think, for
>         people getting
>          > into the hadoop ecosystem and useing bigtop as a venue to do
>         so, an example
>          > app of this sort might be particularly useful.
>          >
>          > Apologies is this is off scope of bigtop but let me know!
>
>         Personally I think Bigtop is a really good place for these types
>         of blueprints
>         to be developed and tested.
>
>         Thanks,
>         Roman.
>
>
>
>
>     --
>     Jay Vyas
>     http://jayunit100.blogspot.com
>
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com