Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Oozie dynamic action


Copy link to this message
-
Re: Oozie dynamic action
If you want to see a simple example of what you are looking for:
https://github.com/cloudera/cdh-twitter-example
It is part of this article:
http://blog.cloudera.com/blog/2012/09/analyzing-twitter-data-with-hadoop/
On Tue, Sep 17, 2013 at 4:20 AM, praveenesh kumar <[EMAIL PROTECTED]>wrote:

> Hi,
>
> I have a scenario in which I want to trigger a hive uploading script every
> day. I have a set of folders created for a set of customer ids everyday. My
> hive script will read the customer id from the path, checks whether the
> table for the customer id exits and if not create a table and will create
> partition based on date for  a "set of unknown customer_ids".
>
> I can get the set of unique customer_ids from a shell action. It can be
> passed as a list or string.
>
> My problem is how can I achieve this dynamic checking/creation of hive
> tables and partitions from oozie.
>
> Currently I am doing everything from the shell script and calling it as a
> shell action in oozie, but I was wondering if these kind of checks or some
> kind of for-loop kind of actions can be done in oozie.
>
> Any thoughts/suggestions on how to tackle the above scenario in the best
> way possible using oozie, would be highly helpful.
>
> Regards
> Praveenesh
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB