Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive CLI and Standalone Server : Need Suggestion


Copy link to this message
-
Re: Hive CLI and Standalone Server : Need Suggestion
Eduardo,

Beside the mapreduce/streaming/hive/pig/sqoop/distcp action, Oozie has a
JAVA action (to execute a Java Main class in the cluster), a SSH action (to
execute a script via SSH in a remote host), and a SHELL action (to execute
a script in the cluster).

Would you mind explaining what does your m6d extension that JAVA, SSH or
SHELL cannot do to in a similar way?

Thanks.

Alejandro

On Mon, Mar 19, 2012 at 12:46 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> This is a bit of a problem. ozzie is great for workflow scheduling but
> oozie does not have "actions" for everything and adding actions is
> non-trivial in current versions.
>
> I have created some "bootleg/generic" oozie actions that make it easy
> to exec pretty much anything and treat it as an action.
>
> https://github.com/edwardcapriolo/m6d_oozie
>
> On Mon, Mar 19, 2012 at 3:38 PM,  <[EMAIL PROTECTED]> wrote:
> > Great topic as I was wondering a similar thing this morning…I want to use
> > oozie to execute my hive job, but I have to pass the job parameters that
> I
> > generate with a shell script.  Some of the literature that I’ve seen says
> > that oozie may or may not allow for calling shell scripts.  Is that true?
> >
> >
> >
> > Thanks
> >
> > Carla
> >
> >
> >
> > From: ext Bejoy Ks [mailto:[EMAIL PROTECTED]]
> > Sent: Monday, March 19, 2012 15:34
> > To: [EMAIL PROTECTED]
> > Subject: Re: Hive CLI and Standalone Server : Need Suggestion
> >
> >
> >
> > Hi LakshmiKanth
> >
> >         In production systems if you have a sequence of command to be
> > executed pack them in order in a file. Then execute the command as
> >
> > hive -f <filename> ;
> >
> >
> >
> > For simplicity, you can use a cron job to run it in a scheduled manner.
> Just
> > give this command in a .sh file call the file in cron. Infact you can use
> > any scheduler that would trigger a .sh file.
> >
> >
> >
> > But for hadoop based work flows the preferred workflow manager is oozie
> and
> > I recommend oozie for hadoop jobs.
> >
> >
> >
> > Regrads
> >
> > Bejoy KS
> >
> >
> >
> > ________________________________
> >
> > From: LakshmiKanth P <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Sent: Tuesday, March 20, 2012 12:19 AM
> > Subject: Hive CLI and Standalone Server : Need Suggestion
> >
> >
> >
> > Hi
> >
> >
> >
> >
> >
> > I need to schedule my hive scripts which needs to process incoming
> weblogs
> > on an hourly basis.
> >
> >
> >
> > Currently, I could process my weblog files by executing my scripts from
> hive
> > command line interface.  Now I want to keep my scripts in a file and
> invoke
> > my scripts at a regular periods of interval.  I came to know that hive
> > command line options provides a facility to pass the .sql file as input
> for
> > execution.  Is it the right approach for any production environment.
> >
> >
> >
> > OR
> >
> >
> >
> > Should I use my hive server in stand alone mode and inovke my hive
> scripts
> > using JDBC API.
> >
> >
> >
> > Request you to suggest me the best approach.
> >
> >
> >
> >
> >
> > Regards,
> >
> > LK
> >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB