Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive CLI and Standalone Server : Need Suggestion


Copy link to this message
-
Re: Hive CLI and Standalone Server : Need Suggestion
Eduardo,

Beside the mapreduce/streaming/hive/pig/sqoop/distcp action, Oozie has a
JAVA action (to execute a Java Main class in the cluster), a SSH action (to
execute a script via SSH in a remote host), and a SHELL action (to execute
a script in the cluster).

Would you mind explaining what does your m6d extension that JAVA, SSH or
SHELL cannot do to in a similar way?

Thanks.

Alejandro

On Mon, Mar 19, 2012 at 12:46 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> This is a bit of a problem. ozzie is great for workflow scheduling but
> oozie does not have "actions" for everything and adding actions is
> non-trivial in current versions.
>
> I have created some "bootleg/generic" oozie actions that make it easy
> to exec pretty much anything and treat it as an action.
>
> https://github.com/edwardcapriolo/m6d_oozie
>
> On Mon, Mar 19, 2012 at 3:38 PM,  <[EMAIL PROTECTED]> wrote:
> > Great topic as I was wondering a similar thing this morning…I want to use
> > oozie to execute my hive job, but I have to pass the job parameters that
> I
> > generate with a shell script.  Some of the literature that I’ve seen says
> > that oozie may or may not allow for calling shell scripts.  Is that true?
> >
> >
> >
> > Thanks
> >
> > Carla
> >
> >
> >
> > From: ext Bejoy Ks [mailto:[EMAIL PROTECTED]]
> > Sent: Monday, March 19, 2012 15:34
> > To: [EMAIL PROTECTED]
> > Subject: Re: Hive CLI and Standalone Server : Need Suggestion
> >
> >
> >
> > Hi LakshmiKanth
> >
> >         In production systems if you have a sequence of command to be
> > executed pack them in order in a file. Then execute the command as
> >
> > hive -f <filename> ;
> >
> >
> >
> > For simplicity, you can use a cron job to run it in a scheduled manner.
> Just
> > give this command in a .sh file call the file in cron. Infact you can use
> > any scheduler that would trigger a .sh file.
> >
> >
> >
> > But for hadoop based work flows the preferred workflow manager is oozie
> and
> > I recommend oozie for hadoop jobs.
> >
> >
> >
> > Regrads
> >
> > Bejoy KS
> >
> >
> >
> > ________________________________
> >
> > From: LakshmiKanth P <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Sent: Tuesday, March 20, 2012 12:19 AM
> > Subject: Hive CLI and Standalone Server : Need Suggestion
> >
> >
> >
> > Hi
> >
> >
> >
> >
> >
> > I need to schedule my hive scripts which needs to process incoming
> weblogs
> > on an hourly basis.
> >
> >
> >
> > Currently, I could process my weblog files by executing my scripts from
> hive
> > command line interface.  Now I want to keep my scripts in a file and
> invoke
> > my scripts at a regular periods of interval.  I came to know that hive
> > command line options provides a facility to pass the .sql file as input
> for
> > execution.  Is it the right approach for any production environment.
> >
> >
> >
> > OR
> >
> >
> >
> > Should I use my hive server in stand alone mode and inovke my hive
> scripts
> > using JDBC API.
> >
> >
> >
> > Request you to suggest me the best approach.
> >
> >
> >
> >
> >
> > Regards,
> >
> > LK
> >
> >
>