Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Can I start a Hadoop job from an EJB?


Copy link to this message
-
Re: Can I start a Hadoop job from an EJB?
Yes you can . Please make sure all Hadoop jars and conf directory is in
classpath.

On Thu, Feb 9, 2012 at 7:02 AM, Sanjeev Verma <[EMAIL PROTECTED]>wrote:

> This is based on my understanding and no real life experience, so going to
> go out on a limb here :-)...assuming that you are planning on kicking off
> this map-reduce job based on a event of sorts (a file arrived and is ready
> to be processed?), and no direct "user wait" is involved, then yes, I would
> imagine you should be able to do something like this from inside a MDB
> (asynchronous so no one is held up in queue). Some random thoughts:
>
> 1. The user under which the app server is running will need to be a setup
> as a hadoop client user - this is rather obvious, just wanted to list it
> for completeness.
> 2. Hadoop, AFAIK, does not support transactions, and no XA. I assume you
> have no need for any of that stuff either.
> 3. Your MDB could potentially log job start/end times, but that info is
> available from Hadoop's monitoring infrastructure also.
>
> I would be very interested in hearing what senior members on the list have
> to say...
>
> HTH
>
> Sanjeev
>
> On Wed, Feb 8, 2012 at 2:18 PM, Andy Doddington <[EMAIL PROTECTED]>
> wrote:
>
> > OK, I have a working Hadoop application that I would like to integrate
> > into an application
> > server environment. So, the question arises: can I do this? E.g. can I
> > create a JobClient
> > instance inside an EJB and run it in the normal way, or is something more
> > complex
> > required? In addition, are there any unpleasant interactions between the
> > application
> > server and the hadoop runtime?
> >
> > Thanks for any guidance.
> >
> >        Andy D.
>

--
https://github.com/zinnia-phatak-dev/Nectar