-Re: Can I start a Hadoop job from an EJB?
madhu phatak 2012-03-08, 11:41
Yes you can . Please make sure all Hadoop jars and conf directory is in
On Thu, Feb 9, 2012 at 7:02 AM, Sanjeev Verma <[EMAIL PROTECTED]>wrote:
> This is based on my understanding and no real life experience, so going to
> go out on a limb here :-)...assuming that you are planning on kicking off
> this map-reduce job based on a event of sorts (a file arrived and is ready
> to be processed?), and no direct "user wait" is involved, then yes, I would
> imagine you should be able to do something like this from inside a MDB
> (asynchronous so no one is held up in queue). Some random thoughts:
> 1. The user under which the app server is running will need to be a setup
> as a hadoop client user - this is rather obvious, just wanted to list it
> for completeness.
> 2. Hadoop, AFAIK, does not support transactions, and no XA. I assume you
> have no need for any of that stuff either.
> 3. Your MDB could potentially log job start/end times, but that info is
> available from Hadoop's monitoring infrastructure also.
> I would be very interested in hearing what senior members on the list have
> to say...
> On Wed, Feb 8, 2012 at 2:18 PM, Andy Doddington <[EMAIL PROTECTED]>
> > OK, I have a working Hadoop application that I would like to integrate
> > into an application
> > server environment. So, the question arises: can I do this? E.g. can I
> > create a JobClient
> > instance inside an EJB and run it in the normal way, or is something more
> > complex
> > required? In addition, are there any unpleasant interactions between the
> > application
> > server and the hadoop runtime?
> > Thanks for any guidance.
> > Andy D.