Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - TetherJob example Issue 512


Copy link to this message
-
Re: TetherJob example Issue 512
Jeremy Lewi 2011-06-30, 17:27
As a first step I created a Java unit test to run the java word count
tether example. In doing so I ran into a couple of issues (e.g clients
were using SaslocketServer but client was using SocketServer which at
least for me seemed problematic).

Should I report these to the mailing list or should I open a JIRA issue?

Right now I'm working on some minor modifications so that when you
specify the executable you can provide additional arguments. I was
planning on trying to produce and post a patch once this was working.
J
On Thu, 2011-06-30 at 10:18 -0700, Doug Cutting wrote:
> On 06/27/2011 07:45 PM, Jeremy Lewi wrote:
> > I've been look at the code for map-reduce tethering (Issue 512
> > https://issues.apache.org/jira/browse/AVRO-512)  to see what it would
> > take to add support for python (Issue 570).
>
> Great!  This would be wonderful to have.
>
> > I've been looking at the example provided by
> > lang/java/mapred/src/test/java...
> >
> > It looks like the test code only includes the code for the tethered task
> > which runs the word count.
> >
> > So it looks like example code for setting up an actual TetherJob which
> > uses this code is still missing.
>
> TetherTask and TetherTaskRunner (in the test tree) are the framework for
> tethered Java jobs.  They're meant to be fully general, not specific to
> WordCount, providing a reference for other implementations.
>
> Cheers,
>
> Doug