-Re: Is it possible to use Pig streaming (StreamToPig) in a way that handles multiple lines as a single input tuple?
Raghu Angadi 2012-04-03, 21:32
why not pipe multi-line xml from the executable through another script that
On Wed, Mar 28, 2012 at 8:24 AM, Ahmed Sobhi <[EMAIL PROTECTED]> wrote:
> I'm streaming data in a pig script through an executable that returns an
> xml fragment for each line of input I stream to it. That xml fragment
> happens to span multiple lines and I have no control whatsoever over the
> output of the executable I stream to
> In relation to Use Hadoop Pig to load data from text file w/ each record on
> multiple lines?<
> the answer was suggesting writing a custom record reader. The problem is,
> this works fine if you want to implement a LoadFunc that reads from a file,
> but to be able to use streaming, it has to implement StreamToPig.
> StreamToPig allows you to only read one line at a time as far as I
> Does anyone know how to handle such a situation?
> Best Regards,
> Ahmed Sobhi