Hi All

Is there any way to load a text file as single record (text:chararray) in
Pig.

I am trying to load a bunch of text files from a directory . But it keeps
each line as single record.
--
**********************************
JAGANADH G
http://jaganadhg.in
*ILUGCBE*
http://ilugcbe.org.in

There is not a pre-built load function to do that.  In fact I am not aware of a Hadoop InputFormat that does that.  So you would first need to subclass Hadoop's FileInputFormat and then write a Load Func.  Both should be fairly straight forward since all you need to do is remove the record and field parsing from existing code.

Alan.

On Oct 3, 2012, at 9:49 AM, JAGANADH G wrote:

> Hi All
>
> Is there any way to load a text file as single record (text:chararray) in
> Pig.
>
> I am trying to load a bunch of text files from a directory . But it keeps
> each line as single record.
>
>
> --
> **********************************
> JAGANADH G
> http://jaganadhg.in
> *ILUGCBE*
> http://ilugcbe.org.in

You will have to write your own loader. You will have to modify the InputFormat which uses \n \r or EOF as split delimiter.

-----JAGANADH G <[EMAIL PROTECTED]> wrote: -----
To: [EMAIL PROTECTED]
From: JAGANADH G <[EMAIL PROTECTED]>
Date: 10/03/2012 10:20PM
Subject: Loading text file

Hi All

Is there any way to load a text file as single record (text:chararray) in
Pig.

I am trying to load a bunch of text files from a directory . But it keeps
each line as single record.
--
**********************************
JAGANADH G
http://jaganadhg.in
*ILUGCBE*
http://ilugcbe.org.in
=====-----=====-----====Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you

Here is an InputFormat that does this:

https://gist.github.com/808035

2012/10/3 Siddharth Tiwari <[EMAIL PROTECTED]>

> You will have to write your own loader. You will have to modify the
> InputFormat which uses \n \r or EOF as split delimiter.
>
>
>
>
>
> -----JAGANADH G <[EMAIL PROTECTED]> wrote: -----
> To: [EMAIL PROTECTED]
> From: JAGANADH G <[EMAIL PROTECTED]>
> Date: 10/03/2012 10:20PM
> Subject: Loading text file
>
> Hi All
>
> Is there any way to load a text file as single record (text:chararray) in
> Pig.
>
> I am trying to load a bunch of text files from a directory . But it keeps
> each line as single record.
>
>
> --
> **********************************
> JAGANADH G
> http://jaganadhg.in
> *ILUGCBE*
> http://ilugcbe.org.in
> =====-----=====-----====> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>