Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Is there a loader that loads a file as a line?

Copy link to this message
Re: Is there a loader that loads a file as a line?
Hello Jonathan,
        Have a look at Hadoop's WholeFileInputFormat..Might fit into
your requirements.
    Mohammad Tariq
On Fri, Jun 22, 2012 at 3:39 AM, Prashant Kommireddi
> I think you will need to implement a RecordReader/InputFormat of your own
> for this and use it with a LoadFunc. Not sure if Hadoop has a Reader that
> you could re-use for this.
> How do you handle the case when a file exceeds block size?
> On Thu, Jun 21, 2012 at 2:34 PM, Jonathan Coveney <[EMAIL PROTECTED]>wrote:
>> It can even be a bytearray. Basically I have a bunch of files, and I want
>> one file -> one row. Is there an easy way to do this? Or will I need to
>> provide a special fileinputformat etc?