Pig, mail # user - reading PigStorage or BinStorage from mapreduce?
reading PigStorage or BinStorage from mapreduce?
Corbin Hoenes 2010-10-07, 19:03
anyone ever read a pig output file with bags/tuples into a java map reduce program?
Re: reading PigStorage or BinStorage from mapreduce?
Jeff Zhang 2010-10-08, 01:29
As far as I know it is possible for you to read the output using java code. I suggest you look at PigServer.openIterator() which will read the output of pig script. But it will run the pig script. And I believe you already run the pig script in other places and want to use the output in another mapreduce job, right ? Then I suggest you look at org.apache.pig.impl.io.ReadToEndLoader which actually do the reading task. But I believe you have to do some hacking by yourself.
Another work around I can think of is that You can store the output in the plain text format, and then read the output using Hadoop FileSystem API, and then using org.apache.pig.data.DataReaderWriter to read the output line by line. On Fri, Oct 8, 2010 at 3:03 AM, Corbin Hoenes <[EMAIL PROTECTED]> wrote: > anyone ever read a pig output file with bags/tuples into a java map reduce program?
-- Best Regards
All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext