Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> loaded records but gone


Copy link to this message
-
loaded records but gone
I wrote a record loader MyLoader  and used it to load

aa = LOAD 'input_on_hdfs' USING MyLoader() AS (
blah:chararray,
blahblah:chararray
);

bb = FOREACH aa generate *;
store bb into 'somewhere_else';
when I ran this, pig shows that nnnnnn records were read in, but it shows
that 0 records were written,  how is this possible?

but between the above 2 statements, if I add the following, to store it
into the regular pigStorage() format, and then load it out back into aa,
then it works perfectly

STORE aa into '/blahblahba';
aa = LOAD aa FROM '/blahblahba' as (blah:chararray, blahblah:chararray)
;
where is it possibly  going wrong? is there a workaround ---- storing it
temporarily takes a lot of space and time
thanks
Yang