Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - loaded records but gone

Copy link to this message
loaded records but gone
Yang 2013-04-27, 06:52
I wrote a record loader MyLoader  and used it to load

aa = LOAD 'input_on_hdfs' USING MyLoader() AS (

bb = FOREACH aa generate *;
store bb into 'somewhere_else';
when I ran this, pig shows that nnnnnn records were read in, but it shows
that 0 records were written,  how is this possible?

but between the above 2 statements, if I add the following, to store it
into the regular pigStorage() format, and then load it out back into aa,
then it works perfectly

STORE aa into '/blahblahba';
aa = LOAD aa FROM '/blahblahba' as (blah:chararray, blahblah:chararray)
where is it possibly  going wrong? is there a workaround ---- storing it
temporarily takes a lot of space and time