Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> loaded records but gone


Copy link to this message
-
loaded records but gone
I wrote a record loader MyLoader  and used it to load

aa = LOAD 'input_on_hdfs' USING MyLoader() AS (
blah:chararray,
blahblah:chararray
);

bb = FOREACH aa generate *;
store bb into 'somewhere_else';
when I ran this, pig shows that nnnnnn records were read in, but it shows
that 0 records were written,  how is this possible?

but between the above 2 statements, if I add the following, to store it
into the regular pigStorage() format, and then load it out back into aa,
then it works perfectly

STORE aa into '/blahblahba';
aa = LOAD aa FROM '/blahblahba' as (blah:chararray, blahblah:chararray)
;
where is it possibly  going wrong? is there a workaround ---- storing it
temporarily takes a lot of space and time
thanks
Yang
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB