Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Using UDF to process whole record

Copy link to this message
Using UDF to process whole record
Stanley Xu 2013-01-22, 06:50
Dear all,

We are using thrift and elephant-bird to store our logs. And I wanted to
use some UDF to do complex processing on a single record, so I write some
pig like the following:

========================================================raw_data = load '$INPUT' using

A = FOREACH raw_data GENERATE com.udf.SomeUDF(raw_data);
B = LIMIT A 10;
But the pig will told me " ERROR 1000: Error during parsing. Scalars can be
only used with projections"

Is there anyway I could run a UDF on the raw_data here?
Best wishes,
Stanley Xu
Vitalii Tymchyshyn 2013-01-22, 11:27
Young Ng 2013-01-22, 07:34
Vitalii Tymchyshyn 2013-01-22, 11:34
Stanley Xu 2013-01-24, 03:56