Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Generate Maps

Copy link to this message
Re: Generate Maps
Haitao Yao 2012-02-09, 07:40
you can write a EvalFuc like this:
 * @author haitao
public class HEDataConverter extends EvalFunc<Tuple> {

private TupleFactory tupleFactory = TupleFactory.getInstance();

public Tuple exec(Tuple input) throws IOException {
byte[] mapValue = (byte[]) input.get(0);
List<Object> all = new ArrayList<Object>(1);
all.set(1, HEFunctions.parsePayload(mapValue));
return tupleFactory.newTuple(all);

public Schema outputSchema(Schema inputSchema) {
FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
return new Schema(newFieldSchema);


and you will get your map like this:

grunt>  A = load '/tmp/test_file' using PigStorage(',') as (uid:long, payload:bytearray, ts:long, type:int);
grunt>  B = foreach A generate he.HEDataConverter(payload);
grunt> describe B;
B: {payload: map[]}

hope this is helpful.

在 2012-2-9,上午11:11, Russell Jurney 写道�

> Is it possible to generate maps in Pig?  Is this type castable in any
> context?
> --
> Russell Jurney
> twitter.com/rjurney
> datasyndrome.com