Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Generate Maps


Copy link to this message
-
Re: Generate Maps
you can write a EvalFuc like this:
/**
 * @author haitao
 *
 */
public class HEDataConverter extends EvalFunc<Tuple> {

private TupleFactory tupleFactory = TupleFactory.getInstance();

@Override
public Tuple exec(Tuple input) throws IOException {
byte[] mapValue = (byte[]) input.get(0);
List<Object> all = new ArrayList<Object>(1);
all.set(1, HEFunctions.parsePayload(mapValue));
return tupleFactory.newTuple(all);
}

@Override
public Schema outputSchema(Schema inputSchema) {
FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
return new Schema(newFieldSchema);
}

}

and you will get your map like this:

grunt>  A = load '/tmp/test_file' using PigStorage(',') as (uid:long, payload:bytearray, ts:long, type:int);
grunt>  B = foreach A generate he.HEDataConverter(payload);
grunt> describe B;
B: {payload: map[]}

hope this is helpful.

在 2012-2-9,上午11:11, Russell Jurney 写道�

> Is it possible to generate maps in Pig?  Is this type castable in any
> context?
>
> --
> Russell Jurney
> twitter.com/rjurney
> [EMAIL PROTECTED]
> datasyndrome.com