Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Generate Maps


Copy link to this message
-
Re: Generate Maps
you can write a EvalFuc like this:
/**
 * @author haitao
 *
 */
public class HEDataConverter extends EvalFunc<Tuple> {

private TupleFactory tupleFactory = TupleFactory.getInstance();

@Override
public Tuple exec(Tuple input) throws IOException {
byte[] mapValue = (byte[]) input.get(0);
List<Object> all = new ArrayList<Object>(1);
all.set(1, HEFunctions.parsePayload(mapValue));
return tupleFactory.newTuple(all);
}

@Override
public Schema outputSchema(Schema inputSchema) {
FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
return new Schema(newFieldSchema);
}

}

and you will get your map like this:

grunt>  A = load '/tmp/test_file' using PigStorage(',') as (uid:long, payload:bytearray, ts:long, type:int);
grunt>  B = foreach A generate he.HEDataConverter(payload);
grunt> describe B;
B: {payload: map[]}

hope this is helpful.

在 2012-2-9,上午11:11, Russell Jurney 写道�

> Is it possible to generate maps in Pig?  Is this type castable in any
> context?
>
> --
> Russell Jurney
> twitter.com/rjurney
> [EMAIL PROTECTED]
> datasyndrome.com

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB