Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Generate Maps


Copy link to this message
-
Re: Generate Maps
The code pasted is wrong, sorry for my mistake.

here's the updated code:
@SuppressWarnings("rawtypes")
public class HEDataConverter extends EvalFunc<Map> {

@Override
public Map<String, String> exec(Tuple input) throws IOException {
byte[] mapValue = ((DataByteArray) input.get(0)).get();
return HEFunctions.parsePayload(mapValue);
}

@Override
public Schema outputSchema(Schema inputSchema) {
FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
return new Schema(newFieldSchema);
}

}
在 2012-2-9,下午3:48, Russell Jurney 写道�

> Thanks, I'll try this tomorrow.
>
> Sent from my iPad
>
> On Feb 8, 2012, at 11:40 PM, Haitao Yao <[EMAIL PROTECTED]> wrote:
>
>> you can write a EvalFuc like this:
>> /**
>> * @author haitao
>> *
>> */
>> public class HEDataConverter extends EvalFunc<Tuple> {
>>
>>   private TupleFactory tupleFactory = TupleFactory.getInstance();
>>
>>   @Override
>>   public Tuple exec(Tuple input) throws IOException {
>>       byte[] mapValue = (byte[]) input.get(0);
>>       List<Object> all = new ArrayList<Object>(1);
>>       all.set(1, HEFunctions.parsePayload(mapValue));
>>       return tupleFactory.newTuple(all);
>>   }
>>
>>   @Override
>>   public Schema outputSchema(Schema inputSchema) {
>>       FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
>>       return new Schema(newFieldSchema);
>>   }
>>
>> }
>>
>> and you will get your map like this:
>>
>> grunt>  A = load '/tmp/test_file' using PigStorage(',') as (uid:long, payload:bytearray, ts:long, type:int);
>> grunt>  B = foreach A generate he.HEDataConverter(payload);
>> grunt> describe B;
>> B: {payload: map[]}
>>
>> hope this is helpful.
>>
>> 在 2012-2-9,上午11:11, Russell Jurney 写道:
>>
>>> Is it possible to generate maps in Pig?  Is this type castable in any
>>> context?
>>>
>>> --
>>> Russell Jurney
>>> twitter.com/rjurney
>>> [EMAIL PROTECTED]
>>> datasyndrome.com
>>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB