Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Generate Maps


Copy link to this message
-
Re: Generate Maps
Haitao Yao 2012-02-09, 09:01
The code pasted is wrong, sorry for my mistake.

here's the updated code:
@SuppressWarnings("rawtypes")
public class HEDataConverter extends EvalFunc<Map> {

@Override
public Map<String, String> exec(Tuple input) throws IOException {
byte[] mapValue = ((DataByteArray) input.get(0)).get();
return HEFunctions.parsePayload(mapValue);
}

@Override
public Schema outputSchema(Schema inputSchema) {
FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
return new Schema(newFieldSchema);
}

}
在 2012-2-9,下午3:48, Russell Jurney 写道�

> Thanks, I'll try this tomorrow.
>
> Sent from my iPad
>
> On Feb 8, 2012, at 11:40 PM, Haitao Yao <[EMAIL PROTECTED]> wrote:
>
>> you can write a EvalFuc like this:
>> /**
>> * @author haitao
>> *
>> */
>> public class HEDataConverter extends EvalFunc<Tuple> {
>>
>>   private TupleFactory tupleFactory = TupleFactory.getInstance();
>>
>>   @Override
>>   public Tuple exec(Tuple input) throws IOException {
>>       byte[] mapValue = (byte[]) input.get(0);
>>       List<Object> all = new ArrayList<Object>(1);
>>       all.set(1, HEFunctions.parsePayload(mapValue));
>>       return tupleFactory.newTuple(all);
>>   }
>>
>>   @Override
>>   public Schema outputSchema(Schema inputSchema) {
>>       FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
>>       return new Schema(newFieldSchema);
>>   }
>>
>> }
>>
>> and you will get your map like this:
>>
>> grunt>  A = load '/tmp/test_file' using PigStorage(',') as (uid:long, payload:bytearray, ts:long, type:int);
>> grunt>  B = foreach A generate he.HEDataConverter(payload);
>> grunt> describe B;
>> B: {payload: map[]}
>>
>> hope this is helpful.
>>
>> 在 2012-2-9,上午11:11, Russell Jurney 写道:
>>
>>> Is it possible to generate maps in Pig?  Is this type castable in any
>>> context?
>>>
>>> --
>>> Russell Jurney
>>> twitter.com/rjurney
>>> [EMAIL PROTECTED]
>>> datasyndrome.com
>>