Is it possible to generate maps in Pig?  Is this type castable in any
context?

--
Russell Jurney
twitter.com/rjurney
[EMAIL PROTECTED]
datasyndrome.com

you can write a EvalFuc like this:
/**
 * @author haitao
 *
 */
public class HEDataConverter extends EvalFunc<Tuple> {

private TupleFactory tupleFactory = TupleFactory.getInstance();

@Override
public Tuple exec(Tuple input) throws IOException {
byte[] mapValue = (byte[]) input.get(0);
List<Object> all = new ArrayList<Object>(1);
all.set(1, HEFunctions.parsePayload(mapValue));
return tupleFactory.newTuple(all);
}

@Override
public Schema outputSchema(Schema inputSchema) {
FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
return new Schema(newFieldSchema);
}

}

and you will get your map like this:

grunt>  A = load '/tmp/test_file' using PigStorage(',') as (uid:long, payload:bytearray, ts:long, type:int);
grunt>  B = foreach A generate he.HEDataConverter(payload);
grunt> describe B;
B: {payload: map[]}

hope this is helpful.

在 2012-2-9，上午11:11， Russell Jurney 写道�
�
> Is it possible to generate maps in Pig?  Is this type castable in any
> context?
>
> --
> Russell Jurney
> twitter.com/rjurney
> [EMAIL PROTECTED]
> datasyndrome.com

Thanks, I'll try this tomorrow.

Sent from my iPad

On Feb 8, 2012, at 11:40 PM, Haitao Yao <[EMAIL PROTECTED]> wrote:

> you can write a EvalFuc like this:
> /**
> * @author haitao
> *
> */
> public class HEDataConverter extends EvalFunc<Tuple> {
>
>    private TupleFactory tupleFactory = TupleFactory.getInstance();
>
>    @Override
>    public Tuple exec(Tuple input) throws IOException {
>        byte[] mapValue = (byte[]) input.get(0);
>        List<Object> all = new ArrayList<Object>(1);
>        all.set(1, HEFunctions.parsePayload(mapValue));
>        return tupleFactory.newTuple(all);
>    }
>
>    @Override
>    public Schema outputSchema(Schema inputSchema) {
>        FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
>        return new Schema(newFieldSchema);
>    }
>
> }
>
> and you will get your map like this:
>
> grunt>  A = load '/tmp/test_file' using PigStorage(',') as (uid:long, payload:bytearray, ts:long, type:int);
> grunt>  B = foreach A generate he.HEDataConverter(payload);
> grunt> describe B;
> B: {payload: map[]}
>
> hope this is helpful.
>
> 在 2012-2-9，上午11:11， Russell Jurney 写道：
>
>> Is it possible to generate maps in Pig?  Is this type castable in any
>> context?
>>
>> --
>> Russell Jurney
>> twitter.com/rjurney
>> [EMAIL PROTECTED]
>> datasyndrome.com
>

The code pasted is wrong, sorry for my mistake.

here's the updated code:
@SuppressWarnings("rawtypes")
public class HEDataConverter extends EvalFunc<Map> {

@Override
public Map<String, String> exec(Tuple input) throws IOException {
byte[] mapValue = ((DataByteArray) input.get(0)).get();
return HEFunctions.parsePayload(mapValue);
}

@Override
public Schema outputSchema(Schema inputSchema) {
FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
return new Schema(newFieldSchema);
}

}
在 2012-2-9，下午3:48， Russell Jurney 写道�
�
> Thanks, I'll try this tomorrow.
>
> Sent from my iPad
>
> On Feb 8, 2012, at 11:40 PM, Haitao Yao <[EMAIL PROTECTED]> wrote:
>
>> you can write a EvalFuc like this:
>> /**
>> * @author haitao
>> *
>> */
>> public class HEDataConverter extends EvalFunc<Tuple> {
>>
>>   private TupleFactory tupleFactory = TupleFactory.getInstance();
>>
>>   @Override
>>   public Tuple exec(Tuple input) throws IOException {
>>       byte[] mapValue = (byte[]) input.get(0);
>>       List<Object> all = new ArrayList<Object>(1);
>>       all.set(1, HEFunctions.parsePayload(mapValue));
>>       return tupleFactory.newTuple(all);
>>   }
>>
>>   @Override
>>   public Schema outputSchema(Schema inputSchema) {
>>       FieldSchema newFieldSchema = new FieldSchema("payload", DataType.MAP);
>>       return new Schema(newFieldSchema);
>>   }
>>
>> }
>>
>> and you will get your map like this:
>>
>> grunt>  A = load '/tmp/test_file' using PigStorage(',') as (uid:long, payload:bytearray, ts:long, type:int);
>> grunt>  B = foreach A generate he.HEDataConverter(payload);
>> grunt> describe B;
>> B: {payload: map[]}
>>
>> hope this is helpful.
>>
>> 在 2012-2-9，上午11:11， Russell Jurney 写道：
>>
>>> Is it possible to generate maps in Pig?  Is this type castable in any
>>> context?
>>>
>>> --
>>> Russell Jurney
>>> twitter.com/rjurney
>>> [EMAIL PROTECTED]
>>> datasyndrome.com
>>