Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Are records tuple

Copy link to this message
Re: Are records tuple
Mohit Anchlia 2012-04-20, 23:38
Additional details, when I try to build a tuple from a line in a file using
below code from my previous email I get . Looks like I need to define
schema somehow. I wonder how others test this. I am trying to test udf and
I need to pass a line from a file, build a tuple and pass it to eval func.

org.apache.pig.backend.executionengine.ExecException: ERROR 2112:
Unexpected datatype 97 while reading tuplefrom binary file.
 at org.apache.pig.data.BinInterSedes.getTupleSize(BinInterSedes.java:132)
 at org.apache.pig.data.BinInterSedes.addColsToTuple(BinInterSedes.java:553)
 at org.apache.pig.data.BinSedesTuple.readFields(BinSedesTuple.java:64)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at java.lang.reflect.Method.invoke(Method.java:597)

On Fri, Apr 20, 2012 at 4:22 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> I am writing unit test but I had a doubt. My understanding is that
> complete record is a tuple. So record "a b
> {(ST:NC),(ZIP:28613),(CITY:Xxxxxxx),(NAM2:Xxxxx X &xxx; Xxxxx X
> Xxxxxx)}        {(OCCUP:xxxxxxx xxxxx),(AGE:55    ),(MARITAL:Married)}"
> which is one line in a file is a tuple? But I somehow feel it's not right.
> Could someone please clarify?
> Below is the code, my test is incomplete but just pasting it to show how I
> am constructing this tuple.
>   TupleFactory mTupleFactory = TupleFactory.getInstance();
>  BagFactory mBagFactory = BagFactory.getInstance();
>  @Test
>  public void evalFuncTest() throws IOException{
>   String record = "a b
> {(ST:NC),(ZIP:28613),(CITY:Xxxxxxx),(NAM2:Xxxxx X &xxx; Xxxxx X
> Xxxxxx)}        {(OCCUP:xxxxxxx xxxxx),(AGE:55    ),(MARITAL:Married)}";
>   Tuple t = mTupleFactory.newTuple();
>   DataInput in = new DataInputStream(new
> ByteArrayInputStream(record.getBytes()));
>   t.readFields(in);
>  }