Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Are records tuple


Copy link to this message
-
Re: Are records tuple
Additional details, when I try to build a tuple from a line in a file using
below code from my previous email I get . Looks like I need to define
schema somehow. I wonder how others test this. I am trying to test udf and
I need to pass a line from a file, build a tuple and pass it to eval func.

org.apache.pig.backend.executionengine.ExecException: ERROR 2112:
Unexpected datatype 97 while reading tuplefrom binary file.
 at org.apache.pig.data.BinInterSedes.getTupleSize(BinInterSedes.java:132)
 at org.apache.pig.data.BinInterSedes.addColsToTuple(BinInterSedes.java:553)
 at org.apache.pig.data.BinSedesTuple.readFields(BinSedesTuple.java:64)
 at
com.intuit.cg.services.dp.analytics.pig.udf.TAXOUTPUTTest.evalFuncTest(TAXOUTPUTTest.java:30)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
 at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
 at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
 at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
...

On Fri, Apr 20, 2012 at 4:22 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> I am writing unit test but I had a doubt. My understanding is that
> complete record is a tuple. So record "a b
> {(ST:NC),(ZIP:28613),(CITY:Xxxxxxx),(NAM2:Xxxxx X &xxx; Xxxxx X
> Xxxxxx)}        {(OCCUP:xxxxxxx xxxxx),(AGE:55    ),(MARITAL:Married)}"
> which is one line in a file is a tuple? But I somehow feel it's not right.
> Could someone please clarify?
>
> Below is the code, my test is incomplete but just pasting it to show how I
> am constructing this tuple.
>
>
>   TupleFactory mTupleFactory = TupleFactory.getInstance();
>  BagFactory mBagFactory = BagFactory.getInstance();
>
>  @Test
>  public void evalFuncTest() throws IOException{
>   String record = "a b
> {(ST:NC),(ZIP:28613),(CITY:Xxxxxxx),(NAM2:Xxxxx X &xxx; Xxxxx X
> Xxxxxx)}        {(OCCUP:xxxxxxx xxxxx),(AGE:55    ),(MARITAL:Married)}";
>   Tuple t = mTupleFactory.newTuple();
>   DataInput in = new DataInputStream(new
> ByteArrayInputStream(record.getBytes()));
>   t.readFields(in);
>  }
>