Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Are records tuple


Copy link to this message
-
Re: Are records tuple
Additional details, when I try to build a tuple from a line in a file using
below code from my previous email I get . Looks like I need to define
schema somehow. I wonder how others test this. I am trying to test udf and
I need to pass a line from a file, build a tuple and pass it to eval func.

org.apache.pig.backend.executionengine.ExecException: ERROR 2112:
Unexpected datatype 97 while reading tuplefrom binary file.
 at org.apache.pig.data.BinInterSedes.getTupleSize(BinInterSedes.java:132)
 at org.apache.pig.data.BinInterSedes.addColsToTuple(BinInterSedes.java:553)
 at org.apache.pig.data.BinSedesTuple.readFields(BinSedesTuple.java:64)
 at
com.intuit.cg.services.dp.analytics.pig.udf.TAXOUTPUTTest.evalFuncTest(TAXOUTPUTTest.java:30)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
 at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
 at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
 at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
...

On Fri, Apr 20, 2012 at 4:22 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> I am writing unit test but I had a doubt. My understanding is that
> complete record is a tuple. So record "a b
> {(ST:NC),(ZIP:28613),(CITY:Xxxxxxx),(NAM2:Xxxxx X &xxx; Xxxxx X
> Xxxxxx)}        {(OCCUP:xxxxxxx xxxxx),(AGE:55    ),(MARITAL:Married)}"
> which is one line in a file is a tuple? But I somehow feel it's not right.
> Could someone please clarify?
>
> Below is the code, my test is incomplete but just pasting it to show how I
> am constructing this tuple.
>
>
>   TupleFactory mTupleFactory = TupleFactory.getInstance();
>  BagFactory mBagFactory = BagFactory.getInstance();
>
>  @Test
>  public void evalFuncTest() throws IOException{
>   String record = "a b
> {(ST:NC),(ZIP:28613),(CITY:Xxxxxxx),(NAM2:Xxxxx X &xxx; Xxxxx X
> Xxxxxx)}        {(OCCUP:xxxxxxx xxxxx),(AGE:55    ),(MARITAL:Married)}";
>   Tuple t = mTupleFactory.newTuple();
>   DataInput in = new DataInputStream(new
> ByteArrayInputStream(record.getBytes()));
>   t.readFields(in);
>  }
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB