Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Meaning of outputSchema - Python UDF


Copy link to this message
-
Re: Meaning of outputSchema - Python UDF
A couple of pointers:

UDFs must declare the schema of their output.
See the documentation here:
http://pig.apache.org/docs/r0.8.0/udf.html#Schema
In Java UDFs, this is done by implementing Schema outputSchema(Schema input)
In Python you use the @outputSchema decorator providing the schema in the format used in LOAD statements
See the schema documentation for reference:
http://pig.apache.org/docs/r0.8.0/piglatin_ref2.html#Schemas

Julien

On 1/7/11 5:42 AM, "Jonathan Coveney" <[EMAIL PROTECTED]> wrote:

The words themselves have no meaning, as far as I know. What does matter is that its a touple with one element, a double. In the case of x:{etc}, the important thing is that pig knows it is a bag. The names are for us, not pig.

So in your own example, one function returns and actual bag, whereas the other only tuples (that may go inyo a bag).

Sent via BlackBerry

-----Original Message-----
From: <[EMAIL PROTECTED]>
Date: Fri, 7 Jan 2011 18:29:03
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Meaning of outputSchema - Python UDF

Hi,

I don't seem to understand the OutputSchema constructs given in the documentation.

What is the significance of the letters 'x', 't', and the parentheses '{}' & '()'

@outputSchema("x:{t:(word:chararray)}")

For instance, the following looks different

#Percent- Percentage
@outputSchema("t:(percent:double)")
def percent(num, total):
  return num * 100 / total

Apologies if my question sounds dumb.
Please do not print this email unless it is absolutely necessary.

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.

WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

www.wipro.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB