Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Re: Override input schema in AvroStorage


Copy link to this message
-
Re: Override input schema in AvroStorage
Resending now that I am subscribed :)

On 4/25/13 4:01 PM, "Enns, Steven" <[EMAIL PROTECTED]> wrote:

>Hi everyone,
>
>I would like to override the input schema in AvroStorage to make a pig
>script robust to schema evolution.  For example, suppose a new field is
>added to an avro schema with a default value of null.  If the input to a
>pig script using this field includes both old and new data, AvroStorage
>will merge the input schemas from the old and new data.  However, if the
>input includes only old data, the new schema will not be available to
>AvroStorage and pig will fail to interpret the script with an error such
>as "projected field [newField] does not exist in schema".  If AvroStorage
>accepted an input schema, the script would be valid for both the new and
>old data.  Is there any plan to implement this?
>
>Thanks,
>Steve
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB