Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - New name requirements for Avro 1.5


+
Matt Pouttu-Clarke 2011-06-10, 17:46
+
Douglas Creager 2011-06-10, 19:23
+
Matt Pouttu-Clarke 2011-06-10, 19:32
Copy link to this message
-
Re: New name requirements for Avro 1.5
Ken Krugler 2011-06-10, 20:48
I'd run into a similar issue, where I used a record name with a "-" in it when creating files with Avro 1.4.

So I've got files that are valid/readable with 1.4, but seem to be unusable with 1.5.

-- Ken

On Jun 10, 2011, at 12:32pm, Matt Pouttu-Clarke wrote:

> OK, thanks.  This is a change from Avro 1.4 which will make older Avro files
> unreadable by 1.5 until the field names are changed.
>
> Can we get away with "tweeking" the field name in the JSON meta data portion
> of the file, or do we have to re-write the whole file again with the new
> meta data?
>
> On 6/10/11 12:23 PM, "Douglas Creager" <[EMAIL PROTECTED]> wrote:
>
>>> I am using Avro for processing/encoding billions of rows using Hadoop.  When
>>> I attempted to upgrade to Avro 1.5 I got an error about illegal field names.
>>> I notice this case happens when I use a O-’ character or a O.’ in a field
>>> name.  
>>>
>>> Are there any other restrictions on fields names I need to know when
>>> upgrading?
>>
>> You can find the full details in the Avro spec [1].  It's basically just like
>> the identifier restrictions in most programming languages.  Names can only
>> contain letters, numbers, and underscores, and can't start with a number.
>> Dots are used to created namespaces, just like fully-qualified Java
>> identifiers.
>>
>> [1] http://avro.apache.org/docs/current/spec.html
>>
>> cheers
>> –doug
>
>
> iCrossing Privileged and Confidential Information
> This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information of iCrossing. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
>
>

--------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
custom data mining solutions
+
Scott Carey 2011-06-10, 23:01
+
Ken Krugler 2011-06-10, 23:16
+
Doug Cutting 2011-06-11, 12:57