Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> New name requirements for Avro 1.5


Copy link to this message
-
Re: New name requirements for Avro 1.5

On Jun 10, 2011, at 4:01pm, Scott Carey wrote:

> Ok, there was a bug in 1.4 that allowed for invalid names.  Sounds like we
> need something that allows reading some older files.
> Please open a JIRA ticket explaining the issue, we can discuss
> solutions/workarounds there.

https://issues.apache.org/jira/browse/AVRO-838

Thanks,

-- Ken
>
> On 6/10/11 1:48 PM, "Ken Krugler" <[EMAIL PROTECTED]> wrote:
>
>> I'd run into a similar issue, where I used a record name with a "-" in it
>> when creating files with Avro 1.4.
>>
>> So I've got files that are valid/readable with 1.4, but seem to be
>> unusable with 1.5.
>>
>> -- Ken
>>
>> On Jun 10, 2011, at 12:32pm, Matt Pouttu-Clarke wrote:
>>
>>> OK, thanks.  This is a change from Avro 1.4 which will make older Avro
>>> files
>>> unreadable by 1.5 until the field names are changed.
>>>
>>> Can we get away with "tweeking" the field name in the JSON meta data
>>> portion
>>> of the file, or do we have to re-write the whole file again with the new
>>> meta data?
>>>
>>> On 6/10/11 12:23 PM, "Douglas Creager" <[EMAIL PROTECTED]> wrote:
>>>
>>>>> I am using Avro for processing/encoding billions of rows using
>>>>> Hadoop.  When
>>>>> I attempted to upgrade to Avro 1.5 I got an error about illegal field
>>>>> names.
>>>>> I notice this case happens when I use a O-¹ character or a O.¹ in a
>>>>> field
>>>>> name.  
>>>>>
>>>>> Are there any other restrictions on fields names I need to know when
>>>>> upgrading?
>>>>
>>>> You can find the full details in the Avro spec [1].  It's basically
>>>> just like
>>>> the identifier restrictions in most programming languages.  Names can
>>>> only
>>>> contain letters, numbers, and underscores, and can't start with a
>>>> number.
>>>> Dots are used to created namespaces, just like fully-qualified Java
>>>> identifiers.
>>>>
>>>> [1] http://avro.apache.org/docs/current/spec.html
>>>>
>>>> cheers
>>>> doug
>>>
>>>
>>> iCrossing Privileged and Confidential Information
>>> This email message is for the sole use of the intended recipient(s) and
>>> may contain confidential and privileged information of iCrossing. Any
>>> unauthorized review, use, disclosure or distribution is prohibited. If
>>> you are not the intended recipient, please contact the sender by reply
>>> email and destroy all copies of the original message.
>>>
>>>
>>
>> --------------------------
>> Ken Krugler
>> +1 530-210-6378
>> http://bixolabs.com
>> custom data mining solutions
>>
>>
>>
>>
>>
>>
>

--------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
custom data mining solutions