Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> New name requirements for Avro 1.5


+
Matt Pouttu-Clarke 2011-06-10, 17:46
+
Douglas Creager 2011-06-10, 19:23
+
Matt Pouttu-Clarke 2011-06-10, 19:32
+
Ken Krugler 2011-06-10, 20:48
+
Scott Carey 2011-06-10, 23:01
Copy link to this message
-
Re: New name requirements for Avro 1.5

On Jun 10, 2011, at 4:01pm, Scott Carey wrote:

> Ok, there was a bug in 1.4 that allowed for invalid names.  Sounds like we
> need something that allows reading some older files.
> Please open a JIRA ticket explaining the issue, we can discuss
> solutions/workarounds there.

https://issues.apache.org/jira/browse/AVRO-838

Thanks,

-- Ken
>
> On 6/10/11 1:48 PM, "Ken Krugler" <[EMAIL PROTECTED]> wrote:
>
>> I'd run into a similar issue, where I used a record name with a "-" in it
>> when creating files with Avro 1.4.
>>
>> So I've got files that are valid/readable with 1.4, but seem to be
>> unusable with 1.5.
>>
>> -- Ken
>>
>> On Jun 10, 2011, at 12:32pm, Matt Pouttu-Clarke wrote:
>>
>>> OK, thanks.  This is a change from Avro 1.4 which will make older Avro
>>> files
>>> unreadable by 1.5 until the field names are changed.
>>>
>>> Can we get away with "tweeking" the field name in the JSON meta data
>>> portion
>>> of the file, or do we have to re-write the whole file again with the new
>>> meta data?
>>>
>>> On 6/10/11 12:23 PM, "Douglas Creager" <[EMAIL PROTECTED]> wrote:
>>>
>>>>> I am using Avro for processing/encoding billions of rows using
>>>>> Hadoop.  When
>>>>> I attempted to upgrade to Avro 1.5 I got an error about illegal field
>>>>> names.
>>>>> I notice this case happens when I use a O-¹ character or a O.¹ in a
>>>>> field
>>>>> name.  
>>>>>
>>>>> Are there any other restrictions on fields names I need to know when
>>>>> upgrading?
>>>>
>>>> You can find the full details in the Avro spec [1].  It's basically
>>>> just like
>>>> the identifier restrictions in most programming languages.  Names can
>>>> only
>>>> contain letters, numbers, and underscores, and can't start with a
>>>> number.
>>>> Dots are used to created namespaces, just like fully-qualified Java
>>>> identifiers.
>>>>
>>>> [1] http://avro.apache.org/docs/current/spec.html
>>>>
>>>> cheers
>>>> doug
>>>
>>>
>>> iCrossing Privileged and Confidential Information
>>> This email message is for the sole use of the intended recipient(s) and
>>> may contain confidential and privileged information of iCrossing. Any
>>> unauthorized review, use, disclosure or distribution is prohibited. If
>>> you are not the intended recipient, please contact the sender by reply
>>> email and destroy all copies of the original message.
>>>
>>>
>>
>> --------------------------
>> Ken Krugler
>> +1 530-210-6378
>> http://bixolabs.com
>> custom data mining solutions
>>
>>
>>
>>
>>
>>
>

--------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
custom data mining solutions
+
Doug Cutting 2011-06-11, 12:57
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB