Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - AVRO Path


Copy link to this message
-
Re: AVRO Path
Scott Carey 2012-01-13, 22:04


On 1/13/12 11:01 AM, "Jason Rutherglen" <[EMAIL PROTECTED]> wrote:

>Scott,
>
>It makes a lot of sense for NoSQL systems that store large AVRO blobs,
>that only require a subset of the data, where the entire blob could
>generate significant network traffic.  In a system like HBase, AVRO
>makes a lot of sense because schema evolution via HBase can be hairy,
>and the storage format is verbose.
>
>I would open a ticket however I do not have time allotted to work on it.

A ticket with an explanation of the use cases and some high level ideas is
fine.  It centralizes the discussion on the idea, and if someone stumbles
upon it later who has time to work on it then its nice to have an easy
reference location for those ideas.

>
>Jason
>
>On Thu, Jan 12, 2012 at 10:05 AM, Scott Carey <[EMAIL PROTECTED]>
>wrote:
>> There are no plans that I know of currently, although the topic came up
>> two times in separate conversations last night at the SF Hadoop MeetUp.
>>
>> I think an ability to extract a subset of a schema from a larger one and
>> read/write/transform data accordingly makes a lot of sense. Currently,
>>the
>> Avro spec allows for schema resolution which is sort of a degenerate
>> schema extraction/transformation at the record level without the ability
>> to address or extract nested elements.  An addition to the spec for
>> describing other schema extractions may be useful.  Further discussion
>> should probably be in a JIRA ticket or at least on the dev list.
>>
>> -Scott
>>
>> On 1/10/12 1:02 PM, "Jason Rutherglen" <[EMAIL PROTECTED]>
>>wrote:
>>
>>>Are there plans for (or is there already) an AVRO Path implementation
>>>(like XPath, or JSON Path).
>>>
>>>Thanks!
>>
>>