Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> unsigned integer types


+
Robert Minsk 2012-12-20, 21:25
+
Martin Kleppmann 2012-12-21, 11:43
+
Robert Minsk 2012-12-21, 19:18
+
Doug Cutting 2012-12-21, 19:42
Copy link to this message
-
Re: unsigned integer types
So what is the second required name field for?  In your example "myFixed".

On 12/21/2012 11:42 AM, Doug Cutting wrote:
> That schema's not quite right.
>
> A fixed schema looks like:
>
>  {"name": "foo", "type": "fixed", "size": 8}
>
> A record field looks like:
>
>  {"name": "foo", "type": <schema>}
>
> So a record with a fixed as a field would look like:
>
> {
> "type": "record",
> "name": "recordWithFixed",
> "fields" : [
> {"name": "fixedValue", "type": {"name": "myFixed", "type": "fixed",
> "size": 8}}
>     ]
> }
>
> Alternately you can perhaps just skip the record and use the fixed
> schema directly.
>
> Doug
>
>
> On Fri, Dec 21, 2012 at 11:18 AM, Robert Minsk
> <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     Fixed does not seem to work.
>
>     avro-1.7.3
>
>     fixed.json:
>     {
>         "type": "record",
>         "name": "fixed",
>         "fields" : [
>             {"name": "foo", "type": "fixed", "size": 8}
>         ]
>     }
>
>     ./avrogencpp -i fixed.json -o fixed.hh
>     Segmentation fault (core dumped)
>
>     If I change the fixed.json to:
>     {
>         "type": "record",
>         "name": "test_fixed",
>         "fields" : [
>             {"name": "foo", "type": "fixed", "size": 8}
>         ]
>     }
>
>     ./avrogencpp -i fixed.json -o fixed.hh
>     Failed to parse or compile schema: Unknown type: fixed
>
>
>     On 12/21/2012 03:43 AM, Martin Kleppmann wrote:
>>     If your numbers are typically small, you can just use a signed type —
>>     the sign bit's overhead is insignificant.
>>
>>     If your numbers typically use the full range of 0 to 2^64-1 (or 0 to
>>     2^32-1), e.g. because they are hashes or random numbers from that
>>     range, you're best off using the 'fixed' type and specifying the
>>     number of bytes you want. In this case 'fixed' is more efficient than
>>     the variable-length encoding of int/long, because there is no overhead
>>     for indicating the length; it's simply stored as that number of bytes,
>>     and nothing else.
>>
>>     Because those two options cover most use cases, I don't think there
>>     are any plans to add unsigned int support to Avro.
>>
>>     Martin
>>
>>
>>     On 20 December 2012 13:25, Robert Minsk<[EMAIL PROTECTED]>  <mailto:[EMAIL PROTECTED]>
>>     wrote:
>>>     I am currently testing Avro for our network serialization for a mix of C++
>>>     and python programs.  I have noticed that Avro does not offer an unsigned
>>>     32-bit or unsigned 64-bit integer types.  How are people currently handling
>>>     unsigned integers?  Are there any plans to add unsigned integer types?
>>>
>>>     --
>>>     Robert Minsk
>>>     Systems and Software Engineer
>>>
>>>     WWW.METHODSTUDIOS.COM  <http://WWW.METHODSTUDIOS.COM>
>>>     730 Arizona Ave, Santa Monica, CA 90401
>>>     O:+1 310 434 6500  <tel:%2B1%20310%20434%206500>  // F:+1 310 434 6501  <tel:%2B1%20310%20434%206501>
>>>
>>>
>
>     --
>     Robert Minsk
>     Systems and Software Engineer
>
>     WWW.METHODSTUDIOS.COM <http://www.methodstudios.com>
>     730 Arizona Ave, Santa Monica, CA 90401
>     O:+1 310 434 6500 <tel:+13104346500> // F:+1 310 434 6501
>     <tel:+13104346501>
>
>     Los Angeles
>     <http://www.methodstudios.com/signature/url/los-angeles><http://www.methodstudios.com/signature/url/los-angeles>
>
>
>

--
Robert Minsk
Systems and Software Engineer

WWW.METHODSTUDIOS.COM <http://www.methodstudios.com>
730 Arizona Ave, Santa Monica, CA 90401
O:+1 310 434 6500 <tel:+13104346500> // F:+1 310 434 6501
<tel:+13104346501>

Los Angeles
<http://www.methodstudios.com/signature/url/los-angeles><http://www.methodstudios.com/signature/url/los-angeles>

This e-mail and any attachments are intended only for use by the addressee(s) named herein and may contain confidential information. If you are not the intended recipient of this e-mail, you are hereby notified any dissemination, distribution or copying of this email and any attachments is strictly prohibited. If you receive this email in error, please immediately notify the sender by return email and permanently delete the original, any copy and any printout thereof. The integrity and security of e-mail cannot be guaranteed.
+
Doug Cutting 2012-12-21, 20:03
+
Robert Minsk 2012-12-21, 19:59