Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - unsigned integer types


Copy link to this message
-
Re: unsigned integer types
Doug Cutting 2012-12-21, 19:42
That schema's not quite right.

A fixed schema looks like:

 {"name": "foo", "type": "fixed", "size": 8}

A record field looks like:

 {"name": "foo", "type": <schema>}

So a record with a fixed as a field would look like:

{
    "type": "record",
    "name": "recordWithFixed",
    "fields" : [
      {"name": "fixedValue", "type": {"name": "myFixed", "type": "fixed",
"size": 8}}
    ]
}

Alternately you can perhaps just skip the record and use the fixed schema
directly.

Doug
On Fri, Dec 21, 2012 at 11:18 AM, Robert Minsk <
[EMAIL PROTECTED]> wrote:

>  Fixed does not seem to work.
>
> avro-1.7.3
>
> fixed.json:
> {
>     "type": "record",
>     "name": "fixed",
>     "fields" : [
>         {"name": "foo", "type": "fixed", "size": 8}
>     ]
> }
>
> ./avrogencpp -i fixed.json -o fixed.hh
> Segmentation fault (core dumped)
>
> If I change the fixed.json to:
> {
>     "type": "record",
>     "name": "test_fixed",
>     "fields" : [
>         {"name": "foo", "type": "fixed", "size": 8}
>     ]
> }
>
> ./avrogencpp -i fixed.json -o fixed.hh
> Failed to parse or compile schema: Unknown type: fixed
>
>
> On 12/21/2012 03:43 AM, Martin Kleppmann wrote:
>
> If your numbers are typically small, you can just use a signed type —
> the sign bit's overhead is insignificant.
>
> If your numbers typically use the full range of 0 to 2^64-1 (or 0 to
> 2^32-1), e.g. because they are hashes or random numbers from that
> range, you're best off using the 'fixed' type and specifying the
> number of bytes you want. In this case 'fixed' is more efficient than
> the variable-length encoding of int/long, because there is no overhead
> for indicating the length; it's simply stored as that number of bytes,
> and nothing else.
>
> Because those two options cover most use cases, I don't think there
> are any plans to add unsigned int support to Avro.
>
> Martin
>
>
> On 20 December 2012 13:25, Robert Minsk <[EMAIL PROTECTED]> <[EMAIL PROTECTED]>
> wrote:
>
>  I am currently testing Avro for our network serialization for a mix of C++
> and python programs.  I have noticed that Avro does not offer an unsigned
> 32-bit or unsigned 64-bit integer types.  How are people currently handling
> unsigned integers?  Are there any plans to add unsigned integer types?
>
> --
> Robert Minsk
> Systems and Software Engineer
> WWW.METHODSTUDIOS.COM
> 730 Arizona Ave, Santa Monica, CA 90401
> O:+1 310 434 6500 // F:+1 310 434 6501
>
>
> --
>  Robert Minsk
>  Systems and Software Engineer
>
>  WWW.METHODSTUDIOS.COM <http://www.methodstudios.com>
>  730 Arizona Ave, Santa Monica, CA 90401
> O:+1 310 434 6500 <+13104346500> // F:+1 310 434 6501 <+13104346501>
>
>   [image: Los Angeles]<http://www.methodstudios.com/signature/url/los-angeles>
> <http://www.methodstudios.com/signature/url/los-angeles>
>
>