Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> avro accessing record within an array in C


Copy link to this message
-
Re: avro accessing record within an array in C
Thanks a ton Mika.
I was able to loop through and get my code correct.
below is the snippet

So today I can do this
1) Given an XML - convert it to json  and further to avro json using a
python script
2) fill the avro contents using a C program and serialize it
3) de-serialise it using either a C or python program.

e.g.

    {
        const char* name = 0;
        int field_index= 0;
        int i=0, j=0, k=0;
        size_t field_count = 0;
        size_t array_count = 0;
        size_t rec_count = 0;
        avro_value_t field, subfield, rec, subrec, array, subarray;
        avro_type_t  type;
        avro_value_get_size(&value, &field_count);
        printf("field count is %d \n", field_count);
        for(i = 0; i < field_count; i++) {
            avro_value_get_by_index(&value, i, &field, &name);
            type = avro_value_get_type(&field);
            printf(" field name is : %s , type =", name);
            print_type(type);
            if(type == AVRO_ARRAY ) {
                avro_value_get_size(&field, &array_count);
                printf(" array number of elements = %d \n", array_count);
                for(j = 0; j < array_count; j++) {
                    avro_value_get_by_index(&field, j, &subfield, &name),
                    printf(" field name is : %s , type =", name);
                    type = avro_value_get_type(&subfield);
                    print_type(type);
                    printf("\n");
                    if(type == AVRO_RECORD) {
                        avro_value_get_size(&subfield, &rec_count);
                        printf(" record number of elements = %d \n",
rec_count);
                        for(k = 0; k < rec_count; k++) {
                            avro_value_get_by_index(&subfield, k, &subrec,
&name),
                            printf(" field name is : %s , type =", name);
                            type = avro_value_get_type(&subfield);
                            print_type(type);
                            printf("\n");
                        }
                    }
                }
            } else if(type == AVRO_RECORD) {
                avro_value_get_size(&field, &rec_count);
                printf(" record number of elements = %d \n", rec_count);
                for(k = 0; k < rec_count; k++) {
                    avro_value_get_by_index(&field, k, &subrec, &name),
                    printf(" field name is : %s , type =", name);
                    type = avro_value_get_type(&field);
                    print_type(type);
                    printf("\n");
                }
            }
            printf("\n");
        }
    }

On Wed, Sep 25, 2013 at 2:46 PM, Mika Ristimaki <[EMAIL PROTECTED]>wrote:

> Hi,
>
> I haven't tried that but it could be done something like this
>
> avro_schema_from_json(some_schema, strlen(some_schema), &schema, &error)
> iface = avro_generic_class_from_schema(schema);
> avro_generic_value_new(iface, &value);
>
> const char *field_name = avro_schema_record_field_name(&value, index);
>
> Then compare
>
> -Mika
>
> On Sep 25, 2013, at 11:18 AM, Mahesh V <[EMAIL PROTECTED]>
> wrote:
>
> > Thanks Mika,
> > It  worked like a charm.
> >
> > I have another question.
> > If given a field name "ip_v4" how can I query the json to reach the
> right place without knowing the names?
> > i.e. I want to get each field by index and check its name to see if it
> matches the one provided.
> >
> > e.g.  pseudo code
> > search_json(schema,  name) {
> >    val = get_value_by_index( 0)
> >    if strcmp(val->name, name) {
> >       I found my field;
> >    }
> > }
> >
> > Is this possible at all?
> > thanks
> > Mahesh
> >
> >
> >
> >
> > On Wed, Sep 25, 2013 at 12:43 AM, Mika Ristimaki <
> [EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > I can't really help with the datum api since I haven't used it myself,
> but with newer avro_value api arrays should work like this:
> >
> > avro_value_iface_t  *iface;
> > avro_value_t value, array, element, field;