Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> avro accessing record within an array in C


Copy link to this message
-
Re: avro accessing record within an array in C
Thanks a ton Mika.
I was able to loop through and get my code correct.
below is the snippet

So today I can do this
1) Given an XML - convert it to json  and further to avro json using a
python script
2) fill the avro contents using a C program and serialize it
3) de-serialise it using either a C or python program.

e.g.

    {
        const char* name = 0;
        int field_index= 0;
        int i=0, j=0, k=0;
        size_t field_count = 0;
        size_t array_count = 0;
        size_t rec_count = 0;
        avro_value_t field, subfield, rec, subrec, array, subarray;
        avro_type_t  type;
        avro_value_get_size(&value, &field_count);
        printf("field count is %d \n", field_count);
        for(i = 0; i < field_count; i++) {
            avro_value_get_by_index(&value, i, &field, &name);
            type = avro_value_get_type(&field);
            printf(" field name is : %s , type =", name);
            print_type(type);
            if(type == AVRO_ARRAY ) {
                avro_value_get_size(&field, &array_count);
                printf(" array number of elements = %d \n", array_count);
                for(j = 0; j < array_count; j++) {
                    avro_value_get_by_index(&field, j, &subfield, &name),
                    printf(" field name is : %s , type =", name);
                    type = avro_value_get_type(&subfield);
                    print_type(type);
                    printf("\n");
                    if(type == AVRO_RECORD) {
                        avro_value_get_size(&subfield, &rec_count);
                        printf(" record number of elements = %d \n",
rec_count);
                        for(k = 0; k < rec_count; k++) {
                            avro_value_get_by_index(&subfield, k, &subrec,
&name),
                            printf(" field name is : %s , type =", name);
                            type = avro_value_get_type(&subfield);
                            print_type(type);
                            printf("\n");
                        }
                    }
                }
            } else if(type == AVRO_RECORD) {
                avro_value_get_size(&field, &rec_count);
                printf(" record number of elements = %d \n", rec_count);
                for(k = 0; k < rec_count; k++) {
                    avro_value_get_by_index(&field, k, &subrec, &name),
                    printf(" field name is : %s , type =", name);
                    type = avro_value_get_type(&field);
                    print_type(type);
                    printf("\n");
                }
            }
            printf("\n");
        }
    }

On Wed, Sep 25, 2013 at 2:46 PM, Mika Ristimaki <[EMAIL PROTECTED]>wrote:

> Hi,
>
> I haven't tried that but it could be done something like this
>
> avro_schema_from_json(some_schema, strlen(some_schema), &schema, &error)
> iface = avro_generic_class_from_schema(schema);
> avro_generic_value_new(iface, &value);
>
> const char *field_name = avro_schema_record_field_name(&value, index);
>
> Then compare
>
> -Mika
>
> On Sep 25, 2013, at 11:18 AM, Mahesh V <[EMAIL PROTECTED]>
> wrote:
>
> > Thanks Mika,
> > It  worked like a charm.
> >
> > I have another question.
> > If given a field name "ip_v4" how can I query the json to reach the
> right place without knowing the names?
> > i.e. I want to get each field by index and check its name to see if it
> matches the one provided.
> >
> > e.g.  pseudo code
> > search_json(schema,  name) {
> >    val = get_value_by_index( 0)
> >    if strcmp(val->name, name) {
> >       I found my field;
> >    }
> > }
> >
> > Is this possible at all?
> > thanks
> > Mahesh
> >
> >
> >
> >
> > On Wed, Sep 25, 2013 at 12:43 AM, Mika Ristimaki <
> [EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > I can't really help with the datum api since I haven't used it myself,
> but with newer avro_value api arrays should work like this:
> >
> > avro_value_iface_t  *iface;
> > avro_value_t value, array, element, field;
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB