Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Getting started with Avro + Reading from an Avro formatted file


+
selvi k 2012-01-24, 15:31
+
Douglas Creager 2012-01-24, 15:54
+
Harsh J 2012-01-24, 16:01
+
selvi k 2012-01-24, 19:37
+
selvi k 2012-01-24, 20:20
+
Harsh J 2012-01-24, 20:44
+
selvi k 2012-01-25, 02:46
+
Douglas Creager 2012-01-24, 21:00
+
selvi k 2012-01-25, 02:50
Copy link to this message
-
Re: Getting started with Avro + Reading from an Avro formatted file
Selvi,

(Forgot to reply to this before)

On Wed, Jan 25, 2012 at 1:07 AM, selvi k <[EMAIL PROTECTED]> wrote:
> 3. With regards to the two suggested ways, would either of these techniques
> allow me to filter my data records using some sort of a condition on a
> field?(or a few fields)  If not it seems like I would have to resort to
> first grepping the log file with the condition I want, and then using either
> of these two techniques to convert to CSV file. This would still be much
> better than what I am doing now, which is through not-so-pretty awk
> invocations to retrieve the fields I need (after the initial grep). But if
> the existing API, allows me to scan through the log file and specify
> conditions for fields, it might be much more efficient. I can imagine that I
> might have to use the low-level API and write a program to do this, but I am
> not sure at this point how to get started on this.

$ avro --help has some options that can help you out.

For "avro cat", the following may help:

--fields=FIELDS     fields to show, comma separated (show all by default)

But no, the utility does not provide a way to filter anything out. Its
a mere reader with some extensibility on fields/format. You'd have to
do filtering via your own full-fledged reader program, or via Bash
using "avro cat" and grep/etc.

--
Harsh J
Customer Ops. Engineer, Cloudera
+
selvi k 2012-01-25, 02:56
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB