Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Getting started with Avro + Reading from an Avro formatted file

selvi k 2012-01-24, 15:31
Douglas Creager 2012-01-24, 15:54
Harsh J 2012-01-24, 16:01
selvi k 2012-01-24, 19:37
selvi k 2012-01-24, 20:20
Harsh J 2012-01-24, 20:44
selvi k 2012-01-25, 02:46
Douglas Creager 2012-01-24, 21:00
selvi k 2012-01-25, 02:50
Copy link to this message
Re: Getting started with Avro + Reading from an Avro formatted file

(Forgot to reply to this before)

On Wed, Jan 25, 2012 at 1:07 AM, selvi k <[EMAIL PROTECTED]> wrote:
> 3. With regards to the two suggested ways, would either of these techniques
> allow me to filter my data records using some sort of a condition on a
> field?(or a few fields)  If not it seems like I would have to resort to
> first grepping the log file with the condition I want, and then using either
> of these two techniques to convert to CSV file. This would still be much
> better than what I am doing now, which is through not-so-pretty awk
> invocations to retrieve the fields I need (after the initial grep). But if
> the existing API, allows me to scan through the log file and specify
> conditions for fields, it might be much more efficient. I can imagine that I
> might have to use the low-level API and write a program to do this, but I am
> not sure at this point how to get started on this.

$ avro --help has some options that can help you out.

For "avro cat", the following may help:

--fields=FIELDS     fields to show, comma separated (show all by default)

But no, the utility does not provide a way to filter anything out. Its
a mere reader with some extensibility on fields/format. You'd have to
do filtering via your own full-fledged reader program, or via Bash
using "avro cat" and grep/etc.

Harsh J
Customer Ops. Engineer, Cloudera
selvi k 2012-01-25, 02:56