Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Trevni status


+
Ken Krugler 2013-03-01, 20:37
Copy link to this message
-
Re: Trevni status
Ken,

Trevni is alive and well.  It's been included in the last two Avro
releases, 1.7.3 and 1.7.4.  There were some good improvements to
Trevni in 1.7.4, so use that version if possible.

Patches that add Trevni support to Pig and Hive are available:

  https://issues.apache.org/jira/browse/PIG-3015
  https://issues.apache.org/jira/browse/HIVE-3585

If you already use Avro then Trevni's easy to incorporate.  For
MapReduce jobs, you can write Trevni output from a program that
produced Avro before by simply changing the OutputFormat.  Similarly,
to read Trevni input in MapReduce simply change the InputFormat and
specify a subset schema (deleting fields you don't need, i.e.,
projecting).

So it shouldn't be hard to use Trevni with Cascading.  If you work on
this, please let us know how it goes.

Cheers,

Doug

On Fri, Mar 1, 2013 at 12:37 PM, Ken Krugler
<[EMAIL PROTECTED]> wrote:
> Hi all,
>
> Any input as to the status of Trevni?
>
> I'm researching column-oriented file formats that aren't tightly coupled to
> specific platforms - this precludes ORCFile, for example.
>
> CIF seemed interesting, but IBM hasn't released the code. And Trevni seems
> to be a reasonable open source implementation of what they describe.
>
> But I hadn't heard much about Trevni recently, or if anybody is using it for
> real work.
>
> I see it mentioned in conjunction with Impala, but it sounds like that's on
> the roadmap versus being available yet.
>
> For context, I'm looking into using a column store to speed up Cascading
> workflows.
>
> Thanks,
>
> -- Ken
>
> PS - resending to user@, since the first email to dev@ seems to have
> disappeared…maybe I'm no longer getting dev@ emails?
>
> --------------------------
> Ken Krugler
> +1 530-210-6378
> http://www.scaleunlimited.com
> custom big data solutions & training
> Hadoop, Cascading, Cassandra & Solr
>
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB