Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Hadoop - How to switch from implementing the writable interface to use an Avro object?


Copy link to this message
-
Hadoop - How to switch from implementing the writable interface to use an Avro object?
I’m using Hadoop to convert JSONs into CSV files to access them with Hive.

 

At the moment the Mapper is filling an own data structure reading the JSONs
with JSON-Smart. Then the reducer is reading out that object and is writing
it to a file, separated by commas.

For making this faster I already implemented the writable interface in the
data structure...

 

Now I want to use Avro for the data structure object to have more
flexibility and performance. How to change my classes to make them exchange
an Avro object instead of a writable?

 

 

Thanks

Tim Bittersohl

Software Engineer
http://www.innoplexia.de/ci/logo/inno_logo_links%20200x80.png

Innoplexia GmbH
Mannheimer Str. 175

69123 Heidelberg

Tel.: +49 (0) 6221 7198033
Fax: +49 (0) 6221 7198034
Web:  <http://www.innoplexia.com/> www.innoplexia.com

Sitz: 69123 Heidelberg, Mannheimer Str. 175 - Steuernummer 32494/62606 -
USt. IdNr.: DE 272 871 728 - Geschäftsführer: Christian Schneider, Walery
Strauch, Prof. Dr. Herbert Schuster

 

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB