Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Hadoop - How to switch from implementing the writable interface to use an Avro object?


Copy link to this message
-
Hadoop - How to switch from implementing the writable interface to use an Avro object?
Tim Bittersohl 2013-02-11, 15:40
I’m using Hadoop to convert JSONs into CSV files to access them with Hive.

 

At the moment the Mapper is filling an own data structure reading the JSONs
with JSON-Smart. Then the reducer is reading out that object and is writing
it to a file, separated by commas.

For making this faster I already implemented the writable interface in the
data structure...

 

Now I want to use Avro for the data structure object to have more
flexibility and performance. How to change my classes to make them exchange
an Avro object instead of a writable?

 

 

Thanks

Tim Bittersohl

Software Engineer
http://www.innoplexia.de/ci/logo/inno_logo_links%20200x80.png

Innoplexia GmbH
Mannheimer Str. 175

69123 Heidelberg

Tel.: +49 (0) 6221 7198033
Fax: +49 (0) 6221 7198034
Web:  <http://www.innoplexia.com/> www.innoplexia.com

Sitz: 69123 Heidelberg, Mannheimer Str. 175 - Steuernummer 32494/62606 -
USt. IdNr.: DE 272 871 728 - Geschäftsführer: Christian Schneider, Walery
Strauch, Prof. Dr. Herbert Schuster