Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: Mapping MySQL schema to Avro


Copy link to this message
-
Re: Mapping MySQL schema to Avro
Ruslan Al-Fakikh 2012-11-24, 00:06
Hey Bart,

Why not using Sqoop for trasforming data from/to hadoop to/from MySQL?
http://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html
see the parameter: -as-avrodatafile

Ruslan
On Sat, Nov 24, 2012 at 2:20 AM, Ted Dunning <[EMAIL PROTECTED]> wrote:

> This is probably the wrong list for your question.
>
> And, no, I don't think that your conversion is correct.
>
> To me it looks like you have lots of values in a fmsession.  In the avro
> version, those values appear to be repeated in the fmsswitchvalues. That
> seems wrong.
>
>
> On Fri, Nov 23, 2012 at 5:12 AM, Bart Verwilst <[EMAIL PROTECTED]> wrote:
>
>> Hello!
>>
>> I'm currently writing an importer to import our MySQL data into hadoop (
>> as Avro files ). Attached you can find the schema i'm converting to Avro,
>> along with the corresponding Avro schema i would like to use for my
>> imported data. I was wondering if you guys could go over the schema and
>> determine if this is sane/optimal, and if not, how i should improve it.
>>
>> As a sidenote, i converted bigints to long, and had 1 occurrence of
>> double, which i also converted to long in the avro, not sure if that's the
>> correct type?
>>
>> Thanks in advance for your expert opinions! ;)
>>
>> Kind regards,
>>
>> Bart
>>
>
>