I am looking for a relatively efficient way of transferring data between a
remote server and Hive without going through the hassle of storing the
data first on memory before loading it to Hive.
>From what I have read so far there is no such command but it would not
hurt to ask.
Is it possible to insert data through an insert query in hive? (The
equivalent to insert into table_name
values (...) in xSQLx)

Thank you in advance for an answer.
Cyrille Djoko
Data Mining Developer Intern

Hi Cyrille,
I'm not exactly sure what exactly you mean, so I'm more or less blindly shooting, but maybe Apache Sqoop [1] might help you?

Jarcec

Links:
1: http://sqoop.apache.org/

On Fri, Feb 15, 2013 at 01:44:45PM -0500, Cyrille Djoko wrote:
> I am looking for a relatively efficient way of transferring data between a
> remote server and Hive without going through the hassle of storing the
> data first on memory before loading it to Hive.
> From what I have read so far there is no such command but it would not
> hurt to ask.
> Is it possible to insert data through an insert query in hive? (The
> equivalent to insert into table_name
> values (...) in xSQLx)
>
> Thank you in advance for an answer.
>
>
> Cyrille Djoko
> Data Mining Developer Intern
>

Hi Jarcec,
I did try Sqoop. I am running sqoop 1.4.2 --hadoop1.0.0 along with hadoop
1.0.4 But I keep running on the following exception.

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
class org.apache.hadoop.mapreduce.JobContext, but interface was expected

So I wrote a small program but all I can do is send queries to the server.
> Hi Cyrille,
> I'm not exactly sure what exactly you mean, so I'm more or less blindly
> shooting, but maybe Apache Sqoop [1] might help you?
>
> Jarcec
>
> Links:
> 1: http://sqoop.apache.org/
>
> On Fri, Feb 15, 2013 at 01:44:45PM -0500, Cyrille Djoko wrote:
>> I am looking for a relatively efficient way of transferring data between
>> a
>> remote server and Hive without going through the hassle of storing the
>> data first on memory before loading it to Hive.
>> From what I have read so far there is no such command but it would not
>> hurt to ask.
>> Is it possible to insert data through an insert query in hive? (The
>> equivalent to insert into table_name
>> values (...) in xSQLx)
>>
>> Thank you in advance for an answer.
>>
>>
>> Cyrille Djoko
>> Data Mining Developer Intern
>>
>
Cyrille Djoko

Agnik LLC
Data Mining Developer Intern

When you want to move data from external system to hive, this means moving data to HDFS first and then point the Hive table to the file in HDFS where you have exported the data.
So, you have couple of commands like -copyFromLocal and fget which move the file to hdfs. If you intent to move in real time fashion try Flume. But end of the day the data movement first happens in HDFS and then hive table can be loaded using Load table command.

Regards,
Manish Bhoge
sent by HTC device. Excuse typo.

----- Reply message -----
From: "Cyrille Djoko" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Subject: Hive Queries
Date: Sat, Feb 16, 2013 1:50 AM
Hi Jarcec,
I did try Sqoop. I am running sqoop 1.4.2 --hadoop1.0.0 along with hadoop
1.0.4 But I keep running on the following exception.

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
class org.apache.hadoop.mapreduce.JobContext, but interface was expected

So I wrote a small program but all I can do is send queries to the server.
> Hi Cyrille,
> I'm not exactly sure what exactly you mean, so I'm more or less blindly
> shooting, but maybe Apache Sqoop [1] might help you?
>
> Jarcec
>
> Links:
> 1: http://sqoop.apache.org/
>
> On Fri, Feb 15, 2013 at 01:44:45PM -0500, Cyrille Djoko wrote:
>> I am looking for a relatively efficient way of transferring data between
>> a
>> remote server and Hive without going through the hassle of storing the
>> data first on memory before loading it to Hive.
>> From what I have read so far there is no such command but it would not
>> hurt to ask.
>> Is it possible to insert data through an insert query in hive? (The
>> equivalent to insert into table_name
>> values (...) in xSQLx)
>>
>> Thank you in advance for an answer.
>>
>>
>> Cyrille Djoko
>> Data Mining Developer Intern
>>
>
Cyrille Djoko

Agnik LLC
Data Mining Developer Intern

Dude sorry for the off topic, but having a rocketmail account is
awesome. I wish I still had mine.

On Sat, Feb 16, 2013 at 9:16 PM, [EMAIL PROTECTED]
<[EMAIL PROTECTED]> wrote:
>
> When you want to move data from external system to hive, this means moving
> data to HDFS first and then point the Hive table to the file in HDFS where
> you have exported the data.
> So, you have couple of commands like -copyFromLocal and fget which move the
> file to hdfs. If you intent to move in real time fashion try Flume. But end
> of the day the data movement first happens in HDFS and then hive table can
> be loaded using Load table command.
>
> Regards,
> Manish Bhoge
> sent by HTC device. Excuse typo.
>
> ----- Reply message -----
> From: "Cyrille Djoko" <[EMAIL PROTECTED]>
> To: <[EMAIL PROTECTED]>
> Subject: Hive Queries
> Date: Sat, Feb 16, 2013 1:50 AM
>
>
> Hi Jarcec,
> I did try Sqoop. I am running sqoop 1.4.2 --hadoop1.0.0 along with hadoop
> 1.0.4 But I keep running on the following exception.
>
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> class org.apache.hadoop.mapreduce.JobContext, but interface was expected
>
> So I wrote a small program but all I can do is send queries to the server.
>> Hi Cyrille,
>> I'm not exactly sure what exactly you mean, so I'm more or less blindly
>> shooting, but maybe Apache Sqoop [1] might help you?
>>
>> Jarcec
>>
>> Links:
>> 1: http://sqoop.apache.org/
>>
>> On Fri, Feb 15, 2013 at 01:44:45PM -0500, Cyrille Djoko wrote:
>>> I am looking for a relatively efficient way of transferring data between
>>> a
>>> remote server and Hive without going through the hassle of storing the
>>> data first on memory before loading it to Hive.
>>> From what I have read so far there is no such command but it would not
>>> hurt to ask.
>>> Is it possible to insert data through an insert query in hive? (The
>>> equivalent to insert into table_name
>>> values (...) in xSQLx)
>>>
>>> Thank you in advance for an answer.
>>>
>>>
>>> Cyrille Djoko
>>> Data Mining Developer Intern
>>>
>>
>
>
> Cyrille Djoko
>
> Agnik LLC
> Data Mining Developer Intern
>

Hi Manish,

If you have data on your local file system,
You can also do something like following from your local file system, without doing put or copyFromLocal.

$ hive -e "load data local inpath 'path on local file system' into table <table name>;"
Thanks,
Anurag Tangri
Sent from my iPhone

On Feb 16, 2013, at 6:16 PM, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:

>
> When you want to move data from external system to hive, this means moving data to HDFS first and then point the Hive table to the file in HDFS where you have exported the data.
> So, you have couple of commands like -copyFromLocal and fget which move the file to hdfs. If you intent to move in real time fashion try Flume. But end of the day the data movement first happens in HDFS and then hive table can be loaded using Load table command.
>
> Regards,
> Manish Bhoge
> sent by HTC device. Excuse typo.
>
> ----- Reply message -----
> From: "Cyrille Djoko" <[EMAIL PROTECTED]>
> To: <[EMAIL PROTECTED]>
> Subject: Hive Queries
> Date: Sat, Feb 16, 2013 1:50 AM
>
>
> Hi Jarcec,
> I did try Sqoop. I am running sqoop 1.4.2 --hadoop1.0.0 along with hadoop
> 1.0.4 But I keep running on the following exception.
>
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> class org.apache.hadoop.mapreduce.JobContext, but interface was expected
>
> So I wrote a small program but all I can do is send queries to the server.
> > Hi Cyrille,
> > I'm not exactly sure what exactly you mean, so I'm more or less blindly
> > shooting, but maybe Apache Sqoop [1] might help you?
> >
> > Jarcec
> >
> > Links:
> > 1: http://sqoop.apache.org/
> >
> > On Fri, Feb 15, 2013 at 01:44:45PM -0500, Cyrille Djoko wrote:
> >> I am looking for a relatively efficient way of transferring data between
> >> a
> >> remote server and Hive without going through the hassle of storing the
> >> data first on memory before loading it to Hive.
> >> From what I have read so far there is no such command but it would not
> >> hurt to ask.
> >> Is it possible to insert data through an insert query in hive? (The
> >> equivalent to insert into table_name
> >> values (...) in xSQLx)
> >>
> >> Thank you in advance for an answer.
> >>
> >>
> >> Cyrille Djoko
> >> Data Mining Developer Intern
> >>
> >
>
>
> Cyrille Djoko
>
> Agnik LLC
> Data Mining Developer Intern
>

Hi everyone,

My setup is Hadoop 1.0.4, Hive 0.9.0, Sqoop 1.4.2-hadoop 1.0.0
Mahout 0.7

I have imported tables from a remote database directly into Hive using
Sqoop.

Somehow when I try to run Sqoop from Hadoop, the content

Hive is giving me trouble in bookkeeping of where the imported tables are
located. I have a Single Node setup.

Thank you for any answer and you can ask question if I was not specific
enough about my issue.

Cyril

any errors you see ?
On Mon, Feb 25, 2013 at 8:48 PM, Cyril Bogus <[EMAIL PROTECTED]> wrote:

> Hi everyone,
>
> My setup is Hadoop 1.0.4, Hive 0.9.0, Sqoop 1.4.2-hadoop 1.0.0
> Mahout 0.7
>
> I have imported tables from a remote database directly into Hive using
> Sqoop.
>
> Somehow when I try to run Sqoop from Hadoop, the content
>
> Hive is giving me trouble in bookkeeping of where the imported tables are
> located. I have a Single Node setup.
>
> Thank you for any answer and you can ask question if I was not specific
> enough about my issue.
>
> Cyril
>

--
Nitin Pawar

I do not get any errors.
It is only when I run hive and try to query the tables I imported. Let's
say I want to only get numeric tuples for a given table. I cannot find the
table (show tables; is empty) unless I go in the hive home folder and run
hive again. I would expect the state of hive to be the same everywhere I
call it.
But so far it is not the case.
On Mon, Feb 25, 2013 at 10:22 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:

> any errors you see ?
>
>
> On Mon, Feb 25, 2013 at 8:48 PM, Cyril Bogus <[EMAIL PROTECTED]> wrote:
>
>> Hi everyone,
>>
>> My setup is Hadoop 1.0.4, Hive 0.9.0, Sqoop 1.4.2-hadoop 1.0.0
>> Mahout 0.7
>>
>> I have imported tables from a remote database directly into Hive using
>> Sqoop.
>>
>> Somehow when I try to run Sqoop from Hadoop, the content
>>
>> Hive is giving me trouble in bookkeeping of where the imported tables are
>> located. I have a Single Node setup.
>>
>> Thank you for any answer and you can ask question if I was not specific
>> enough about my issue.
>>
>> Cyril
>>
>
>
>
> --
> Nitin Pawar
>

Hi Cyril

I believe you are using the derby meta store and then it should be an issue with the hive configs.

Derby is trying to create a metastore at your current dir from where you are starting hive. The tables exported by sqoop would be inside HIVE_HOME and hence you are not able to see the tables from getting on to hive CLI from other locations.

To have a universal metastore db configure a specific dir in javax.jdo.option.ConnectionURL in hive-site.xml . In your conn url configure the db name as "databaseName=/home/hive/metastore_db"

Regards
Bejoy KS

Sent from remote device, Please excuse typos

-----Original Message-----
From: Cyril Bogus <[EMAIL PROTECTED]>
Date: Mon, 25 Feb 2013 10:34:29
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Hive queries

I do not get any errors.
It is only when I run hive and try to query the tables I imported. Let's
say I want to only get numeric tuples for a given table. I cannot find the
table (show tables; is empty) unless I go in the hive home folder and run
hive again. I would expect the state of hive to be the same everywhere I
call it.
But so far it is not the case.
On Mon, Feb 25, 2013 at 10:22 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:

> any errors you see ?
>
>
> On Mon, Feb 25, 2013 at 8:48 PM, Cyril Bogus <[EMAIL PROTECTED]> wrote:
>
>> Hi everyone,
>>
>> My setup is Hadoop 1.0.4, Hive 0.9.0, Sqoop 1.4.2-hadoop 1.0.0
>> Mahout 0.7
>>
>> I have imported tables from a remote database directly into Hive using
>> Sqoop.
>>
>> Somehow when I try to run Sqoop from Hadoop, the content
>>
>> Hive is giving me trouble in bookkeeping of where the imported tables are
>> located. I have a Single Node setup.
>>
>> Thank you for any answer and you can ask question if I was not specific
>> enough about my issue.
>>
>> Cyril
>>
>
>
>
> --
> Nitin Pawar
>

Thank you so much Bejoy,
That was my issue.
Now that I saw the config file I see that I was the one needing a universal
database.

Thanks again,
Regards
Cyril
On Mon, Feb 25, 2013 at 10:47 AM, <[EMAIL PROTECTED]> wrote:

> **
> Hi Cyril
>
> I believe you are using the derby meta store and then it should be an
> issue with the hive configs.
>
> Derby is trying to create a metastore at your current dir from where you
> are starting hive. The tables exported by sqoop would be inside HIVE_HOME
> and hence you are not able to see the tables from getting on to hive CLI
> from other locations.
>
> To have a universal metastore db configure a specific dir in
> javax.jdo.option.ConnectionURL in hive-site.xml . In your conn url
> configure the db name as "databaseName=/home/hive/metastore_db"
> Regards
> Bejoy KS
>
> Sent from remote device, Please excuse typos
> ------------------------------
> *From: * Cyril Bogus <[EMAIL PROTECTED]>
> *Date: *Mon, 25 Feb 2013 10:34:29 -0500
> *To: *<[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Hive queries
>
> I do not get any errors.
> It is only when I run hive and try to query the tables I imported. Let's
> say I want to only get numeric tuples for a given table. I cannot find the
> table (show tables; is empty) unless I go in the hive home folder and run
> hive again. I would expect the state of hive to be the same everywhere I
> call it.
> But so far it is not the case.
>
>
> On Mon, Feb 25, 2013 at 10:22 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>
>> any errors you see ?
>>
>>
>> On Mon, Feb 25, 2013 at 8:48 PM, Cyril Bogus <[EMAIL PROTECTED]>wrote:
>>
>>> Hi everyone,
>>>
>>> My setup is Hadoop 1.0.4, Hive 0.9.0, Sqoop 1.4.2-hadoop 1.0.0
>>> Mahout 0.7
>>>
>>> I have imported tables from a remote database directly into Hive using
>>> Sqoop.
>>>
>>> Somehow when I try to run Sqoop from Hadoop, the content
>>>
>>> Hive is giving me trouble in bookkeeping of where the imported tables
>>> are located. I have a Single Node setup.
>>>
>>> Thank you for any answer and you can ask question if I was not specific
>>> enough about my issue.
>>>
>>> Cyril
>>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>