-Re: Does Sqoop support importing non-relational data to Hadoop
Chalcy 2012-11-28, 16:03
We have used flume to collect files from windows using old flume (0.94, I
believe). There is a flume windows agent. It was working well but did not
scale well for us. When I tested collecting from windows 5 servers, it
worked well. With like 20, collector would fail ans stop collecting data.
There is no windows agent in flume-ng (may be there now) when I checked few
On Wed, Nov 28, 2012 at 10:45 AM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
> Hi Jia,
> thank you very much for your questions. Sqoop is designed as a batch tool
> transfer data from database and warehouse systems to Hadoop ecosystem and
> vice-versa. Right now, Sqoop is supporting only JDBC compliant databases.
> requirement will however fade away with Sqoop 2.
> Based on your questions, it seems to me that you're more looking for an
> ingest system rather than batch one. In such case, I would recommend
> Apache Flume project  that aim to address online data ingesting issue.
> 1: http://flume.apache.org/
> On Wed, Nov 28, 2012 at 02:19:57PM +0800, jia jimin wrote:
> > Hi there,
> > I am investigating Sqoop on windows for importing data to HDFS and have
> > some questions :
> > 1. Does Sqoop support importing non-relational data such as event log or
> > text file to HDFS ?
> > 2. If our client machine changed frequently ( recycle old machines and
> > new machines) , Can sqoop automatically import data by changing some
> > configurations dynamically ?
> > Thanks for looking at these questions !
> > Regards
> > Benjamin