Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Issue against Hadoop-2.0.4-alpha


+
sam liu 2013-06-20, 09:36
+
sam liu 2013-06-21, 01:39
+
Abraham Elmahrek 2013-06-21, 05:37
+
sam liu 2013-07-16, 03:29
+
Jarek Jarcec Cecho 2013-07-16, 18:12
Copy link to this message
-
Re: Issue against Hadoop-2.0.4-alpha
Hi Jarek,

Below are my configurations:

1) Env Parameters:
export HADOOP_HOME=/opt/hadoop-2.0.3-alpha
export PATH=$HADOOP_HOME/bin:$PATH
export PATH=$HADOOP_HOME/sbin:$PATH
export HADOOP_MAPARED_HOME=${HADOOP_HOME}
export HADOOP_COMMON_HOME=${HADOOP_HOME}
export HADOOP_HDFS_HOME=${HADOOP_HOME}
export YARN_HOME=${HADOOP_HOME}
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export HDFS_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export YARN_CONF_DIR=${HADOOP_HOME}/etc/hadoop

2) hdfs-site.xml:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>

  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>

  <property>
    <name>dfs.name.dir</name>
    <value>/home/temp/hadoop/dfs_name_dir</value>
  </property>

  <property>
    <name>dfs.data.dir</name>
    <value>/home/temp/hadoop/dfs_data_dir</value>
  </property>

  <property>
    <name>dfs.webhdfs.enabled</name>
    <value>true</value>
  </property>
</configuration>
2013/7/17 Jarek Jarcec Cecho <[EMAIL PROTECTED]>

> Hi sir,
> the exception is suggesting that FileSystem implementation for your
> default FS can't be found. I would check HDFS configuration to ensure that
> it's configured properly and that Sqoop is properly picking the
> configuration and all the HDFS libraries.
>
> Jarcec
>
> On Tue, Jul 16, 2013 at 11:29:07AM +0800, sam liu wrote:
> > I also tried another sqoop build sqoop-1.4.2.bin__hadoop-2.0.0-alpha on
> > hadoop 2.0.3-alpha, but failed as well. The exception is same as above:
> > 'java.lang.UnsupportedOperationException: Not implemented by the
> > DistributedFileSystem FileSystem implementation'.
> >
> > This issue blocks me quite a while...
> >
> >
> > 2013/6/21 Abraham Elmahrek <[EMAIL PROTECTED]>
> >
> > > Hey Sam,
> > >
> > > My understanding is that Sqoop 1.4.3 should work with Hadoop 2.0.x
> (which
> > > would include Hadoop 2.0.4 alpha). Any ways, there seems to be some
> version
> > > conflicting going on here. Do you have any other builds of sqoop
> installed?
> > >
> > > -Abe
> > >
> > >
> > > On Thu, Jun 20, 2013 at 6:39 PM, sam liu <[EMAIL PROTECTED]>
> wrote:
> > >
> > >> Anyone could provide a answer? We are making decision whether to
> leverage
> > >> Sqoop 1.4.3 on yarn or not.
> > >>
> > >> Thanks!
> > >>
> > >>
> > >> 2013/6/20 sam liu <[EMAIL PROTECTED]>
> > >>
> > >>> Hi,
> > >>>
> > >>> Sqoop website says Sqoop 1.4.3 support Hadoop 2.0, but I failed to
> run
> > >>> import tool against hadoop-2.0.4-alpha, using
> > >>> sqoop-1.4.3.bin__hadoop-2.0.0-alpha. Can anyone help provide
> > >>> triage/suggestion? Thanks in advance!
> > >>>
> > >>> - Command:
> > >>> sqoop import --connect jdbc:db2://host:50000/SAMPLE --table
> > >>> DB2ADMIN.DB2TEST_TBL001 --username user --password pwd -m 1
> --target-dir
> > >>> /tmp/DB2TEST_TBL001
> > >>>
> > >>> - Exception:
> > >>> 13/06/19 23:28:28 INFO manager.SqlManager: Executing SQL statement:
> > >>> SELECT t.* FROM DB2ADMIN.DB2TEST_TBL001 AS t WHERE 1=0
> > >>> 13/06/19 23:28:28 ERROR sqoop.Sqoop: Got exception running Sqoop:
> > >>> java.lang.UnsupportedOperationException: Not implemented by the
> > >>> DistributedFileSystem FileSystem implementation
> > >>> java.lang.UnsupportedOperationException: Not implemented by the
> > >>> DistributedFileSystem FileSystem implementation
> > >>>         at
> org.apache.hadoop.fs.FileSystem.getScheme(FileSystem.java:207)
> > >>>         at
> > >>> org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2245)
> > >>>         at
> > >>>
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2255)
> > >>>         at
> > >>>
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2272)
> > >>>         at
> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:86)
> > >>>         at
> > >>>
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2311)
> > >>>         at
> > >>> org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2293)
Sam Liu
+
Jarek Jarcec Cecho 2013-07-17, 15:01
+
sam liu 2013-07-19, 05:41
+
sam liu 2013-07-22, 05:42
+
Abraham Elmahrek 2013-07-22, 07:08
+
Jarek Jarcec Cecho 2013-07-22, 14:44
+
sam liu 2013-07-24, 05:30
+
sam liu 2013-07-25, 03:07
+
Jarek Jarcec Cecho 2013-07-25, 15:27
+
sam liu 2013-07-26, 07:26
+
Jarek Jarcec Cecho 2013-07-26, 20:28
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB