Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop Error Not replicated yet:


Copy link to this message
-
Re: Sqoop Error Not replicated yet:
Hi,

This seems to be related to HDFS configuration and not necessarily a Sqoop
issue. Please check your NN/DN logs to see if there are other exceptions
that could give an idea of what is going on with your cluster.

Regards,
Arvind Prabhakar

On Mon, Jun 18, 2012 at 8:00 AM, cmdr taco <[EMAIL PROTECTED]> wrote:

> When Sqoop tries to import data into Hive, it creates a data file
> under.i.e. /export/home/myusername, before publishing the data to the hive
> table.
>
> I'm wondering if the $HOME should be part of the HDFS...? .I'm assuming
> HDFS is not able to replicate the data when using direct mode for high
> volume table.
>
> Thx
> Taco!!
>
>
>
> On Fri, Jun 15, 2012 at 10:03 AM, cmdr taco <[EMAIL PROTECTED]>wrote:
>
>> Experts!!
>>
>> I use sqoop to import data from RDBMS (postgres) to HIVE Table
>> and Occasionally the Job fails with below exception..any idea, why this
>> happens..What I noticed is the failure only happens with Larger Volume
>> tables.
>>
>> Log:
>>
>>
>>
>> 12/06/15 03:46:35 INFO hdfs.DFSClient:
>> org.apache.hadoop.ipc.RemoteException:
>> org.apache.hadoop.hdfs.server.namenode.NotReplicatedYetException: Not
>> replicated yet:/user/etl_user/fact_web_events/data-00000
>>         at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1401)
>>         at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588)
>>         at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
>>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
>>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)
>>
>>         at org.apache.hadoop.ipc.Client.call(Client.java:1054)
>>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>>         at $Proxy0.addBlock(Unknown Source)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>         at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>         at $Proxy0.addBlock(Unknown Source)
>>         at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3166)
>>         at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3036)
>>         at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1900(DFSClient.java:2288)
>>         at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2483)
>>
>>  12/06/15 03:46:35 WARN hdfs.DFSClient: NotReplicatedYetException
>> sleeping /user/etl_user/fact_web_events/data-00000 retries left 4
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB