Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Sqoop Error Not replicated yet:
cmdr taco
2012-06-15, 17:03
cmdr taco
2012-06-18, 15:00
Copy link to this message
-
Re: Sqoop Error Not replicated yet:
Arvind Prabhakar 2012-06-18, 20:25
Hi,

This seems to be related to HDFS configuration and not necessarily a Sqoop
issue. Please check your NN/DN logs to see if there are other exceptions
that could give an idea of what is going on with your cluster.

Regards,
Arvind Prabhakar

On Mon, Jun 18, 2012 at 8:00 AM, cmdr taco <[EMAIL PROTECTED]> wrote:

> When Sqoop tries to import data into Hive, it creates a data file
> under.i.e. /export/home/myusername, before publishing the data to the hive
> table.
>
> I'm wondering if the $HOME should be part of the HDFS...? .I'm assuming
> HDFS is not able to replicate the data when using direct mode for high
> volume table.
>
> Thx
> Taco!!
>
>
>
> On Fri, Jun 15, 2012 at 10:03 AM, cmdr taco <[EMAIL PROTECTED]>wrote:
>
>> Experts!!
>>
>> I use sqoop to import data from RDBMS (postgres) to HIVE Table
>> and Occasionally the Job fails with below exception..any idea, why this
>> happens..What I noticed is the failure only happens with Larger Volume
>> tables.
>>
>> Log:
>>
>>
>>
>> 12/06/15 03:46:35 INFO hdfs.DFSClient:
>> org.apache.hadoop.ipc.RemoteException:
>> org.apache.hadoop.hdfs.server.namenode.NotReplicatedYetException: Not
>> replicated yet:/user/etl_user/fact_web_events/data-00000
>>         at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1401)
>>         at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588)
>>         at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
>>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
>>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)
>>
>>         at org.apache.hadoop.ipc.Client.call(Client.java:1054)
>>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>>         at $Proxy0.addBlock(Unknown Source)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>         at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>         at $Proxy0.addBlock(Unknown Source)
>>         at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3166)
>>         at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3036)
>>         at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1900(DFSClient.java:2288)
>>         at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2483)
>>
>>  12/06/15 03:46:35 WARN hdfs.DFSClient: NotReplicatedYetException
>> sleeping /user/etl_user/fact_web_events/data-00000 retries left 4
>>
>>
>