Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Hadoop property precedence


+
Shalish VJ 2013-07-12, 16:50
+
Shumin Guo 2013-07-13, 23:21
+
varun kumar 2013-07-14, 10:44
+
Harsh J 2013-07-14, 10:54
+
Shekhar Sharma 2013-07-14, 17:32
+
Kiran Dangeti 2013-07-13, 08:42
+
Azuryy Yu 2013-07-13, 09:49
+
Shalish VJ 2013-07-13, 19:10
Copy link to this message
-
Re: Hadoop property precedence
Hi Shalish,

The client side conf will take precedence. Further you can use the FileSystem API which can set the block size:
create(Path f, boolean overwrite, int bufferSize, short replication, long blockSize)

Cheers,
Subroto Sanyal

On Jul 13, 2013, at 9:10 PM, Shalish VJ wrote:

> Hi,
>  
> Ru sure?
> or have u ever tried it out.
> Pls advice.
>
> From: Azuryy Yu <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Saturday, July 13, 2013 3:19 PM
> Subject: Re: Hadoop property precedence
>
> the conf that client running on will take effect.
> On Jul 13, 2013 4:42 PM, "Kiran Dangeti" <[EMAIL PROTECTED]> wrote:
> Shalish,
>  
> The default block size is 64MB which is good at the client end. Make sure the same at your end also in conf. You can increase the size of each block to 128MB or greater than that only thing you can see the processing will be fast but at end there may be  chances of losing data.
>  
> Thanks,
> Kiran
>
>
> On Fri, Jul 12, 2013 at 10:20 PM, Shalish VJ <[EMAIL PROTECTED]> wrote:
> Hi,
>  
>  
>     Suppose block size set in configuration file at client side is 64MB,
> block size set in configuration file at name node side is 128MB and block size set in configuration file at datanode side is something else.
> Please advice, If the client is writing a file to hdfs,which property would be executed.
>  
> Thanks,
> Shalish.
>
>
>

+
Shalish VJ 2013-07-13, 07:59
+
Raj K Singh 2013-07-12, 16:54
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB