Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Hadoop property precedence


Copy link to this message
-
Re: Hadoop property precedence
Hi Shalish,

The client side conf will take precedence. Further you can use the FileSystem API which can set the block size:
create(Path f, boolean overwrite, int bufferSize, short replication, long blockSize)

Cheers,
Subroto Sanyal

On Jul 13, 2013, at 9:10 PM, Shalish VJ wrote:

> Hi,
>  
> Ru sure?
> or have u ever tried it out.
> Pls advice.
>
> From: Azuryy Yu <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Saturday, July 13, 2013 3:19 PM
> Subject: Re: Hadoop property precedence
>
> the conf that client running on will take effect.
> On Jul 13, 2013 4:42 PM, "Kiran Dangeti" <[EMAIL PROTECTED]> wrote:
> Shalish,
>  
> The default block size is 64MB which is good at the client end. Make sure the same at your end also in conf. You can increase the size of each block to 128MB or greater than that only thing you can see the processing will be fast but at end there may be  chances of losing data.
>  
> Thanks,
> Kiran
>
>
> On Fri, Jul 12, 2013 at 10:20 PM, Shalish VJ <[EMAIL PROTECTED]> wrote:
> Hi,
>  
>  
>     Suppose block size set in configuration file at client side is 64MB,
> block size set in configuration file at name node side is 128MB and block size set in configuration file at datanode side is something else.
> Please advice, If the client is writing a file to hdfs,which property would be executed.
>  
> Thanks,
> Shalish.
>
>
>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB