Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> rename operation


Hi,

I would like to know if the *rename* operation (i.e. renaming a directory or
a single file) can be consider as an atomic operation in HDFS.

Basically what i am trying to achieve is having one process that
continiously add new file into the HDFS and another process that will start
every 15 minutes a map/reduce flow on file that were newly added into the
HDFS.

In other words a process A continuously read a *local directory
"A/in"*where new files are moved there continuously and put each file
in a
*"A/tmp" directory on the HDFS*. When A finish to put one file in
"*A/tmp"*it will
*move/rename that file into a "B/in" directory*. At the same time a process
B will, every 15 minutes, push all the files present in "B/in" to a
map/reduce flow.

Regards,

-- JP
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB