This scenario i am thinking is to: 1) output to local file system(like Linux ) instead of hdfs 2) each regserver only output its only data to it's node's file system
To elaborate the 2) a bit. Basically, this will be like export Hbase data to local file system without going through network. And on each node, one file will be created.
Is there a way to achieve it? Actually the receiving side of 1) doesn't have to be a file system , it can be another process to process the data. But let's use file system to simplify the scenario for now.
This sounds an awful lot like a map-only MR job... With Hadoop Streaming, you should be able to achieve your goal of piping to an arbitrary process. On Tue, Aug 19, 2014 at 4:26 PM, Demai Ni <[EMAIL PROTECTED]> wrote:
A coprocessor is certainly possible. You haven't shared your motivation, only a specific implementation, so I cannot assist further. On Tue, Aug 19, 2014 at 6:28 PM, Demai Ni <[EMAIL PROTECTED]> wrote:
I am not sure exactly the use case yet, just doing some experiment. Current idea is to have a join with data from a mpp database, and have a program from mpp run on each node of Hbase, so instead of get a collection of all data, the join operation can occur at each regserver lever. Actually join may not be a good example here. The idea is to access data at regserver level but still be able to leverage Hbase filters.
Demai on the run
On Aug 19, 2014, at 7:39 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext