Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - completebulkload does 'copy' StoreFiles instead of 'cut'


+
M. BagherEsmaeily 2013-09-14, 06:31
+
rajesh babu Chintaguntla 2013-09-14, 10:20
+
M. BagherEsmaeily 2013-09-15, 06:10
+
Ted Yu 2013-09-15, 07:26
+
M. BagherEsmaeily 2013-09-15, 08:09
+
Ted Yu 2013-09-15, 09:42
+
M. BagherEsmaeily 2013-09-15, 09:52
Copy link to this message
-
Re: completebulkload does 'copy' StoreFiles instead of 'cut'
Ted Yu 2013-09-15, 14:04
In the region server log file, do you see the following ?

      LOG.info("File " + srcPath + " on different filesystem than " +
          "destination store - moving to this filesystem.");
On Sun, Sep 15, 2013 at 2:52 AM, M. BagherEsmaeily <[EMAIL PROTECTED]>wrote:

> Actually that was my fault duplicating the issue.I had no permission to
> delete the second issue. sorry!
>
>
> On Sun, Sep 15, 2013 at 2:12 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Looks like HBASE-9538 is for the same issue.
> >
> > On Sep 15, 2013, at 1:09 AM, "M. BagherEsmaeily" <[EMAIL PROTECTED]>
> > wrote:
> >
> > > HBASE-9537
> > >
> > >
> > > On Sun, Sep 15, 2013 at 11:56 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >
> > >> File a JIRA for the issue ?
> > >>
> > >> On Sep 14, 2013, at 11:10 PM, "M. BagherEsmaeily" <
> [EMAIL PROTECTED]
> > >
> > >> wrote:
> > >>
> > >>> Hi rajesh,
> > >>>
> > >>> I use Hbase 0.94.11 and Hadoop 1.2.1. The file system of bulkload
> > >>> output directory and hbase cluster are the same, too.
> > >>>
> > >>> I've also coded a MapReduce job using HFileOutputFormat. When I use
> > >>> LoadIncrementalHFiles
> > >>> to move the output of my job to HBase table, it still copies instead
> of
> > >> cut.
> > >>>
> > >>> Thanks
> > >>>
> > >>>
> > >>> On Sat, Sep 14, 2013 at 2:50 PM, rajesh babu Chintaguntla <
> > >>> [EMAIL PROTECTED]> wrote:
> > >>>
> > >>>> Hi BagherEsmaeily,
> > >>>>
> > >>>>      which version of hbase you are using? Is the file system of
> > >> bulkload
> > >>>> output directory and hbase cluster are same?
> > >>>>
> > >>>>      If you are using hbase older than 0.94.5 version, the
> Storefiles
> > >>>> generated by importtsv are getting copied instead of        moving
> > even
> > >> if
> > >>>> the file system of bulkload output directory and hbase cluster are
> > same.
> > >>>>
> > >>>>      Its a bug and solved in 0.94.5 (HBASE-5498).
> > >>>>
> > >>>> Thanks.
> > >>>> Rajeshbabu
> > >>>>
> > >>>> On Sat, Sep 14, 2013 at 12:01 PM, M. BagherEsmaeily <
> > >> [EMAIL PROTECTED]
> > >>>>> wrote:
> > >>>>
> > >>>>> Hello,
> > >>>>>
> > >>>>> I was using HBase complete bulk load to transfer the output of
> > >> ImportTsv
> > >>>> to
> > >>>>> a table in HBase, and I noticed that it copies the output instead
> of
> > >>>>> cutting. This takes long time for my gigabytes of data.
> > >>>>>
> > >>>>> In HBase documentation (
> > >>>>> http://hbase.apache.org/book/ops_mgt.html#completebulkload) I read
> > >> that
> > >>>>> the
> > >>>>> files would be moved not copied. Can anyone help me with this?
> > >>>>>
> > >>>>> Kind Regards
> > >>
> >
>
+
lars hofhansl 2013-09-15, 22:23
+
M. BagherEsmaeily 2013-09-16, 05:50
+
M. BagherEsmaeily 2013-09-16, 06:33