|
registration@...
2012-07-09, 19:17
Amandeep Khurana
2012-07-09, 19:21
registration@...
2012-07-09, 21:30
Dhaval Shah
2012-07-09, 21:36
registration@...
2012-07-09, 21:56
Dhaval Shah
2012-07-09, 22:03
Suraj Varma
2012-07-10, 22:35
|
-
HBASE -- YCSB ?registration@... 2012-07-09, 19:17
Now that I have a stable cluster, I would like to use YCSB to test its performance; however, I am a bit confused after reading several different website posting about YCSB. 1) Be default will YCSB read my hbase-site.xml file or do I have to copy it into the YCSB conf directory? I plan on using on of my nodes with no Hadoop/HBASE processes running on it, but it has all the environmental stuff in place. 2) Does the hbase.master property have to be site in the hbase-site.xml file for YCSB to work? 3) After working through all the workloads is there a script/tool that will clean up my HBase? Thank You --- Jay Wilson
-
Re: HBASE -- YCSB ?Amandeep Khurana 2012-07-09, 19:21
Inline.
On Monday, July 9, 2012 at 12:17 PM, [EMAIL PROTECTED] wrote: > > > Now that I have a stable cluster, I would like to use YCSB to test > its performance; however, I am a bit confused after reading several > different website posting about YCSB. > > 1) Be default will YCSB read my hbase-site.xml file or do I have to > copy it into the YCSB conf directory? I plan on using on of my nodes > with no Hadoop/HBASE processes running on it, but it has all the > environmental stuff in place. > > You have to put the hbase-site.xml in YCSB/hbase/src/main/conf/. > > 2) Does the hbase.master property have to be site in the > hbase-site.xml file for YCSB to work? > > The only property that has to be there is the zookeeper quorum list. That's what the HBase client needs to talk to the cluster. > > 3) After working through all the workloads is there a script/tool > that will clean up my HBase? > > Nope. You'll need to go in and disable, drop the table you wrote too. You can do that from the shell. disable 'mytable' drop 'mytable' That's all you'll need to do to clean it up. > > Thank You > > --- > > Jay Wilson
-
Re: HBASE -- YCSB ?registration@... 2012-07-09, 21:30
Thank you Amandeep for your input. I go into "hbase shell" to create a table from my HMaster, which isn't running a DN process and I get the following. Could this be caused by a number of my DNs being offline, by the fact that the node isn't running a DN process, or something else? hbase(main):013:0> create 'usertable', 'testcol' ERROR: java.net.NoRouteToHostException: java.netNoRouteToHostException: No route to host Here is some help for this command: Create table; pass table name, a dictionary of specifications per column family, and optionally a dictionary of table configuration. Dictionaries are described below in the GENERAL NOTES section. Examples: hbase> create 't1', {NAME => 'f1', VERSIONS => 5} hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'} hbase> # The above in shorthand would be the following: hbase> create 't1', 'f1', 'f2', 'f3' hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000, BLOCKCACHE => true} hbase> create 't1', 'f1', {SPLITS => ['10', '20', '30', '40']} hbase> create 't1', 'f1', {SPLITS_FILE => 'splits.txt'} I can see in the ZK logs and the RS logs that they talk to the shell, so I know that communication is good and I find no errors or exceptions in them. Also I can do a hbase shell status, hbase shell zk_dump, and hadoop dfsadmin -report all from the node I am trying to create the table from with no issue. If I get on a node with the DataNode process running on it and try, I get the following: [hadoop@srack0-11 ~]$ hbase shell HBase Shell; enter 'help' for list of supported commands. Type "exit" to leave the HBase Shell Version 0.90.6-cdh3u4, r, Mon May 7 13:14:00 PDT 2012 hbase(main):001:0> status 3 servers, 0 dead, 0.6667 average load hbase(main):002:0> create 'usertable', 'tempcol' ERROR: java.io.IOException: java.io.IOException: Bad connect ack with firstBadLink as 172.18.0.9:50010 I assume this means it is trying to talk to a DN process on a node that I know is down. --- Jay Wilson ----- Original Message ----- From: [EMAIL PROTECTED] To:, Cc: Sent:Mon, 9 Jul 2012 12:21:22 -0700 Subject:Re: HBASE -- YCSB ? Inline. On Monday, July 9, 2012 at 12:17 PM, [EMAIL PROTECTED] [1] wrote: > > > Now that I have a stable cluster, I would like to use YCSB to test > its performance; however, I am a bit confused after reading several > different website posting about YCSB. > > 1) Be default will YCSB read my hbase-site.xml [2] file or do I have to > copy it into the YCSB conf directory? I plan on using on of my nodes > with no Hadoop/HBASE processes running on it, but it has all the > environmental stuff in place. > > You have to put the hbase-site.xml [3] in YCSB/hbase/src/main/conf/. > > 2) Does the hbase.master [4] property have to be site in the > hbase-site.xml [5] file for YCSB to work? > > The only property that has to be there is the zookeeper quorum list. That's what the HBase client needs to talk to the cluster. > > 3) After working through all the workloads is there a script/tool > that will clean up my HBase? > > Nope. You'll need to go in and disable, drop the table you wrote too. You can do that from the shell. disable 'mytable' drop 'mytable' That's all you'll need to do to clean it up. > > Thank You > > --- > > Jay Wilson Links: ------ [1] mailto:[EMAIL PROTECTED] [2] http://hbase-site.xml [3] http://hbase-site.xml [4] http://hbase.master [5] http://hbase-site.xml
-
Re: HBASE -- YCSB ?Dhaval Shah 2012-07-09, 21:36
This exception is generally caused when one of your server names returned does not map to a valid IP address on that host.. The services being up or not does not matter but the hostname should resolve to a valid IP
Regards, Dhaval ________________________________ From: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Sent: Monday, 9 July 2012 5:30 PM Subject: Re: HBASE -- YCSB ? Thank you Amandeep for your input. I go into "hbase shell" to create a table from my HMaster, which isn't running a DN process and I get the following. Could this be caused by a number of my DNs being offline, by the fact that the node isn't running a DN process, or something else? hbase(main):013:0> create 'usertable', 'testcol' ERROR: java.net.NoRouteToHostException: java.netNoRouteToHostException: No route to host Here is some help for this command: Create table; pass table name, a dictionary of specifications per column family, and optionally a dictionary of table configuration. Dictionaries are described below in the GENERAL NOTES section. Examples: hbase> create 't1', {NAME => 'f1', VERSIONS => 5} hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'} hbase> # The above in shorthand would be the following: hbase> create 't1', 'f1', 'f2', 'f3' hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000, BLOCKCACHE => true} hbase> create 't1', 'f1', {SPLITS => ['10', '20', '30', '40']} hbase> create 't1', 'f1', {SPLITS_FILE => 'splits.txt'} I can see in the ZK logs and the RS logs that they talk to the shell, so I know that communication is good and I find no errors or exceptions in them. Also I can do a hbase shell status, hbase shell zk_dump, and hadoop dfsadmin -report all from the node I am trying to create the table from with no issue. If I get on a node with the DataNode process running on it and try, I get the following: [hadoop@srack0-11 ~]$ hbase shell HBase Shell; enter 'help' for list of supported commands. Type "exit" to leave the HBase Shell Version 0.90.6-cdh3u4, r, Mon May 7 13:14:00 PDT 2012 hbase(main):001:0> status 3 servers, 0 dead, 0.6667 average load hbase(main):002:0> create 'usertable', 'tempcol' ERROR: java.io.IOException: java.io.IOException: Bad connect ack with firstBadLink as 172.18.0.9:50010 I assume this means it is trying to talk to a DN process on a node that I know is down. --- Jay Wilson ----- Original Message ----- From: [EMAIL PROTECTED] To:, Cc: Sent:Mon, 9 Jul 2012 12:21:22 -0700 Subject:Re: HBASE -- YCSB ? Inline. On Monday, July 9, 2012 at 12:17 PM, [EMAIL PROTECTED] [1] wrote: > > > Now that I have a stable cluster, I would like to use YCSB to test > its performance; however, I am a bit confused after reading several > different website posting about YCSB. > > 1) Be default will YCSB read my hbase-site.xml [2] file or do I have to > copy it into the YCSB conf directory? I plan on using on of my nodes > with no Hadoop/HBASE processes running on it, but it has all the > environmental stuff in place. > > You have to put the hbase-site.xml [3] in YCSB/hbase/src/main/conf/. > > 2) Does the hbase.master [4] property have to be site in the > hbase-site.xml [5] file for YCSB to work? > > The only property that has to be there is the zookeeper quorum list. That's what the HBase client needs to talk to the cluster. > > 3) After working through all the workloads is there a script/tool > that will clean up my HBase? > > Nope. You'll need to go in and disable, drop the table you wrote too. You can do that from the shell. disable 'mytable' drop 'mytable' That's all you'll need to do to clean it up. > > Thank You > > --- > > Jay Wilson Links: ------ [1] mailto:[EMAIL PROTECTED] [2] http://hbase-site.xml [3] http://hbase-site.xml [4] http://hbase.master [5] http://hbase-site.xml
-
Re: HBASE -- YCSB ?registration@... 2012-07-09, 21:56
Is there a "debug" flag I can use with hbase shell that will tell me the name it's trying to resolve? Thank you --- Jay Wilson ----- Original Message ----- From: To:"[EMAIL PROTECTED]" , "[EMAIL PROTECTED]" Cc: Sent:Tue, 10 Jul 2012 05:36:44 +0800 (SGT) Subject:Re: HBASE -- YCSB ? This exception is generally caused when one of your server names returned does not map to a valid IP address on that host.. The services being up or not does not matter but the hostname should resolve to a valid IP Regards, Dhaval ________________________________ From: "[EMAIL PROTECTED] [1]" To: [EMAIL PROTECTED] [2] Sent: Monday, 9 July 2012 5:30 PM Subject: Re: HBASE -- YCSB ?     Thank you Amandeep for your input.    I go into "hbase shell" to create a table from my HMaster, which isn't running a DN process and I get the following. �� Could this be caused by a number of my DNs being offline, by the fact that the node isn't running a DN process, or something else?    hbase(main):013:0> create 'usertable', 'testcol' ERROR: java.net.NoRouteToHostException: [3] java.netNoRouteToHostException: [4] No route to host Here is some help for this command: Create table; pass table name, a dictionary of specifications per column family, and optionally a dictionary of table configuration. Dictionaries are described below in the GENERAL NOTES section. Examples: hbase> create 't1', {NAME => 'f1', VERSIONS => 5} hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'} hbase> # The above in shorthand would be the following: hbase> create 't1', 'f1', 'f2', 'f3' hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000, BLOCKCACHE => true} hbase> create 't1', 'f1', {SPLITS => ['10', '20', '30', '40']} hbase> create 't1', 'f1', {SPLITS_FILE => 'splits.txt'} [5]    I can see in the ZK logs and the RS logs that they talk to the shell, so I know that communication is good and I find no errors or exceptions in them. ��  Also I can do a hbase shell status, hbase shell zk_dump, and hadoop dfsadmin -report all from the node I am trying to create the table from with no issue.    If I get on a node with the DataNode process running on it and try, I get the following:    [hadoop@srack0-11 ~]$ hbase shell HBase Shell; enter 'help' for list of supported commands. Type "exit" to leave the HBase Shell Version 0.90.6-cdh3u4, r, Mon May 7 13:14:00 PDT 2012 hbase(main):001:0> status 3 servers, 0 dead, 0.6667 average load hbase(main):002:0> create 'usertable', 'tempcol' ERROR: java.io.IOException: [6] java.io.IOException: [7] Bad connect ack with firstBadLink as 172.18.0.9:50010    I assume this means it is trying to talk to a DN process on a node that I know is down.      ---    Jay Wilson ----- Original Message ----- From: [EMAIL PROTECTED] [8] To:, Cc: Sent:Mon, 9 Jul 2012 12:21:22 -0700 Subject:Re: HBASE -- YCSB ? Inline. On Monday, July 9, 2012 at 12:17 PM, [EMAIL PROTECTED] [9] [1] wrote: > > > Now that I have a stable cluster, I would like to use YCSB to test > its performance; however, I am a bit confused after reading several > different website posting about YCSB. > > 1) Be default will YCSB read my hbase-site.xml [10] [2] file or do I have to > copy it into the YCSB conf directory? I plan on using on of my nodes > with no Hadoop/HBASE processes running on it, but it has all the > environmental stuff in place. > > You have to put the hbase-site.xml [11] [3] in YCSB/hbase/src/main/conf/. > > 2) Does the hbase.master [12] [4] property have to be site in the > hbase-site.xml [13] [5] file for YCSB to work? > > The only property that has to be there is the zookeeper quorum list. That's what the HBase client needs to talk to the cluster. > > 3) After working through all the workloads is there a script/tool > that will clean up my HBase? > > Nope. You'll need to go in and disable, drop the table you wrote too. You can do that from the shell. disable 'mytable' drop 'mytable' That's all you'll need to do to clean it up. > > Thank You > > --- > > Jay Wilson Links: ------ [1] mailto:[EMAIL PROTECTED] [14] [2] http://hbase-site.xml [15] [3] http://hbase-site.xml [16] [4] http://hbase.master [17] [5] http://hbase-site.xml [18] Links: [1] mailto:[EMAIL PROTECTED] [2] mailto:[EMAIL PROTECTED] [3] http://java.net.NoRouteToHostException [4] http://java.netNoRouteToHostException [5] http://sitemail.hostway.com/http: [6] http://java.io.IOException [7] http://java.io.IOException [8] mailto:[EMAIL PROTECTED] [9] mailto:[EMAIL PROTECTED] [10] http://hbase-site.xml [11] http://hbase-sitexml [12] http://hbase.master [13] http://hbase-site.xml [14] mailto:[EMAIL PROTECTED] [15] http://hbase-site.xml [16] http://hbase-site.xml [17] http://hbase.master [18] http://hbase-site.xml
-
Re: HBASE -- YCSB ?Dhaval Shah 2012-07-09, 22:03
There is definitely a debug flag on hbase.. You can find out details on http://hbase.apache.org/shell.html.. I am not sure how much details would it log though.. I have never used it personally
Regards, Dhaval ----- Original Message ----- From: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> To: '[EMAIL PROTECTED]' Cc: Sent: Monday, 9 July 2012 5:56 PM Subject: Re: HBASE -- YCSB ? Is there a "debug" flag I can use with hbase shell that will tell me the name it's trying to resolve? Thank you --- Jay Wilson ----- Original Message ----- From: To:"[EMAIL PROTECTED]" , "[EMAIL PROTECTED]" Cc: Sent:Tue, 10 Jul 2012 05:36:44 +0800 (SGT) Subject:Re: HBASE -- YCSB ? This exception is generally caused when one of your server names returned does not map to a valid IP address on that host.. The services being up or not does not matter but the hostname should resolve to a valid IPÂ Regards, Dhaval ________________________________ From: "[EMAIL PROTECTED] [1]" To: [EMAIL PROTECTED] [2] Sent: Monday, 9 July 2012 5:30 PM Subject: Re: HBASE -- YCSB ? Â Â Â Â Thank you Amandeep for your input. Â Â Â I go into "hbase shell" to create a table from my HMaster, which isn't running a DN process and I get the following. Â Could this be caused by a number of my DNs being offline, by the fact that the node isn't running a DN process, or something else? Â Â Â hbase(main):013:0> create 'usertable', 'testcol' ERROR: java.net.NoRouteToHostException: [3] java.netNoRouteToHostException: [4] No route to host Here is some help for this command: Create table; pass table name, a dictionary of specifications per column family, and optionally a dictionary of table configuration. Dictionaries are described below in the GENERAL NOTES section. Examples: hbase> create 't1', {NAME => 'f1', VERSIONS => 5} hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'} hbase> # The above in shorthand would be the following: hbase> create 't1', 'f1', 'f2', 'f3' hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000, BLOCKCACHE => true} hbase> create 't1', 'f1', {SPLITS => ['10', '20', '30', '40']} hbase> create 't1', 'f1', {SPLITS_FILE => 'splits.txt'} [5] Â Â Â I can see in the ZK logs and the RS logs that they talk to the shell, so I know that communication is good and I find no errors or exceptions in them. Â Â Â Also I can do a hbase shell status, hbase shell zk_dump, and hadoop dfsadmin -report all from the node I am trying to create the table from with no issue. Â Â Â If I get on a node with the DataNode process running on it and try, I get the following: Â Â Â [hadoop@srack0-11 ~]$ hbase shell HBase Shell; enter 'help' for list of supported commands. Type "exit" to leave the HBase Shell Version 0.90.6-cdh3u4, r, Mon May 7 13:14:00 PDT 2012 hbase(main):001:0> status 3 servers, 0 dead, 0.6667 average load hbase(main):002:0> create 'usertable', 'tempcol' ERROR: java.io.IOException: [6] java.io.IOException: [7] Bad connect ack with firstBadLink as 172.18.0.9:50010 Â Â Â I assume this means it is trying to talk to a DN process on a node that I know is down. Â Â Â Â Â --- Â Â Â Jay Wilson ----- Original Message ----- From: [EMAIL PROTECTED] [8] To:, Cc: Sent:Mon, 9 Jul 2012 12:21:22 -0700 Subject:Re: HBASE -- YCSB ? Inline. On Monday, July 9, 2012 at 12:17 PM, [EMAIL PROTECTED] [9] [1] wrote: > > > Now that I have a stable cluster, I would like to use YCSB to test > its performance; however, I am a bit confused after reading several > different website posting about YCSB. > > 1) Be default will YCSB read my hbase-site.xml [10] [2] file or do I have to > copy it into the YCSB conf directory? I plan on using on of my nodes > with no Hadoop/HBASE processes running on it, but it has all the > environmental stuff in place. > > You have to put the hbase-site.xml [11] [3] in YCSB/hbase/src/main/conf/. > > 2) Does the hbase.master [12] [4] property have to be site in the The only property that has to be there is the zookeeper quorum list. That's what the HBase client needs to talk to the cluster. Nope. You'll need to go in and disable, drop the table you wrote too. You can do that from the shell. disable 'mytable' drop 'mytable' That's all you'll need to do to clean it up. Links: [1] mailto:[EMAIL PROTECTED] [14] [2] http://hbase-site.xml [15] [3] http://hbase-site.xml [16] [4] http://hbase.master [17] [5] http://hbase-site.xml [18] Links: [1] mailto:[EMAIL PROTECTED] [2] mailto:[EMAIL PROTECTED] [3] http://java.net.NoRouteToHostException [4] http://java.netNoRouteToHostException [5] http://sitemail.hostway.com/http: [6] http://java.io.IOException [7] http://java.io.IOException [8] mailto:[EMAIL PROTECTED] [9] mailto:[EMAIL PROTECTED] [10] http://hbase-site.xml [11] http://hbase-sitexml [12] http://hbase.master [13] http://hbase-site.xml [14] mailto:[EMAIL PROTECTED] [15] http://hbase-site.xml [16] http://hbase-site.xml [17] http://hbase.master [18] http://hbase-site.xml
-
Re: HBASE -- YCSB ?Suraj Varma 2012-07-10, 22:35
Search for "hadoop-dns-checker" in http://hbase.apache.org/book.html
That tool might help figure out if your cluster networking is all right. --S On Mon, Jul 9, 2012 at 3:03 PM, Dhaval Shah <[EMAIL PROTECTED]> wrote: > There is definitely a debug flag on hbase.. You can find out details on http://hbase.apache.org/shell.html.. I am not sure how much details would it log though.. I have never used it personally > > Regards, > Dhaval > > > ----- Original Message ----- > From: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> > To: '[EMAIL PROTECTED]' > Cc: > Sent: Monday, 9 July 2012 5:56 PM > Subject: Re: HBASE -- YCSB ? > > > > Is there a "debug" flag I can use with hbase shell that will tell > me the name it's trying to resolve? > > Thank you > > --- > > Jay Wilson > > ----- Original Message ----- > From: > To:"[EMAIL PROTECTED]" , "[EMAIL PROTECTED]" > Cc: > Sent:Tue, 10 Jul 2012 05:36:44 +0800 (SGT) > Subject:Re: HBASE -- YCSB ? > > This exception is generally caused when one of your server names > returned does not map to a valid IP address on that host.. The > services being up or not does not matter but the hostname should > resolve to a valid IPÂ > > Regards, > Dhaval > > ________________________________ > From: "[EMAIL PROTECTED] [1]" > To: [EMAIL PROTECTED] [2] > Sent: Monday, 9 July 2012 5:30 PM > Subject: Re: HBASE -- YCSB ? > > Â Â Â Â Thank you Amandeep for your input. > > Â Â Â I go into "hbase shell" to create a table from my > HMaster, which > isn't running a DN process and I get the following. Â Could this > be > caused by a number of my DNs being offline, by the fact that the > node > isn't running a DN process, or something else? > > Â Â Â hbase(main):013:0> create 'usertable', 'testcol' > > ERROR: java.net.NoRouteToHostException: [3] > java.netNoRouteToHostException: [4] No route to host > > Here is some help for this command: > Create table; pass table name, a dictionary of specifications per > column family, and optionally a dictionary of table configuration. > Dictionaries are described below in the GENERAL NOTES section. > Examples: > > hbase> create 't1', {NAME => 'f1', VERSIONS => 5} > hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'} > hbase> # The above in shorthand would be the following: > hbase> create 't1', 'f1', 'f2', 'f3' > hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000, > BLOCKCACHE => true} > hbase> create 't1', 'f1', {SPLITS => ['10', '20', '30', '40']} > hbase> create 't1', 'f1', {SPLITS_FILE => 'splits.txt'} [5] > > Â Â Â I can see in the ZK logs and the RS logs that they talk > to the > shell, so I know that communication is good and I find no errors or > exceptions in them. > > Â Â Â Also I can do a hbase shell status, hbase shell > zk_dump, and hadoop > dfsadmin -report all from the node I am trying to create the table > from with no issue. > > Â Â Â If I get on a node with the DataNode process running on > it and try, > I get the following: > > Â Â Â [hadoop@srack0-11 ~]$ hbase shell > HBase Shell; enter 'help' for list of supported commands. > Type "exit" to leave the HBase Shell > Version 0.90.6-cdh3u4, r, Mon May 7 13:14:00 PDT 2012 > > hbase(main):001:0> status > 3 servers, 0 dead, 0.6667 average load > > hbase(main):002:0> create 'usertable', 'tempcol' > > ERROR: java.io.IOException: [6] java.io.IOException: [7] Bad connect > ack with > firstBadLink as 172.18.0.9:50010 > > Â Â Â I assume this means it is trying to talk to a DN > process on a node > that I know is down. > > Â Â Â Â Â --- > > Â Â Â Jay Wilson > > ----- Original Message ----- > From: [EMAIL PROTECTED] [8] > To:, > Cc: > Sent:Mon, 9 Jul 2012 12:21:22 -0700 > Subject:Re: HBASE -- YCSB ? > > Inline. > > On Monday, July 9, 2012 at 12:17 PM, > [EMAIL PROTECTED] [9] [1] wrote: > >> >> >> Now that I have a stable cluster, I would like to use YCSB to test >> its performance; however, I am a bit confused after reading |