|
|
-
Reading table sequentially...
Something Something 2010-01-12, 17:12
I know I am doing something really dumb, so apologies in advance.
All I want to do is to read a table sequentially. I tried several versions of this, but nothing gives me the "value of the Key". I am sure there's an easier way.... please help. Thanks.
HTable table; table = new HTable(new HBaseConfiguration(), "mytable"); Scan scan = new Scan(); // scan.addFamily("myfamily");
ResultScanner scanner = table.getScanner(scan); Result result; while ((result = scanner.next()) != null) { NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>> map = result.getMap(); for (Map.Entry<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>> entry : map.entrySet()) { byte[] key = entry.getKey(); *LOG.info("key = " + Bytes.toString(key));* NavigableMap<byte[], NavigableMap<Long, byte[]>> value entry.getValue(); for (Entry<byte[], NavigableMap<Long, byte[]>> entry1 : value.entrySet()) { byte[] key1 = entry1.getKey(); *LOG.info("key1 = " + Bytes.toString(key1));* NavigableMap<byte[], NavigableMap<Long, byte[]>> value1 entry.getValue(); for (Entry<byte[], NavigableMap<Long, byte[]>> entry2 : value1.entrySet()) { String key2 = Bytes.toString(entry2.getKey()); *LOG.info("key2 = " + key2);*
} } } }
-
Re: Reading table sequentially...
stack 2010-01-12, 17:36
See below: On Tue, Jan 12, 2010 at 9:12 AM, Something Something < [EMAIL PROTECTED]> wrote: > > NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, > byte[]>>> map = result.getMap(); > Above returns a map keyed by families: http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#getMap%28%29> for (Map.Entry<byte[], NavigableMap<byte[], NavigableMap<Long, > byte[]>>> entry : map.entrySet()) { > byte[] key = entry.getKey(); > This is family name, not key. > *LOG.info("key = " + Bytes.toString(key));* > NavigableMap<byte[], NavigableMap<Long, byte[]>> value > entry.getValue(); > This is a map keyed by column qualifiers. > for (Entry<byte[], NavigableMap<Long, byte[]>> entry1 : > value.entrySet()) { > byte[] key1 = entry1.getKey(); > *LOG.info("key1 = " + Bytes.toString(key1));* > This is the family qualifier. > NavigableMap<byte[], NavigableMap<Long, byte[]>> value1 > entry.getValue(); > I do not think you intended to do this. I think you meant entry1, not 'entry' and map type should be NavigableMap<Long, byte[]> rather than above. St.Ack > for (Entry<byte[], NavigableMap<Long, byte[]>> entry2 : > value1.entrySet()) { > String key2 = Bytes.toString(entry2.getKey()); > *LOG.info("key2 = " + key2);* > > } > } > } > } >
-
Re: Reading table sequentially...
Something Something 2010-01-12, 18:24
Sorry. That was a typo. In any case, it seems like I am using the wrong API. Here's what my table contains: ABC_111 column=info:estimate, timestamp=1263319888463, value=179.59 ABC_222 column=info:estimate, timestamp=1263319888463, value=191.50 ABC_333 column=info:estimate, timestamp=1263319888463, value=180.65 ABC_444 column=info:estimate, timestamp=1263319888463, value=183.63 & so on.... I want to retrieve: ABC_111 179.59 ABC_222 191.50 ABC_333 180.65 ABC_444 183.63 & so on... What API should I use? Please let me know. Thanks for your help. On Tue, Jan 12, 2010 at 9:36 AM, stack <[EMAIL PROTECTED]> wrote: > See below: > > On Tue, Jan 12, 2010 at 9:12 AM, Something Something < > [EMAIL PROTECTED]> wrote: > > > > NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, > > byte[]>>> map = result.getMap(); > > > > Above returns a map keyed by families: > > http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#getMap%28%29> > > > > > for (Map.Entry<byte[], NavigableMap<byte[], NavigableMap<Long, > > byte[]>>> entry : map.entrySet()) { > > byte[] key = entry.getKey(); > > > > This is family name, not key. > > > > > *LOG.info("key = " + Bytes.toString(key));* > > NavigableMap<byte[], NavigableMap<Long, byte[]>> value > > entry.getValue(); > > > > This is a map keyed by column qualifiers. > > > > > for (Entry<byte[], NavigableMap<Long, byte[]>> entry1 : > > value.entrySet()) { > > byte[] key1 = entry1.getKey(); > > *LOG.info("key1 = " + Bytes.toString(key1));* > > > > > This is the family qualifier. > > > > > NavigableMap<byte[], NavigableMap<Long, byte[]>> value1 > > > entry.getValue(); > > > > > I do not think you intended to do this. I think you meant entry1, not > 'entry' and map type should be NavigableMap<Long, byte[]> rather than > above. > > St.Ack > > > > > for (Entry<byte[], NavigableMap<Long, byte[]>> entry2 : > > value1.entrySet()) { > > String key2 = Bytes.toString(entry2.getKey()); > > *LOG.info("key2 = " + key2);* > > > > } > > } > > } > > } > > >
-
Re: Reading table sequentially...
stack 2010-01-12, 19:12
Setup the scanner and next it as you did previous. Then on the Result object, do something like: for (KeyValue kv: result.raw()) { System.out.println(Bytes.toString(kv.getRow()) + " " + Bytes.toString(kv.getValue())); } St.Ack On Tue, Jan 12, 2010 at 10:24 AM, Something Something < [EMAIL PROTECTED]> wrote: > Sorry. That was a typo. In any case, it seems like I am using the wrong > API. > > Here's what my table contains: > > > ABC_111 column=info:estimate, timestamp=1263319888463, > value=179.59 > ABC_222 column=info:estimate, timestamp=1263319888463, > value=191.50 > ABC_333 column=info:estimate, timestamp=1263319888463, > value=180.65 > ABC_444 column=info:estimate, timestamp=1263319888463, > value=183.63 > & so on.... > > > I want to retrieve: > > ABC_111 179.59 > ABC_222 191.50 > ABC_333 180.65 > ABC_444 183.63 > & so on... > > What API should I use? Please let me know. Thanks for your help. > > > On Tue, Jan 12, 2010 at 9:36 AM, stack <[EMAIL PROTECTED]> wrote: > > > See below: > > > > On Tue, Jan 12, 2010 at 9:12 AM, Something Something < > > [EMAIL PROTECTED]> wrote: > > > > > > NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, > > > byte[]>>> map = result.getMap(); > > > > > > > Above returns a map keyed by families: > > > > > http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#getMap%28%29> > > > > > > > > > > for (Map.Entry<byte[], NavigableMap<byte[], > NavigableMap<Long, > > > byte[]>>> entry : map.entrySet()) { > > > byte[] key = entry.getKey(); > > > > > > > This is family name, not key. > > > > > > > > > *LOG.info("key = " + Bytes.toString(key));* > > > NavigableMap<byte[], NavigableMap<Long, byte[]>> value > > > entry.getValue(); > > > > > > > This is a map keyed by column qualifiers. > > > > > > > > > for (Entry<byte[], NavigableMap<Long, byte[]>> entry1 : > > > value.entrySet()) { > > > byte[] key1 = entry1.getKey(); > > > *LOG.info("key1 = " + Bytes.toString(key1));* > > > > > > > > > This is the family qualifier. > > > > > > > > > NavigableMap<byte[], NavigableMap<Long, byte[]>> > value1 > > > > > entry.getValue(); > > > > > > > > > I do not think you intended to do this. I think you meant entry1, not > > 'entry' and map type should be NavigableMap<Long, byte[]> rather than > > above. > > > > St.Ack > > > > > > > > > for (Entry<byte[], NavigableMap<Long, byte[]>> entry2 > : > > > value1.entrySet()) { > > > String key2 = Bytes.toString(entry2.getKey()); > > > *LOG.info("key2 = " + key2);* > > > > > > } > > > } > > > } > > > } > > > > > >
-
Re: Reading table sequentially...
Something Something 2010-01-12, 22:01
Cool. That works. Just couple quick questions to confirm: All the Keys returned by this code are guaranteed to be in order by the key values, correct? Also, for some other table I am retrieving all column names for a particular key, and those all seem to be in the correct order as well. Is this always guaranteed? On Tue, Jan 12, 2010 at 11:12 AM, stack <[EMAIL PROTECTED]> wrote: > Setup the scanner and next it as you did previous. Then on the Result > object, do something like: > > for (KeyValue kv: result.raw()) { > System.out.println(Bytes.toString(kv.getRow()) + " " + > Bytes.toString(kv.getValue())); > } > > St.Ack > > On Tue, Jan 12, 2010 at 10:24 AM, Something Something < > [EMAIL PROTECTED]> wrote: > > > Sorry. That was a typo. In any case, it seems like I am using the wrong > > API. > > > > Here's what my table contains: > > > > > > ABC_111 column=info:estimate, > timestamp=1263319888463, > > value=179.59 > > ABC_222 column=info:estimate, > timestamp=1263319888463, > > value=191.50 > > ABC_333 column=info:estimate, > timestamp=1263319888463, > > value=180.65 > > ABC_444 column=info:estimate, > timestamp=1263319888463, > > value=183.63 > > & so on.... > > > > > > I want to retrieve: > > > > ABC_111 179.59 > > ABC_222 191.50 > > ABC_333 180.65 > > ABC_444 183.63 > > & so on... > > > > What API should I use? Please let me know. Thanks for your help. > > > > > > On Tue, Jan 12, 2010 at 9:36 AM, stack <[EMAIL PROTECTED]> wrote: > > > > > See below: > > > > > > On Tue, Jan 12, 2010 at 9:12 AM, Something Something < > > > [EMAIL PROTECTED]> wrote: > > > > > > > > NavigableMap<byte[], NavigableMap<byte[], > NavigableMap<Long, > > > > byte[]>>> map = result.getMap(); > > > > > > > > > > Above returns a map keyed by families: > > > > > > > > > http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#getMap%28%29> > > > > > > > > > > > > > > > for (Map.Entry<byte[], NavigableMap<byte[], > > NavigableMap<Long, > > > > byte[]>>> entry : map.entrySet()) { > > > > byte[] key = entry.getKey(); > > > > > > > > > > This is family name, not key. > > > > > > > > > > > > > *LOG.info("key = " + Bytes.toString(key));* > > > > NavigableMap<byte[], NavigableMap<Long, byte[]>> value > > > > entry.getValue(); > > > > > > > > > > This is a map keyed by column qualifiers. > > > > > > > > > > > > > for (Entry<byte[], NavigableMap<Long, byte[]>> entry1 : > > > > value.entrySet()) { > > > > byte[] key1 = entry1.getKey(); > > > > *LOG.info("key1 = " + Bytes.toString(key1));* > > > > > > > > > > > > > This is the family qualifier. > > > > > > > > > > > > > NavigableMap<byte[], NavigableMap<Long, byte[]>> > > value1 > > > > > > > entry.getValue(); > > > > > > > > > > > > > I do not think you intended to do this. I think you meant entry1, not > > > 'entry' and map type should be NavigableMap<Long, byte[]> rather than > > > above. > > > > > > St.Ack > > > > > > > > > > > > > for (Entry<byte[], NavigableMap<Long, byte[]>> > entry2 > > : > > > > value1.entrySet()) { > > > > String key2 = Bytes.toString(entry2.getKey()); > > > > *LOG.info("key2 = " + key2);* > > > > > > > > } > > > > } > > > > } > > > > } > > > > > > > > > >
-
Re: Reading table sequentially...
stack 2010-01-13, 00:46
On Tue, Jan 12, 2010 at 2:01 PM, Something Something < [EMAIL PROTECTED]> wrote: > All the Keys returned by this code are guaranteed to be in order by the key > values, correct? > No. Explicitly, the order is NOT guaranteed, for performance reasons (I don't see this in the javadoc. It should be there). If just getting latest version from a set of columns, I believe it will come across in the right order but not guaranteed. To get a guaranteed sorted order, call http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#sorted()insteadof the raw() method. The sort will be done client-side. > > Also, for some other table I am retrieving all column names for a > particular > key, and those all seem to be in the correct order as well. Is this always > guaranteed? > Call sort if you need guarantee. St.Ack > > On Tue, Jan 12, 2010 at 11:12 AM, stack <[EMAIL PROTECTED]> wrote: > > > Setup the scanner and next it as you did previous. Then on the Result > > object, do something like: > > > > for (KeyValue kv: result.raw()) { > > System.out.println(Bytes.toString(kv.getRow()) + " " + > > Bytes.toString(kv.getValue())); > > } > > > > St.Ack > > > > On Tue, Jan 12, 2010 at 10:24 AM, Something Something < > > [EMAIL PROTECTED]> wrote: > > > > > Sorry. That was a typo. In any case, it seems like I am using the > wrong > > > API. > > > > > > Here's what my table contains: > > > > > > > > > ABC_111 column=info:estimate, > > timestamp=1263319888463, > > > value=179.59 > > > ABC_222 column=info:estimate, > > timestamp=1263319888463, > > > value=191.50 > > > ABC_333 column=info:estimate, > > timestamp=1263319888463, > > > value=180.65 > > > ABC_444 column=info:estimate, > > timestamp=1263319888463, > > > value=183.63 > > > & so on.... > > > > > > > > > I want to retrieve: > > > > > > ABC_111 179.59 > > > ABC_222 191.50 > > > ABC_333 180.65 > > > ABC_444 183.63 > > > & so on... > > > > > > What API should I use? Please let me know. Thanks for your help. > > > > > > > > > On Tue, Jan 12, 2010 at 9:36 AM, stack <[EMAIL PROTECTED]> wrote: > > > > > > > See below: > > > > > > > > On Tue, Jan 12, 2010 at 9:12 AM, Something Something < > > > > [EMAIL PROTECTED]> wrote: > > > > > > > > > > NavigableMap<byte[], NavigableMap<byte[], > > NavigableMap<Long, > > > > > byte[]>>> map = result.getMap(); > > > > > > > > > > > > > Above returns a map keyed by families: > > > > > > > > > > > > > > http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#getMap%28%29> > > > > > > > > > > > > > > > > > > > > for (Map.Entry<byte[], NavigableMap<byte[], > > > NavigableMap<Long, > > > > > byte[]>>> entry : map.entrySet()) { > > > > > byte[] key = entry.getKey(); > > > > > > > > > > > > > This is family name, not key. > > > > > > > > > > > > > > > > > *LOG.info("key = " + Bytes.toString(key));* > > > > > NavigableMap<byte[], NavigableMap<Long, byte[]>> value > > > > > entry.getValue(); > > > > > > > > > > > > > This is a map keyed by column qualifiers. > > > > > > > > > > > > > > > > > for (Entry<byte[], NavigableMap<Long, byte[]>> entry1 > : > > > > > value.entrySet()) { > > > > > byte[] key1 = entry1.getKey(); > > > > > *LOG.info("key1 = " + Bytes.toString(key1));* > > > > > > > > > > > > > > > > > This is the family qualifier. > > > > > > > > > > > > > > > > > NavigableMap<byte[], NavigableMap<Long, byte[]>> > > > value1 > > > > > > > > > entry.getValue(); > > > > > > > > > > > > > > > > > I do not think you intended to do this. I think you meant entry1, > not > > > > 'entry' and map type should be NavigableMap<Long, byte[]> rather than > > > > above. > > > > > > > > St.Ack > > > > > > > > > > > > > > > > > for (Entry<byte[], NavigableMap<Long, byte[]>>
-
Re: Reading table sequentially...
Something Something 2010-01-13, 05:32
Thanks for the explanation. So as per the link you provided the list() method would return a sorted list, correct? About the 2nd point, I am using result.getFamilyMap() which returns NavigableMap which extends SortedMap (as per http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true), so I guess the returned Map would be sorted by key values, so I am safe there as well, correct? On Tue, Jan 12, 2010 at 4:46 PM, stack <[EMAIL PROTECTED]> wrote: > On Tue, Jan 12, 2010 at 2:01 PM, Something Something < > [EMAIL PROTECTED]> wrote: > > > All the Keys returned by this code are guaranteed to be in order by the > key > > values, correct? > > > > No. Explicitly, the order is NOT guaranteed, for performance reasons (I > don't see this in the javadoc. It should be there). If just getting > latest > version from a set of columns, I believe it will come across in the right > order but not guaranteed. To get a guaranteed sorted order, call > > http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#sorted()instead<http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#sorted%28%29instead>> of the raw() method. The sort will be done client-side. > > > > > > Also, for some other table I am retrieving all column names for a > > particular > > key, and those all seem to be in the correct order as well. Is this > always > > guaranteed? > > > > Call sort if you need guarantee. > > St.Ack > > > > > > > > > > > On Tue, Jan 12, 2010 at 11:12 AM, stack <[EMAIL PROTECTED]> wrote: > > > > > Setup the scanner and next it as you did previous. Then on the Result > > > object, do something like: > > > > > > for (KeyValue kv: result.raw()) { > > > System.out.println(Bytes.toString(kv.getRow()) + " " + > > > Bytes.toString(kv.getValue())); > > > } > > > > > > St.Ack > > > > > > On Tue, Jan 12, 2010 at 10:24 AM, Something Something < > > > [EMAIL PROTECTED]> wrote: > > > > > > > Sorry. That was a typo. In any case, it seems like I am using the > > wrong > > > > API. > > > > > > > > Here's what my table contains: > > > > > > > > > > > > ABC_111 column=info:estimate, > > > timestamp=1263319888463, > > > > value=179.59 > > > > ABC_222 column=info:estimate, > > > timestamp=1263319888463, > > > > value=191.50 > > > > ABC_333 column=info:estimate, > > > timestamp=1263319888463, > > > > value=180.65 > > > > ABC_444 column=info:estimate, > > > timestamp=1263319888463, > > > > value=183.63 > > > > & so on.... > > > > > > > > > > > > I want to retrieve: > > > > > > > > ABC_111 179.59 > > > > ABC_222 191.50 > > > > ABC_333 180.65 > > > > ABC_444 183.63 > > > > & so on... > > > > > > > > What API should I use? Please let me know. Thanks for your help. > > > > > > > > > > > > On Tue, Jan 12, 2010 at 9:36 AM, stack <[EMAIL PROTECTED]> wrote: > > > > > > > > > See below: > > > > > > > > > > On Tue, Jan 12, 2010 at 9:12 AM, Something Something < > > > > > [EMAIL PROTECTED]> wrote: > > > > > > > > > > > > NavigableMap<byte[], NavigableMap<byte[], > > > NavigableMap<Long, > > > > > > byte[]>>> map = result.getMap(); > > > > > > > > > > > > > > > > Above returns a map keyed by families: > > > > > > > > > > > > > > > > > > > > http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#getMap%28%29> > > > > > > > > > > > > > > > > > > > > > > > > > for (Map.Entry<byte[], NavigableMap<byte[], > > > > NavigableMap<Long, > > > > > > byte[]>>> entry : map.entrySet()) { > > > > > > byte[] key = entry.getKey(); > > > > > > > > > > > > > > > > This is family name, not key. > > > > > > > > > > > > > > > > > > > > > *LOG.info("key = " + Bytes.toString(key));* > > > > > > NavigableMap<byte[], NavigableMap<Long, byte[]>> value > > > > > > > entry.getValue(); > > > > > > > > > > > > >
-
Re: Reading table sequentially...
stack 2010-01-13, 05:51
On Tue, Jan 12, 2010 at 9:32 PM, Something Something < [EMAIL PROTECTED]> wrote: > Thanks for the explanation. So as per the link you provided the list() > method would return a sorted list, correct? > > Thats what it says in the javadoc. > About the 2nd point, I am using result.getFamilyMap() which returns > NavigableMap which extends SortedMap (as per > > http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true> ), > so I guess the returned Map would be sorted by key values, so I am safe > there as well, correct? > Yes. That sounds right. The raw method is there so applications that can deal, can get the raw results without client machinations getting in the way. St.Ack > > > On Tue, Jan 12, 2010 at 4:46 PM, stack <[EMAIL PROTECTED]> wrote: > > > On Tue, Jan 12, 2010 at 2:01 PM, Something Something < > > [EMAIL PROTECTED]> wrote: > > > > > All the Keys returned by this code are guaranteed to be in order by the > > key > > > values, correct? > > > > > > > No. Explicitly, the order is NOT guaranteed, for performance reasons (I > > don't see this in the javadoc. It should be there). If just getting > > latest > > version from a set of columns, I believe it will come across in the right > > order but not guaranteed. To get a guaranteed sorted order, call > > > > > http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#sorted()instead> < > http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#sorted%28%29instead> > > > of the raw() method. The sort will be done client-side. > > > > > > > > > > Also, for some other table I am retrieving all column names for a > > > particular > > > key, and those all seem to be in the correct order as well. Is this > > always > > > guaranteed? > > > > > > > Call sort if you need guarantee. > > > > St.Ack > > > > > > > > > > > > > > > > > > > > On Tue, Jan 12, 2010 at 11:12 AM, stack <[EMAIL PROTECTED]> wrote: > > > > > > > Setup the scanner and next it as you did previous. Then on the > Result > > > > object, do something like: > > > > > > > > for (KeyValue kv: result.raw()) { > > > > System.out.println(Bytes.toString(kv.getRow()) + " " + > > > > Bytes.toString(kv.getValue())); > > > > } > > > > > > > > St.Ack > > > > > > > > On Tue, Jan 12, 2010 at 10:24 AM, Something Something < > > > > [EMAIL PROTECTED]> wrote: > > > > > > > > > Sorry. That was a typo. In any case, it seems like I am using the > > > wrong > > > > > API. > > > > > > > > > > Here's what my table contains: > > > > > > > > > > > > > > > ABC_111 column=info:estimate, > > > > timestamp=1263319888463, > > > > > value=179.59 > > > > > ABC_222 column=info:estimate, > > > > timestamp=1263319888463, > > > > > value=191.50 > > > > > ABC_333 column=info:estimate, > > > > timestamp=1263319888463, > > > > > value=180.65 > > > > > ABC_444 column=info:estimate, > > > > timestamp=1263319888463, > > > > > value=183.63 > > > > > & so on.... > > > > > > > > > > > > > > > I want to retrieve: > > > > > > > > > > ABC_111 179.59 > > > > > ABC_222 191.50 > > > > > ABC_333 180.65 > > > > > ABC_444 183.63 > > > > > & so on... > > > > > > > > > > What API should I use? Please let me know. Thanks for your help. > > > > > > > > > > > > > > > On Tue, Jan 12, 2010 at 9:36 AM, stack <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > See below: > > > > > > > > > > > > On Tue, Jan 12, 2010 at 9:12 AM, Something Something < > > > > > > [EMAIL PROTECTED]> wrote: > > > > > > > > > > > > > > NavigableMap<byte[], NavigableMap<byte[], > > > > NavigableMap<Long, > > > > > > > byte[]>>> map = result.getMap(); > > > > > > > > > > > > > > > > > > > Above returns a map keyed by families: > > > > > > > > > > > > > > > > > > > > > > > > > > > http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/Result.html#getMap%28%29
|
|