Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Use of MD5 as row keys - is this safe?


Copy link to this message
-
Use of MD5 as row keys - is this safe?
Hi,

I know it is a commonly suggested to use an MD5 checksum to create a row
key from some other identifier, such as a string or long. This is usually
done to guard against hot-spotting and seems to work well.

My concern is that there no guard against collision when this is done - two
different strings or longs could produce the same row-key. Although this is
very unlikely, it is bothersome to consider this possibility for large
systems.

So what I usually do is concatenate the MD5 with the original identifier...

MD5(id) + id

which assures that the rowkey is both randomly distributed and unique.

Is this necessary, or is it the common practice to just use the MD5
checksum itself?

Thanks,

Jon
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB