Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Map having <string, Object>

Gaurav 2011-12-07, 13:16
Tatu Saloranta 2011-12-07, 15:47
Gaurav Nanda 2011-12-07, 17:10
Copy link to this message
Re: Map having <string, Object>
On Wed, Dec 7, 2011 at 9:10 AM, Gaurav Nanda <[EMAIL PROTECTED]> wrote:
> I agree that in this case Json would be equally helpful. But In my
> application there is one more type of message, where untagged data can
> provide compact data encoding. So to maintain consistency, I preferred
> to send these kind of messages also using avro.
> @where untagged data can provide compact data encoding.
> In that case also, my schema has to be dynamically generated (i.e. on
> runtime), so has to be passed to client. So would avro be better to
> compressed json is that case?

It seems to me that hassle of dynamic generation of one-off schemas
would make this bit sub-optimal use case.
Or, conversely, if you just define generic schema that allows sending
of key value pairs (and perhaps type), there is no size benefit as you
add all things that schema would help take out of payload.
Another alternative is to use content-type or related metadata to
allow use of different low-level data formats.

Beyond compressed JSON (which can be very fast with LZF or Snappy),
you could also consider one of binary encodings for Json.

-+ Tatu +-
Scott Carey 2011-12-07, 18:01
Doug Cutting 2011-12-07, 17:36