Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> String Representation of DataBag and its Schema


+
Dan DeCapria, CivicScienc... 2013-03-18, 20:18
Copy link to this message
-
Re: String Representation of DataBag and its Schema
Why not just use PigStorage? This is essentially what it does. It saves a
bag as text, and then loads it again.

I suppose the question becomes: why do you need to do this?
2013/3/18 Dan DeCapria, CivicScience <[EMAIL PROTECTED]>

> In Java, I am trying to convert a DataBag from it's String representation
> with its schema String to a valid DataBag Object:
>
> String databag_string = "{(apples,1024)}";
> String schema_string = "b1:bag{t1:tuple(a:chararray,b:long)}";
>
> I've tried implementing something along the lines of this, but I believe
> it's in the wrong direction, and then I get stuck:
>
>         String[] aliases = {"b1", "t1", "a", "b"};
>         byte[] types = {DataType.BAG, DataType.TUPLE, DataType.CHARARRAY,
> DataType.LONG};
>         List<Schema.FieldSchema> fsList = new
> ArrayList<Schema.FieldSchema>();
>         for (int i = 0; i < aliases.length; i++) {
>             fsList.add(new Schema.FieldSchema(aliases[i], types[i])) ;
>         }
>         Schema origSchema = new Schema(fsList);
>         ResourceSchema rsSchema = new ResourceSchema(origSchema);
>         Schema genSchema = Schema.getPigSchema(rsSchema);
>         ResourceSchema.ResourceFieldSchema[] rfschema > rsSchema.getFields();
>         ... lost here, maybe Utf8StorageConverter c = new
> Utf8StorageConverter(); ???
>
>
> An ideal process would be along the lines of:
>
> DataBag d = BagFactory.getInstance().newDefaultBag();
> d.something(databag_string, schema_string);    // ??? no idea what this
> process could be
> d.toString().equals(databag_string) == true.
>
> Thanks, -Dan
>
+
Dan DeCapria, CivicScienc... 2013-03-19, 13:37
+
Dan DeCapria, CivicScienc... 2013-03-19, 15:16
+
Jonathan Coveney 2013-03-19, 15:27
+
Dan DeCapria, CivicScienc... 2013-03-19, 15:37
+
Jonathan Coveney 2013-03-19, 15:43
+
Dan DeCapria, CivicScienc... 2013-03-19, 15:52
+
Jonathan Coveney 2013-03-19, 16:08
+
Dan DeCapria, CivicScienc... 2013-03-19, 16:40
+
Jonathan Coveney 2013-03-19, 16:53
+
Jonathan Coveney 2013-03-19, 16:54
+
Dan DeCapria, CivicScienc... 2013-03-19, 17:20
+
William Oberman 2013-03-21, 15:51
+
Dan DeCapria, CivicScienc... 2013-03-19, 15:43
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB