Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> oddness with large numeric map[] values


Copy link to this message
-
RE: oddness with large numeric map[] values
In the second case, there is a warning message:

- Encountered Warning FIELD_DISCARDED_TYPE_CONVERSION_FAILED 1 time(s).

The conversion of the text data into a map failed for some reason. If
you examine the system logs on the job tracker UI, you should probably
see why the conversion failed. You can also achieve the same result by
turning warning aggregation off with the -w option.

Santhosh
 

-----Original Message-----
From: Guy Bayes [mailto:[EMAIL PROTECTED]]
Sent: Saturday, July 18, 2009 9:37 PM
To: [EMAIL PROTECTED]
Subject: oddness with large numeric map[] values

Hello all, new to this list, new to pig, running into some odd behavior
with
map[] data types.

Please forgive me if these are known issues or problems with my syntax,

What am i doing wrong here? missing some cast somewhere?

This works:

grunt> cat data2
[apache#2000000000000000000000zzz,foo#foo1]
grunt> T1 = load 'data2' as (f1:map[]);
grunt> dump T1;
2009-07-18 21:35:11,871 [Thread-11] WARN
org.apache.hadoop.mapred.JobClient
- Use GenericOptionsParser for parsing the arguments. Applications
should
implement Tool for the same.
2009-07-18 21:35:21,889 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- 100% complete
2009-07-18 21:35:21,889 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- Success!
([foo#foo1,apache#2000000000000000000000zzz])

This doesn't return anything.

grunt> cat data
[apache#2000000000000000000000,foo#foo1]
grunt> T1 = load 'data' as (f1:map[]);
grunt> dump T1;
2009-07-18 21:36:09,873 [Thread-21] WARN
org.apache.hadoop.mapred.JobClient
- Use GenericOptionsParser for parsing the arguments. Applications
should
implement Tool for the same.
2009-07-18 21:36:19,886 [main] WARN
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- Encountered Warning FIELD_DISCARDED_TYPE_CONVERSION_FAILED 1 time(s).
2009-07-18 21:36:19,886 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- 100% complete
2009-07-18 21:36:19,886 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- Success!
()
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB