Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa, mail # dev - questions about pig


Copy link to this message
-
Re: questions about pig
Ariel Rabkin 2011-01-05, 02:05
Hm.

Table is biggish; awk. to scan by hand. Can we modify the script to
ignore empty rows?

--Ari

On Tue, Jan 4, 2011 at 8:35 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> This looks like the row key is empty after parsing.  What does the row key look like in SystemMetrics table?
> The expected format is:
>
> 1234567890000-hostname
>
> Make sure there is no empty row key in SystemMetrics table.
>
> Regards,
> Eric
>
> On 1/4/11 5:09 PM, "Ariel Rabkin" <[EMAIL PROTECTED]> wrote:
>
> So I have pig+hbase running. Thanks so much!
>
> But now I get the following error, from the System Metrics aggregation:
>
> java.io.IOException: java.lang.IllegalArgumentException: Row key is invalid
>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:438)
>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:401)
>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:381)
>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
>        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
>        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216)
> Caused by: java.lang.IllegalArgumentException: Row key is invalid
>        at org.apache.hadoop.hbase.client.Put.<init>(Put.java:79)
>        at org.apache.hadoop.hbase.client.Put.<init>(Put.java:69)
>        at org.apache.pig.backend.hadoop.hbase.HBaseStorage.putNext(HBaseStorage.java:355)
>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:138)
>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97)
>        at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:508)
>        at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:436)
>        ... 7 more
>
>
> Thoughts?
>
>
>
> --
> Ari Rabkin [EMAIL PROTECTED]
> UC Berkeley Computer Science Department
>
>

--
Ari Rabkin [EMAIL PROTECTED]
UC Berkeley Computer Science Department