Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Spark >> mail # user >> RDD operation examples with data?


Copy link to this message
-
RDD operation examples with data?
Hi,

I'm learning Spark and I am confused about when to use the many different
operations on RDDs. Does anyone have any examples which show example inputs
and resulting outputs for the various RDD operations and if the operation
takes an Function a simple example of the code?

For example, something like this for flatMap

One row -> "the quick brown fox"

Passed to:

JavaRDD<String> words = lines.flatMap(new FlatMapFunction<String, String>() {
      @Override
      public Iterable<String> call(String s) {
        return Arrays.asList(SPACE.split(s));
      }
    });

When completed: words would contain
"the"
"quick"
"brown"
"fox"

(Yes this one is pretty obvious but some of the others aren't).

If such examples don't exist, is there a shared wiki or someplace we
could start building one?

Thanks,

Chris