Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Fwd: Introducing Watchtower - Light Table for Pig


Copy link to this message
-
Fwd: Introducing Watchtower - Light Table for Pig
Awesome development from our pals at Mortar Data!

Russell Jurney http://datasyndrome.com

Begin forwarded message:

*From:* Scott from Mortar Data <[EMAIL PROTECTED]>
*Date:* August 7, 2013, 8:55:23 AM PDT
*To:* Russell <[EMAIL PROTECTED]>
*Subject:* *Introducing Watchtower - Light Table for Pig*
*Reply-To:* Scott from Mortar Data <[EMAIL PROTECTED]>

 Introducing Watchtower - Light Table for Pig

Introducing Watchtower - Light Table for Pig
 <http://mortardata.us2.list-manage1.com/track/click?u=0fe27b9daa7ad5b83c21a89ff&id=a4c8e132bb&e=769327f9b3>

There are no two ways around it, Hadoop development iterations are slow.
Traditional programmers have always had the benefit of re-compiling their
app, running it, and seeing the results within seconds. They have near
instant validation that what they’re building is actually working. When
you’re working with Hadoop, dealing with gigabytes of data, your
development iteration time is more like hours.

Inspired by the amazing real-time feedback experience of Light
Table<http://mortardata.us2.list-manage1.com/track/click?u=0fe27b9daa7ad5b83c21a89ff&id=ac5adb97d4&e=769327f9b3>,
we’ve built Mortar Watchtower to bring back that almost instant iteration
cycle developers are used to. Not only that, Watchtower also helps surface
the semantics of your Pig scripts, to give you insight into how your
scripts are working, not just that they are working.
 Instant Feedback

Watchtower is a daemon that sits in the background, continuously flowing a
sample of your data through your script while your work. It captures what
your data looks like, and shows how it mutates at each step, directly
inline with your script.

[image: image]
 Better Error Handling

Watchtower helps flush out the errors in your script. A common problem for
Pig developers is making sure your UDFs don’t have errors. When we’re
talking about a job that can take eight hours to run, finding an error in
your UDF that runs at the end of your execution can be painful. Since
Watchtower is constantly sending data through your entire script, errors
are surfaced and displayed instantly. No longer will you be surprised when
at the eleventh hour your Hadoop job fails.

[image: image]
 Watchtower Sees Everything

Watchtower is always looking for changes in your Mortar Project. If you
edit anything that alters the functionality of your script (including data
samples, UDFs, or parameters), Watchtower will know, start recompiling the
results, and almost instantly feed the results back to you. All you have to
do is save.
 Getting Started is Easy

Best of all Watchtower is incredibly simple to get started with. If you’re
already setup with Mortar Development Framework then all you have to do is:

$ mortar plugins:install https://github.com/mortardata/watchtower.git

 Then start watching a script. For example:

$ git clone https://github.com/mortardata/mortar-examples.git
$ cd mortar-examples
$ mortar watch pigscripts/coffee_tweets.pig \
  -f params/coffee_tweets/local.large.params

 We hope you love getting real-time feedback on your Pig scripts. Enjoy!
want to make our engineers cry? unsubscribe
here<http://mortardata.us2.list-manage2.com/unsubscribe?u=0fe27b9daa7ad5b83c21a89ff&id=22111863f9&e=769327f9b3&c=c060ee6777>.
   update subscription
preferences<http://mortardata.us2.list-manage1.com/profile?u=0fe27b9daa7ad5b83c21a89ff&id=22111863f9&e=769327f9b3>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB