Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Review Request 13122: PIG-3400 FS commands do not work with S3 paths


Copy link to this message
-
Re: Review Request 13122: PIG-3400 FS commands do not work with S3 paths
Cheolsoo Park 2013-08-01, 00:30


> On Aug. 1, 2013, 12:06 a.m., Rohini Palaniswamy wrote:
> > src/docs/src/documentation/content/xdocs/cmds.xml, line 169
> > <https://reviews.apache.org/r/13122/diff/1/?file=331562#file331562line169>
> >
> >     fs cat URI. Same for other fs commands.

No, actually "cat URI" is correct. If you look at GruntParser, processCat() is different from processFsCommand(). In fact, we should deprecate one of them, but as of now, they exist.
> On Aug. 1, 2013, 12:06 a.m., Rohini Palaniswamy wrote:
> > src/docs/src/documentation/content/xdocs/cmds.xml, line 191
> > <https://reviews.apache.org/r/13122/diff/1/?file=331562#file331562line191>
> >
> >     We should mention local and hdfs, but should not specifically state s3. With your fix, it will work with any file system implementation. So can mention that. Same for other fs commands.

Sure I will replace s3 with "any other file system".
> On Aug. 1, 2013, 12:06 a.m., Rohini Palaniswamy wrote:
> > src/docs/src/documentation/content/xdocs/cmds.xml, line 296
> > <https://reviews.apache.org/r/13122/diff/1/?file=331562#file331562line296>
> >
> >     White spaces in many places

Will fix.
- Cheolsoo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13122/#review24393
-----------------------------------------------------------
On July 31, 2013, 7:37 p.m., Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13122/
> -----------------------------------------------------------
>
> (Updated July 31, 2013, 7:37 p.m.)
>
>
> Review request for pig.
>
>
> Bugs: PIG-3400
>     https://issues.apache.org/jira/browse/PIG-3400
>
>
> Repository: pig-git
>
>
> Description
> -------
>
> Makes fs utility commands work with s3 paths. Documents fs utility commands in the Pig manual. (They were not documented at all.)
>
>
> Diffs
> -----
>
>   src/docs/src/documentation/content/xdocs/cmds.xml 38babd2
>   src/org/apache/pig/tools/grunt/GruntParser.java c785084
>
> Diff: https://reviews.apache.org/r/13122/diff/
>
>
> Testing
> -------
>
> All unit tests pass.
>
> Manually verified the following commands with s3 paths:
> pig -e 'ls s3://<path>'
> pig -e 'mkdir s3://<path>'
> pig -e 'rm s3://<path>'
> pig -e 'cp s3://<path1> s3://<path2>'
> pig -e 'mv s3://<path1> s3://<path2>'
> pig -e 'copyToLocal s3://<path>/<file> .'
> pig -e 'copyFromLocal <file> s3://<path>'
>
>
> Thanks,
>
> Cheolsoo Park
>
>