Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # dev >> Review Request: Fix Sqoop unit test failures on Windows


Copy link to this message
-
Re: Review Request: Fix Sqoop unit test failures on Windows


> On April 5, 2013, 4:21 p.m., Venkat Ranganathan wrote:
> > src/java/org/apache/sqoop/util/ClassLoaderStack.java, line 78
> > <https://reviews.apache.org/r/10229/diff/1/?file=276974#file276974line78>
> >
> >     Why is the JarURL format getting changed.   File(jarFile).toURI().toURL() will still have the scheme as file right?
>
> Ahmed El Baz wrote:
>     The objective is to construct the URL path using URL class which resolves the failures on Windows due to the preceding "jar:" and ending "!/" parts. I believe this is also favorable than handcrafting the URL string.
>    
>     Verified this is working on both Windows and Linux
>
> Venkat Ranganathan wrote:
>     Are you saying this does not work on Windows?
>    
>     String urlPath = "jar:file:" + new File(jarFile).getAbsolutePath() + "!/";
>
> Ahmed El Baz wrote:
>     Correct. Using the crafted String urlPath is failing on Windows, thus the need to use the URL class.

That is interesting. remember using it in my earlier work.  I understand that you want to get it consistent across Unix and windows.   Can you please see the content below and try it on Windows and Linux? I don't think the issue is because of jar: and !/ suffixes.   The issue is because file urls crafted with file:// syntax expect the file part to start with / and not a drive letter.   That is why using file: (without // after file) helps.  In any case, URLClassLoader can work with Jar URLs without the prefix if we want the full jarfile contents.   I think there is a windows issue with dynamically replacing jarfiles because of a Java bug.   That is a different issue.

===========import java.io.File;
import java.net.JarURLConnection;
import java.net.URL;
import java.util.Map;
import java.util.jar.Attributes;
import java.util.jar.Manifest;
public class JarURLConn {

public static void main(String args[]) {
if (args.length < 1) {
System.out.println("Usage JavaURLConn jar1 ...");
System.exit(1);
}
for (int i = 0; i < args.length; ++i) {
System.out.println("Information on jar file " + args[i]);
System.out.println("\n\n");
try {
printJarInfo(args[i]);
}
catch (Exception e) {
System.out.println("Caught exception : " + e.toString());
}
}
}


private static void printJarInfo(String jarFile) throws Exception {
File f = new File(jarFile);
if (f.canRead()) {
StringBuilder sb = new StringBuilder();
sb.append("jar:file:");
sb.append(f.getAbsolutePath());
sb.append("!/");
String jarURLStr = sb.toString();
                        System.out.println("Jar URL is " + jarURLStr);
URL url = new URL(jarURLStr);
JarURLConnection conn = (JarURLConnection) url.openConnection();
Manifest manifest = conn.getManifest();
Map<String, Attributes> attrMap = manifest.getEntries();

for (String key : attrMap.keySet()) {
Attributes attrs = attrMap.get(key);
for (Object attrKey : attrs.keySet()) {
System.out.println("Attribute " + attrKey + " = "
+ attrs.get(attrKey));
}
}

}
else {
System.out.println("Can't access jar file " + jarFile);
}

}
}
======

> On April 5, 2013, 4:21 p.m., Venkat Ranganathan wrote:
> > src/test/com/cloudera/sqoop/hive/TestHiveImport.java, line 192
> > <https://reviews.apache.org/r/10229/diff/1/?file=276975#file276975line192>
> >
> >     Nit: Trailing spaces
>
> Ahmed El Baz wrote:
>     Did not see a trailing space in this line, but fixed a trailing space in the next one.
>     Thanks

Can you please make sure after a upload your diff to check if there are no red bars in the beginning or end.   That will signify the space issues.
> On April 5, 2013, 4:21 p.m., Venkat Ranganathan wrote:
> > src/test/com/cloudera/sqoop/orm/TestClassWriter.java, line 177
> > <https://reviews.apache.org/r/10229/diff/1/?file=276979#file276979line177>
> >
> >     Nit - tabs instead of spaces

Please check as mentioned above
- Venkat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10229/#review18705
On April 1, 2013, 8:48 p.m., Ahmed El Baz wrote: