|
|
-
Setting Configuration for local file:///
Mohit Anchlia 2012-08-07, 19:40
I am trying to write a test on local file system but this test keeps taking xml files in the path even though I am setting a different Configuration object. Is there a way for me to override it? I thought the way I am doing overwrites the configuration but doesn't seem to be working:
@Test public void testOnLocalFS() throws Exception{ Configuration conf = new Configuration(); conf.set("fs.default.name", "file:///"); conf.set("mapred.job.tracker", "local"); Path input = new Path("geoinput/geo.dat"); Path output = new Path("geooutput/"); FileSystem fs = FileSystem.getLocal(conf); fs.delete(output, true);
log.info("Here"); GeoLookupConfigRunner configRunner = new GeoLookupConfigRunner(); configRunner.setConf(conf); int exitCode = configRunner.run(new String[]{input.toString(), output.toString()}); Assert.assertEquals(exitCode, 0); }
+
Mohit Anchlia 2012-08-07, 19:40
-
Re: Setting Configuration for local file:///
Harsh J 2012-08-07, 19:50
What is GeoLookupConfigRunner and how do you utilize the setConf(conf) object within it?
On Wed, Aug 8, 2012 at 1:10 AM, Mohit Anchlia <[EMAIL PROTECTED]> wrote: > I am trying to write a test on local file system but this test keeps taking > xml files in the path even though I am setting a different Configuration > object. Is there a way for me to override it? I thought the way I am doing > overwrites the configuration but doesn't seem to be working: > > @Test > public void testOnLocalFS() throws Exception{ > Configuration conf = new Configuration(); > conf.set("fs.default.name", "file:///"); > conf.set("mapred.job.tracker", "local"); > Path input = new Path("geoinput/geo.dat"); > Path output = new Path("geooutput/"); > FileSystem fs = FileSystem.getLocal(conf); > fs.delete(output, true); > > log.info("Here"); > GeoLookupConfigRunner configRunner = new GeoLookupConfigRunner(); > configRunner.setConf(conf); > int exitCode = configRunner.run(new String[]{input.toString(), > output.toString()}); > Assert.assertEquals(exitCode, 0); > }
-- Harsh J
+
Harsh J 2012-08-07, 19:50
-
Re: Setting Configuration for local file:///
Mohit Anchlia 2012-08-07, 20:10
On Tue, Aug 7, 2012 at 12:50 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> What is GeoLookupConfigRunner and how do you utilize the setConf(conf) > object within it? Thanks for the pointer I wasn't setting my JobConf object with the conf that I passed. Just one more related question, if I use JobConf conf = new JobConf(getConf()) and I don't pass in any configuration then does the data from xml files in the path used? I want this to work for all the scenarios. > > On Wed, Aug 8, 2012 at 1:10 AM, Mohit Anchlia <[EMAIL PROTECTED]> > wrote: > > I am trying to write a test on local file system but this test keeps > taking > > xml files in the path even though I am setting a different Configuration > > object. Is there a way for me to override it? I thought the way I am > doing > > overwrites the configuration but doesn't seem to be working: > > > > @Test > > public void testOnLocalFS() throws Exception{ > > Configuration conf = new Configuration(); > > conf.set("fs.default.name", "file:///"); > > conf.set("mapred.job.tracker", "local"); > > Path input = new Path("geoinput/geo.dat"); > > Path output = new Path("geooutput/"); > > FileSystem fs = FileSystem.getLocal(conf); > > fs.delete(output, true); > > > > log.info("Here"); > > GeoLookupConfigRunner configRunner = new GeoLookupConfigRunner(); > > configRunner.setConf(conf); > > int exitCode = configRunner.run(new String[]{input.toString(), > > output.toString()}); > > Assert.assertEquals(exitCode, 0); > > } > > > > -- > Harsh J >
+
Mohit Anchlia 2012-08-07, 20:10
-
Re: Setting Configuration for local file:///
Harsh J 2012-08-07, 21:07
If you instantiate the JobConf with your existing conf object, then you needn't have that fear.
On Wed, Aug 8, 2012 at 1:40 AM, Mohit Anchlia <[EMAIL PROTECTED]> wrote: > On Tue, Aug 7, 2012 at 12:50 PM, Harsh J <[EMAIL PROTECTED]> wrote: > >> What is GeoLookupConfigRunner and how do you utilize the setConf(conf) >> object within it? > > > Thanks for the pointer I wasn't setting my JobConf object with the conf > that I passed. Just one more related question, if I use JobConf conf = new > JobConf(getConf()) and I don't pass in any configuration then does the data > from xml files in the path used? I want this to work for all the scenarios. > > >> >> On Wed, Aug 8, 2012 at 1:10 AM, Mohit Anchlia <[EMAIL PROTECTED]> >> wrote: >> > I am trying to write a test on local file system but this test keeps >> taking >> > xml files in the path even though I am setting a different Configuration >> > object. Is there a way for me to override it? I thought the way I am >> doing >> > overwrites the configuration but doesn't seem to be working: >> > >> > @Test >> > public void testOnLocalFS() throws Exception{ >> > Configuration conf = new Configuration(); >> > conf.set("fs.default.name", "file:///"); >> > conf.set("mapred.job.tracker", "local"); >> > Path input = new Path("geoinput/geo.dat"); >> > Path output = new Path("geooutput/"); >> > FileSystem fs = FileSystem.getLocal(conf); >> > fs.delete(output, true); >> > >> > log.info("Here"); >> > GeoLookupConfigRunner configRunner = new GeoLookupConfigRunner(); >> > configRunner.setConf(conf); >> > int exitCode = configRunner.run(new String[]{input.toString(), >> > output.toString()}); >> > Assert.assertEquals(exitCode, 0); >> > } >> >> >> >> -- >> Harsh J >>
-- Harsh J
+
Harsh J 2012-08-07, 21:07
|
|