-executing linux command from hadoop (python)
jamal sasha 2013-08-15, 23:23
Lets say that I have a data which interacts with a rest api like
%curl hostname data
Now, I have the following script:
cmd = """curl http://localhost --data '"""
string = " "
for line in sys.stdin:
line = line.rstrip(os.linesep)
string += line
os.system(cmd + string+"'")
Now, if i give a sample file for data, and run the above script with
cat data.txt | python mapper.py
It works perfectly. But will this work if i execute on hadoop as well?
I am trying to set up hadoop on local mode to check it out but I think it
will take me sometime to get there?
Any experiences, suggestions?