ezoic

Thursday, May 10, 2018

Scrapy linux cron job not work, how I make it work

I tried to automate  a scrapy job using cron on linux. But it did not work. I searched and found the solution.
First use "which scrapy " to find where the scrapy is. For my machine the scrapy is:

/home/ubuntu/anaconda2/bin/scrapy


Then in the  shell script, write:

cd /path/to/spider 

nohup  /home/ubuntu/anaconda2/bin/scrapy crawl quotes   >> log.txt &

It resolved the problem. Or can write:

cd /path/to/spider 
PATH=$PATH:/home/ubuntu/anaconda2/bin
export PATH
nohup  /home/ubuntu/anaconda2/bin/scrapy crawl quotes   >> log.txt &








No comments:

Post a Comment

looking for a man

 I am a mid aged woman. I live in southern california.  I was born in 1980. I do not have any kid. no compliacted dating.  I am looking for ...