class PabhSpider(CrawlSpider):
name = 'pabh'
allowed_domains = ['xxx']
def start_requests(self):
url = 'http://xxx'
num1 = '01'
formdata = {
"depart":num,
"years":'2014'
}
return [FormRequest(url=url,formdata=formdata,method='get',callback=self.parse)]
def parse(self, response):
item = XXXItem()
hxs = Selector(response)
item['bh'] = hxs.xpath('/html/body/form/p/font/select[3]/option/@value').extract()
yield item
num = ['02','03','04','05','06','07','08','09','10','11','12','13','14','21','31','40','51','61']
for x in num:
url = 'http://xxx'
formdata={
"depart":x,
"years":'2014'
}
yield FormRequest(url=url,formdata=formdata,method='get',callback=self.parse)
I wrote about the solutions to some problems I found from programming and data analytics. They may help you on your work. Thank you.
ezoic
Monday, June 11, 2018
Scrapy Spider, one url, multiple request sample code
Subscribe to:
Post Comments (Atom)
looking for a man
I am a mid aged woman. I was born in 1980. I do not have any kid. no complicated dating before . I am looking for a man here for marriage...
-
I tried to commit script to bitbucket using sourcetree. I first cloned from bitbucket using SSH, and I got an error, "authentication ...
-
https://github.com/boto/boto3/issues/134 import boto3 import botocore client = boto3.client('s3') result = client.list_obje...
-
Previously, I wanted to install "script" on Atom to run PHP. And there was some problem, like the firewall. So I tried atom-runner...
No comments:
Post a Comment