Scrapy 302 error

Scrapy 302 error


in setting.py change COOKIES_ENABLED to be true. 

Linux buffers , cache youtube video

Linux buffers , cache youtube video


https://www.youtube.com/watch?v=eqahEvCb8NM

stderr, stdout, stdin , how to output to linux file

http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-3.html

There are 3 file descriptors, stdin, stdout and stderr (std=standard).

Basically you can:

1. redirect stdout to a file
2. redirect stderr to a file
3. redirect stdout to a stderr
4. redirect stderr to a stdout
5. redirect stderr and stdout to a file
6. redirect stderr and stdout to stdout
7. redirect stderr and stdout to stderr

I  stderr to a file. 

scrapy crawl XXX 2> nohup.txt

or 

scrapy crawl XXX 2>> nohup.txt

">>"  means append. 

Scrapy transport item over several parse

Scrapy transport item over several parse

https://stackoverflow.com/questions/14798639/scrapy-transport-item-over-several-parse-and-collect-data

Scrapy with multiple callbacks.

Scrapy with multiple callbacks.


https://stackoverflow.com/questions/23695716/scrapy-crawlspider-rules-with-multiple-callbacks

Scrapy Spider, one url, multiple request sample code

class PabhSpider(CrawlSpider):
    name = 'pabh'
    allowed_domains = ['xxx']

    def start_requests(self):
        url = 'http://xxx'
        num1 = '01'
        formdata = {
            "depart":num,
            "years":'2014'
        }
        return [FormRequest(url=url,formdata=formdata,method='get',callback=self.parse)]


    def parse(self, response):
        item = XXXItem()
        hxs = Selector(response)
        item['bh'] = hxs.xpath('/html/body/form/p/font/select[3]/option/@value').extract()
        yield item

        num = ['02','03','04','05','06','07','08','09','10','11','12','13','14','21','31','40','51','61']

        for x in  num:
            url = 'http://xxx'
            formdata={
                "depart":x,
                "years":'2014'
            }
            yield FormRequest(url=url,formdata=formdata,method='get',callback=self.parse)

how to get rid of garbage characters when opening a txt file with excel and the txt file has east Asian characters

Sometimes when we open a txt file with excel  and the txt file has east Asian characters, we will see some garbage characters. How to get rid of them.

To open a txt file with excel. First open an empty excel, then click File=>Open=> Go the the file you want to open => click open.

Some ones say  if we open it with  option Windows(ANSI) we will get rid of garbage characters. But I tried my file with Windows(ANSI), did not get rid of garbage characters.

So I tried some other options, I tried Unicode (UTF-8) and got rid of garbage characters.