所花时间(包括上课): |
2h |
代码量(行): |
150左右 |
搏客量(篇): |
1 |
了解到的知识点: |
python |
备注(其他): |
进行数据爬取的几个练习
1、简易网页采集器
import requests if __name__ == '__main__':
#UA检测#UA伪装:让爬虫对应的请求载体身份标识伪装成某一款浏览器headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 SLBrowser/9.0.0.10191 SLBChan/103'}url='https://www.sogou.com/web'kw=input('输入字段')param = {'query':kw}response=requests.get(url=url,params=param,headers=headers)page_text =response.textfileName =kw+'.html'with open(fileName, 'w', encoding='utf-8') as fp:fp.write(page_text)print('爬取数据完成')
显示效果