request的基本使用

Requests 的官方中文文档如下

Requests: 让 HTTP 服务人类 — Requests 2.18.1 文档

Requests 是 Python 中的一个 HTTP 请求库

首先要下载好 Requests 库

pip install requests

发送 GET 请求

首先我们来尝试一个例子——获取百度首页的数据

import requests

url = "https://www.baidu.com/"
# 请求百度首页
response = requests.get(url)
print(response.text)

得到的结果很长，部分内容如下

s.svnMod+":"+modName}}else if(mType=="[object Array]"){for(var i=0,
len=modName.length;i<len;i++){if(modName[i].indexOf(":")<0){modName[i]=this.svnMod+":"+modName[i]}}}evt.on(modName,evtName,handler)});F._setContextMethod("unListen",function(modName,evtName,handler){evt.un(this.svnMod+":"+this.modName,evtName,handler)})});F._loadScriptTimeout=15e3;F._useConfig=true;F._firstScreenCSS=F._firstScreenCSS||[];F._firstScreenCSS.push("/css/super_min-6f52ab3317.css");F._firstScreenJS=F._firstScreenJS||[];F._firstScreenJS.push("/js/min_super-804cf403f6.js");</script><script>
$(window).on("load", function() {
    var e = $('#virus-2020');
    e.click(function() {
        $.setCookie('virus-2020-clicked', '1');
        e.removeClass('dot');
    });
    var hasClicked = $.getCookie && $.getCookie('virus-2020-clicked');
    if (!hasClicked) {
        e.addClass('dot');
    }
});
</script><script src="https://pss.bdstatic.com/static/superman/js/s_super_index-3fffae8d60.js"></script><script src="https://pss.bdstatic.com/static/superman/js/min_super-804cf403f6.js"></script><script> 
        if(navigator.cookieEnabled){
            document.cookie="NOJS=;expires=Sat, 01 Jan 2000 00:00:00 GMT";
        }
        </script><script src="https://pss.bdstatic.com/static/superman/js/components/hotsearch-644cff7480.js"></script><script defer src="//hectorstatic.baidu.com/cd37ed75a9387c5b.js"></script></body></html>

requests 的其他属性

import requests

url = "http://www.baidu.com/"
# 请求百度首页
response = requests.get(url)

# 获取响应状态码
status = response.status_code
# 获取对应的请求头
request_head = response.request.headers
# 获取响应头
head = response.headers
# 获取响应的cookie
cookie = response.cookies

print(status)
print(request_head)
print(head)
print(cookie)

得到的结果为

200
{'User-Agent': 'python-requests/2.32.3', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}
{'Bdpagetype': '1', 'Bdqid': '0xd97812d1002eeb05', 'Connection': 'keep-alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html; charset=utf-8', 'Date': 'Mon, 15 Jul 2024 12:04:00 GMT', 'Server': 'BWS/1.1', 'Set-Cookie': 'BIDUPSID=85A1D546E37C7500DA0C6CCA9E36DFDC; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com, PSTM=1721045040; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com, H_PS_PSSID=60275_60362_60359; path=/; expires=Tue, 15-Jul-25 12:04:00 GMT; domain=.baidu.com, BDSVRTM=11; path=/, BD_HOME=1; path=/, BAIDUID=85A1D546E37C7500DA0C6CCA9E36DFDC:FG=1; Path=/; Domain=baidu.com; Max-Age=31536000, BAIDUID_BFESS=85A1D546E37C7500DA0C6CCA9E36DFDC:FG=1; Path=/; Domain=baidu.com; Max-Age=31536000; Secure; SameSite=None', 'Traceid': '1721045040275433959415670295592296180485', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'X-Xss-Protection': '1;mode=block', 'Transfer-Encoding': 'chunked'}      
<RequestsCookieJar[<Cookie BIDUPSID=85A1D546E37C7500DA0C6CCA9E36DFDC for .baidu.com/>, <Cookie PSTM=1721045040 for .baidu.com/>, <Cookie H_PS_PSSID=60275_60362_60359 for .baidu.com/>, <Cookie BAIDUID=85A1D546E37C7500DA0C6CCA9E36DFDC:FG=1 for .baidu.com/>, <Cookie BAIDUID_BFESS=85A1D546E37C7500DA0C6CCA9E36DFDC:FG=1 for .baidu.com/>, <Cookie BDSVRTM=11 for www.baidu.com/>, <Cookie BD_HOME=1 for www.baidu.com/>]>

下载图片

然后，我们来尝试一个简单的例子——下载一个图片，首先我们要准备好图片的地址

src=http://pics5.baidu.com/feed/caef76094b36acafbd4f6c0abe60df1e03e99ce5.jpeg@f_auto?token=cc20e191f31e78a3b57e0440d8a3005a&refer=http://www.baidu.com&app=2021&size=f360,240&n=0&g=0n&q=75&fmt=auto (312×208)

我们的代码是如下形式

import requests

url = "https://gimg3.baidu.com/search/src=http%3A%2F%2Fpics5.baidu.com%2Ffeed%2Fcaef76094b36acafbd4f6c0abe60df1e03e99ce5.jpeg%40f_auto%3Ftoken%3Dcc20e191f31e78a3b57e0440d8a3005a&refer=http%3A%2F%2Fwww.baidu.com&app=2021&size=f360,240&n=0&g=0n&q=75&fmt=auto?sec=1721235600&t=284d970f8c9d4fb8683a1d580a97634e"

# 模仿浏览器地址输入URL，到达欺骗网站的效果
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 Edg/126.0.0.0"
}

response = requests.get(url, headers=headers)

with open("baidu.jpg", "wb") as f:
    f.write(response.content)

其中我们引入请求头，这样可以模仿浏览器访问网址的方式，从而避免被阻拦。否则，有些网站会识别出是爬虫进入，从而拒绝你的访问，导致爬虫失败

可以看到图片已经下载完成

下载音乐

注意，此处演示的是非 VIP 音乐，请不要将爬虫技术用于非法用途

首先准备好音频地址

m10.music.126.net/20240716094006/346e9db2c883254500f37dae78c6552e/ymusic/8ad2/b067/a314/33065a696014694ef97a704d7fe54acb.mp3

import requests

url = "https://m10.music.126.net/20240716094006/346e9db2c883254500f37dae78c6552e/ymusic/8ad2/b067/a314/33065a696014694ef97a704d7fe54acb.mp3"

# 模仿浏览器地址输入URL，到达欺骗网站的效果
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 Edg/126.0.0.0"
}

response = requests.get(url, headers=headers)

with open("多远都要在一起.mp3", "wb") as f:
    f.write(response.content)

最后在文件中可以看到音频已经下载完成

注意：文件的格式要和原文件的格式一致