Yadda ake amfani da proxy a Python Requests (HTTP/HTTPS & Session pooling)
Updated:
Laburaren requests na Python yana karɓar proxies na VinaProxy masu tantancewa ta hanyar kamus na proxies wanda ke haɗa tsarin 'http' da 'https' biyu zuwa http://user:pass@ip:port. Haɗa wannan kamus zuwa requests.Session() mai dorewa yana sake amfani da hanyoyin TCP na ƙasa don yin web scraping cikin sauri da ƙarancin nauyi.
proxies = {'http': 'http://user:pass@ip:port', 'https': 'http://user:pass@ip:port'}- IP 1₫ 40,000
- IPs 5+₫ 35,000
- IPs 10+₫ 25,000
- IPs 20+₫ 25,000
- IPs 50+₫ 25,000
- IPs 100+₫ 25,000
- IPs 200+₫ 25,000
- IPs 500+₫ 25,000
- IPs 1000+₫ 22,000
- IPs 2000+₫ 20,000
- IP 1₫ 40,000
- IPs 5+₫ 35,000
- IPs 10+₫ 25,000
- IPs 20+₫ 25,000
- IPs 50+₫ 25,000
- IPs 100+₫ 25,000
- IPs 200+₫ 25,000
- IPs 500+₫ 25,000
- IPs 1000+₫ 22,000
- IPs 2000+₫ 20,000
Tsara kamus na proxies & sake amfani da Sessions
Kiran requests.get() a kowane URL yana ƙirƙirar sabuwar haɗin TCP da musayar hannu ta TLS a kowane lokaci. Haɗa proxy ɗinka zuwa session = requests.Session() yana barin hanyar proxy a buɗe a tsakanin buƙatu masu biyo baya.
| Parameter na Requests | Tsarin da aka Ba da Shawara | Manufar Fasaha |
|---|---|---|
Mabuɗin 'http' | 'http://user:pass@ip:port' | Yana tura buƙatun HTTP na yau da kullum ta hanyar proxy |
Mabuɗin 'https' | 'http://user:pass@ip:port' | Yana amfani da tsarin http:// domin urllib3 ta samar da hanyar HTTP CONNECT don TLS |
Tuple na timeout | timeout=(5, 15) | Lokacin connect timeout na 5s da read timeout na 15s yana hana worker threads makalewa |
| Juyawar pool | itertools.cycle(proxy_list) | Yana zagayawa daidai a cikin rukunin dedicated static IP ɗinka |
Kauce wa kura-kuran da aka saba gani na urllib3 & ProxyError
- A koyaushe yi amfani da
http://(bahttps://ba) azaman tsarin URL na proxy don mabuɗan kamus na'http'da'https'duka, sai dai idan tashar proxy ɗinka tana buƙatar TLS-over-TLS a sarari - Yi URL-encode ga haruffa na musamman a kalmomin sirri ta amfani da
urllib.parse.quote(password, safe='') - Kuskuren
407 Proxy Authentication Requiredyana nufin sunan mai amfani ko kalmar sirri a cikin rubutun URL ɗinka bai dace da bayanan shiga na dashboard ɗinka ba - Yana aiki daidai da
httpx.Client(proxy='http://user:pass@ip:port')daaiohttpsession.get(url, proxy=...)
Mataki-mataki
- 1
Maida ip:port:user:pass zuwa tsarin URL
Yi amfani da Proxy Format Converter ɗinmu don canza jerin VinaProxy ɗinka zuwa rubutun
http://user:pass@ip:port. - 2
Bayyana kamus na proxies
Saita
proxies = {'http': 'http://user:pass@ip:port', 'https': 'http://user:pass@ip:port'}a cikin Python script ɗinka. - 3
Haɗa zuwa ɓangaren requests.Session()
Fara
s = requests.Session()sannan kiras.proxies.update(proxies)don dorewar haɗinKeep-Alive. - 4
Tabbatar da IP na fita tare da buƙatar gwaji
Gudanar da
print(s.get('https://vinaproxy.com/api/my-ip', timeout=10).json())don tabbatar da IP na proxy mai aiki.
Tambayoyin da aka fi yi
Me ya sa mabuɗin 'https' ke amfani da 'http://user:pass@ip:port'?
Ƙarin farko na http:// yana gaya wa urllib3 yadda za ta haɗu da uwar garken proxy da kanta (ta amfani da daidaitaccen HTTP CONNECT), yayin da zirga-zirgarka zuwa shafin yanar gizo na https:// da aka nufa take kasancewa a ɓoye gaba ɗaya da TLS.
Ta yaya zan juya static proxies 25 a tsakanin buƙatu a Python?
Loda adiresoshin URL na proxy 25 ɗinka zuwa pool = itertools.cycle(PROXIES) sannan tura proxies={'http': p, 'https': p} a kowace buƙata tare da kiyaye ƙa'idar rage sauri ta kowane IP.
Shin VinaProxy tana auna bandwidth akan rubutun scraping na Python?
A'a. Duk tsare-tsaren Dedicated IPv4 da Static ISP sun haɗa da zirga-zirgar bayanai marar iyaka a kowane wata.
Gina rukunin scraping na Python marar iyaka
Proxies na Dedicated IPv4 da Static ISP nan take a cikin ƙasashe 7.
Sayi Python Proxies