algorithm/Python
1
0
Fork 0
mirror of https://github.com/TheAlgorithms/Python.git synced 2025-07-06 02:13:15 +08:00
Code Issues Packages Projects Releases Wiki Activity

Lukazlim: Replace dependency requests with httpx (#12744)

Browse Source 
* Replace dependency `requests` with `httpx`

Fixes #12742
Signed-off-by: Lim, Lukaz Wei Hwang <lukaz.wei.hwang.lim@intel.com>

* updating DIRECTORY.md

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Lim, Lukaz Wei Hwang <lukaz.wei.hwang.lim@intel.com>
Co-authored-by: Lim, Lukaz Wei Hwang <lukaz.wei.hwang.lim@intel.com>
Co-authored-by: cclauss <cclauss@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Christian Clauss
2025-05-14 03:42:11 +02:00
committed by GitHub
parent 6e4d1b3765
commit a2fa32c7ad
40 changed files with 957 additions and 640 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
web_programming/crawl_google_results.py
View File

@ -1,14 +1,28 @@
# /// script
# requires-python = ">=3.13"
# dependencies = [
# "beautifulsoup4",
# "fake-useragent",
# "httpx",
# ]
# ///
import sys
import webbrowser
import requests
import httpx
from bs4 import BeautifulSoup
from fake_useragent import UserAgent
if __name__ == "__main__":
print("Googling.....")
url = "https://www.google.com/search?q=" + " ".join(sys.argv[1:])
res = requests.get(url, headers={"UserAgent": UserAgent().random}, timeout=10)
res = httpx.get(
url,
headers={"UserAgent": UserAgent().random},
timeout=10,
follow_redirects=True,
)
# res.raise_for_status()
with open("project1a.html", "wb") as out_file: # only for knowing the class
for data in res.iter_content(10000):