Python Requests Modules

 Requests Module:

Request Module is used for sending the HTTP request means its simplify the way to send the data to HTTP request and handles back the response.

By using request method you can perform the CRUD operation easily.

HTTP request methods are GET, POST, PUT, PATCH, and DELETE.

Installation of Requests Modules:

By pip:

pip install requests

By Cloning the source code:

git clone git://github.com/psf/requests.git

After download the package , you have to copy the package inside your package directory

then 

cd requests

pip install

Ways to check the Requests module successfully installed:

1. pip list -- This will list out all the module installed in your kernel.

2. pip freeze > requirements.txt -- This will write all the modules installed in your kernel to requirements.txt.

Lets start Some practical in Jupyter Notebooks:

checking Pip installed or not

In [6]:

pip

Usage:   
  c:\jupyter_note\virtual_jupter\scripts\python.exe -m pip <command> [options]

Commands:
  install                     Install packages.
  download                    Download packages.
  uninstall                   Uninstall packages.
  freeze                      Output installed packages in requirements format.
  list                        List installed packages.
  show                        Show information about installed packages.
  check                       Verify installed packages have compatible dependencies.
  config                      Manage local and global configuration.
  search                      Search PyPI for packages.
  cache                       Inspect and manage pip's wheel cache.
  index                       Inspect information available from package indexes.
  wheel                       Build wheels from your requirements.
  hash                        Compute hashes of package archives.
  completion                  A helper command used for command completion.
  debug                       Show information useful for debugging.
  help                        Show help for commands.

General Options:
  -h, --help                  Show help.
  --debug                     Let unhandled exceptions propagate outside the
                              main subroutine, instead of logging them to
                              stderr.
  --isolated                  Run pip in an isolated mode, ignoring
                              environment variables and user configuration.
  --require-virtualenv        Allow pip to only run in a virtual environment;
                              exit with an error otherwise.
  -v, --verbose               Give more output. Option is additive, and can be
                              used up to 3 times.
  -V, --version               Show version and exit.
  -q, --quiet                 Give less output. Option is additive, and can be
                              used up to 3 times (corresponding to WARNING,
                              ERROR, and CRITICAL logging levels).
  --log <path>                Path to a verbose appending log.
  --no-input                  Disable prompting for input.
  --proxy <proxy>             Specify a proxy in the form
                              [user:passwd@]proxy.server:port.
  --retries <retries>         Maximum number of retries each connection should
                              attempt (default 5 times).
  --timeout <sec>             Set the socket timeout (default 15 seconds).
  --exists-action <action>    Default action when a path already exists:
                              (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.
  --trusted-host <hostname>   Mark this host or host:port pair as trusted,
                              even though it does not have valid or any HTTPS.
  --cert <path>               Path to PEM-encoded CA certificate bundle. If
                              provided, overrides the default. See 'SSL
                              Certificate Verification' in pip documentation
                              for more information.
  --client-cert <path>        Path to SSL client certificate, a single file
                              containing the private key and the certificate
                              in PEM format.
  --cache-dir <dir>           Store the cache data in <dir>.
  --no-cache-dir              Disable the cache.
  --disable-pip-version-check
                              Don't periodically check PyPI to determine
                              whether a new version of pip is available for
                              download. Implied with --no-index.
  --no-color                  Suppress colored output.
  --no-python-version-warning
                              Silence deprecation warnings for upcoming
                              unsupported Pythons.
  --use-feature <feature>     Enable new functionality, that may be backward
                              incompatible.
  --use-deprecated <feature>  Enable deprecated functionality, that will be
                              removed in the future.
Note: you may need to restart the kernel to use updated packages.

In [7]:

# installing the requests module

In [8]:

pip install requests

Collecting requestsNote: you may need to restart the kernel to use updated packages.

WARNING: You are using pip version 22.0.3; however, version 22.2.2 is available.
You should consider upgrading via the 'c:\jupyter_note\virtual_jupter\scripts\python.exe -m pip install --upgrade pip' command.

  Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Requirement already satisfied: certifi>=2017.4.17 in c:\jupyter_note\virtual_jupter\lib\site-packages (from requests) (2021.10.8)
Requirement already satisfied: charset-normalizer<3,>=2 in c:\jupyter_note\virtual_jupter\lib\site-packages (from requests) (2.0.11)
Requirement already satisfied: idna<4,>=2.5 in c:\jupyter_note\virtual_jupter\lib\site-packages (from requests) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\jupyter_note\virtual_jupter\lib\site-packages (from requests) (1.26.8)
Installing collected packages: requests
Successfully installed requests-2.28.1

In [14]:

#Successfully installed requests-2.28.1 

In [9]:

# warning of updating pip version so lets update

In [11]:

pip install --upgrade pip

Requirement already satisfied: pip in c:\jupyter_note\virtual_jupter\lib\site-packages (22.0.3)
Collecting pip
  Downloading pip-22.2.2-py3-none-any.whl (2.0 MB)
     ---------------------------------------- 2.0/2.0 MB 7.2 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.0.3
    Uninstalling pip-22.0.3:
      Successfully uninstalled pip-22.0.3
Successfully installed pip-22.2.2
Note: you may need to restart the kernel to use updated packages.

In [12]:

#importing the requests modules

In [13]:

import requests  #so requests modules successfully installed

In [15]:

#Crud Operation by using requests

In [16]:

#First we are performing the Get Operation

In [18]:

#We are using our blog link for exploring the get opertion. So lets get started

In [19]:

url = "https://pythonknowledgesharing.blogspot.com/2022/09/python-requests-modules.html"

In [20]:

url

Out[20]:

'https://pythonknowledgesharing.blogspot.com/2022/09/python-requests-modules.html'

In [21]:

# so we are get method of request module
#Syntax -- requests.get("url links")
r = requests.get(url)

In [23]:

# simple type r and check 

In [22]:

r

Out[22]:

<Response [200]>

In [46]:

# Checking the urls is correct or not

In [47]:

r.url

Out[47]:

'https://pythonknowledgesharing.blogspot.com/2022/09/python-requests-modules.html'

In [48]:

r.url== url

Out[48]:

True

In [24]:

#some of the things which you can check with the response

In [25]:

#Response status code

In [26]:

r.status_code

Out[26]:

200

In [27]:

#Response Headers

In [28]:

r.headers

Out[28]:

{'Content-Type': 'text/html; charset=UTF-8', 'Expires': 'Tue, 06 Sep 2022 11:38:26 GMT', 'Date': 'Tue, 06 Sep 2022 11:38:26 GMT', 'Cache-Control': 'private, max-age=0', 'Last-Modified': 'Tue, 06 Sep 2022 11:35:47 GMT', 'ETag': 'W/"82eae109f78d409299501376206e61b0f57e136c17e08d5f1c8daf71e16466ea"', 'Content-Encoding': 'gzip', 'X-Content-Type-Options': 'nosniff', 'X-XSS-Protection': '1; mode=block', 'Server': 'GSE', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'Transfer-Encoding': 'chunked'}

In [29]:

# here you can check the content-type. charset etc.

In [30]:

r.headers.get("Content-Type")

Out[30]:

'text/html; charset=UTF-8'

In [34]:

#Response Content

In [35]:

r.text

Out[35]:

'<!DOCTYPE html>\n<html dir=\'ltr\' lang=\'en\'>\n<head>\n<meta content=\'width=device-width, initial-scale=1\' name=\'viewport\'/>\n<title>Python Requests Modules</title>\n<meta content=\'text/html; charset=UTF-8\' http-equiv=\'Content-Type\'/>\n<!-- Chrome, Firefox OS and Opera -->\n<meta content=\'#eeeeee\' name=\'theme-color\'/>\n<!-- Windows Phone -->\n<meta content=\'#eeeeee\' name=\'msapplication-navbutton-color\'/>\n<meta content=\'blogger\' name=\'generator\'/>\n<link href=\'https://pythonknowledgesharing.blogspot.com/favicon.ico\' rel=\'icon\' type=\'image/x-icon\'/>\n<link href=\'https://pythonknowledgesharing.blogspot.com/2022/09/python-requests-modules.html\' rel=\'canonical\'/>\n<link rel="alternate" type="application/atom+xml" title="PythonKnowledgeSharing - Atom" href="https://pythonknowledgesharing.blogspot.com/feeds/posts/default" />\n<link rel="alternate .......

In [36]:

#response is already decoded in the server , so to checking the encoding

In [37]:

r.encoding

Out[37]:

'UTF-8'

In [38]:

# 'UTF-8' is encoding of the content 

In [41]:

#checking Binary Encoding 

In [42]:

r.content

Out[42]:

b'<!DOCTYPE html>\n<html dir=\'ltr\' lang=\'en\'>\n<head>\n<meta content=\'width=device-width, initial-scale=1\' name=\'viewport\'/>\n<title>Python Requests Modules</title>\n<meta content=\'text/html; charset=UTF-8\' http-equiv=\'Content-Type\'/>\n<!-- Chrome, Firefox OS and Opera -->\n<meta content=\'#eeeeee\' name=\'theme-color\'/>\n<!-- Windows Phone -->\n<meta content=\'#eeeeee\' name=\'msapplication-navbutton-color\'/>\n<meta content=\'blogger\' name=\'generator\'/>\n<link href=\'https://pythonknowledgesharing.blogspot.com/favicon.ico\' rel=\'icon\' type=\'image/x-icon\'/>\n<link href=\'https://pythonknowledgesharing.blogspot.com/2022/09/python-requests-modules.html\' rel=\'canonical\'/>\n<link rel="alternate" type="application/atom+xml" title="PythonKnowledgeSharing - Atom" href="https://pythonknowledgesharing.blogspot.com/feeds/posts/default" />\n<link rel="alternate" type="application/rss+xm..................

In [43]:

#if you server sending the raw response

In [44]:

r.raw

Out[44]:

<urllib3.response.HTTPResponse at 0x1a32a848940>

In [45]:

# This is all about the Get Method with our Blog urls. Lets works with some others urls

Lets working some Payload open source urls:

In [60]:

payload = {'key1': 'value1', 'key2': 'value2'}
r = requests.get('https://httpbin.org/get', params=payload)

In [61]:

r

Out[61]:

<Response [200]>

In [62]:

r.url #lets check the embedding of the urls

Out[62]:

'https://httpbin.org/get?key1=value1&key2=value2'

In [63]:

r.status_code

Out[63]:

200

In [68]:

json_response = r.json()

In [69]:

json_response

Out[69]:

{'args': {'key1': 'value1', 'key2': 'value2'},
 'headers': {'Accept': '*/*',
  'Accept-Encoding': 'gzip, deflate',
  'Host': 'httpbin.org',
  'User-Agent': 'python-requests/2.28.1',
  'X-Amzn-Trace-Id': 'Root=1-63173510-4384452a3eebd980597b54bd'},
 'origin': '59.89.180.191',
 'url': 'https://httpbin.org/get?key1=value1&key2=value2'}

In [71]:

json_response.get("args")

Out[71]:

{'key1': 'value1', 'key2': 'value2'}

In [72]:

json_response.get("headers")

Out[72]:

{'Accept': '*/*',
 'Accept-Encoding': 'gzip, deflate',
 'Host': 'httpbin.org',
 'User-Agent': 'python-requests/2.28.1',
 'X-Amzn-Trace-Id': 'Root=1-63173510-4384452a3eebd980597b54bd'}

This Blog only contain the Simple Get operation with request modules . In further blog will be going more deep inside the requests modules.

So lets started our learning with RK27 ...