What is the urllib module in Python 3?
Urllib is a Python 3 package that allows you to access, and interact with, websites using their URL’s (Uniform Resource Locator). It has several modules for working with URL’s, these are shown in the illustration below:
urllib.request
Using urllib.request, with urlopen, allows you to open the specified URL. This is shown in the code snippet below:
from urllib.request import urlopenmyURL = urlopen("http://www.google.com/")print(myURL.read())
Once the URL has been opened, the read() function is used to get the entire HTML code for the webpage.
urllib.parse
The code snippet below shows the usage of urllib.parse:
from urllib.parse import urlparseparsedUrl = urlparse('https://www.educative.io/track/python-for-programmers')print(parsedUrl)
The URL is split into its components such as the protocol scheme used, the network location netloc and the path to the webpage.
urllib.error
This module is used to catch exceptions encountered from url.request. These exceptions, or errors, are classified as follows:
- URL Error, which is raised when the URL is incorrect, or when there is an internet connectivity issue.
- HTTP Error, which is raised because of HTTP errors such as 404 (request not found) and 403 (request forbidden).
The following code snippet shows the usage of
urlib.error:
from urllib.request import urlopen, HTTPError, URLErrortry:myURL = urlopen("http://ww.educative.xyz/")except HTTPError as e:print('HTTP Error code: ', e.code)except URLError as e:print('URL Error: ', e.reason)else:print('No Error.')
A request to open http://ww.educative.xyz/ is caught by the URLError exception; the URL is invalid. Experiment with the exceptions by opening different URL’s.
Free Resources
