Back To Module Home

Advanced Concepts in Python

0% completed

An Intro to Argument Parsing using argparse

How To Process Command Line Arguments?Adding Arguments Options That Conflict

The collections module

Collections Module in Python Counter defaultdict deque namedtuple OrderedDict Quiz Yourself on Collections Module

Context Managers

Context Manager and Contextlib in Python Contextlib and Its Classes ExitStack and Reentrant Context Managers

The functools module

Introduction to functools functool.partial Function Overloading with functools.singledispatch functools.wraps Quiz Yourself on Functools Module

All About Imports

Introduction To Imports Using "from module import something"Relative Imports Optional and Local Imports Import Pitfalls Quiz Yourself on Python Imports

The importlib Module

importlib Package Dynamic Imports Module Import Checking Import From Source File Import from github.com

Iterators and Generators

Introduction to Iterators Creating Our Own Iterators Generators Quiz Yourself on Iterators and Generators

The itertools Module

Iterators Iterators That Terminate compress(), dropwhile() and filterfalse()groupby(), islice() and starmap()takewhile(), tee() and zip_longest()The Combinatoric Generators Quiz Yourself on the Iterators Module

Regular Expressions

Introduction to Regular expressions The Matching Characters Pattern Matching Using Search and Escape Codes Compiling Compilation Flags Finding Multiple Instances Backslashes Are Complicated Quiz Yourself on Regular Expressions

The typing Module

Introduction to Typing Module Type Hints and Overloaded Functions

Python Built-ins

Introduction to Built-ins enumerate and eval filter and map zip Quiz Yourself on Python Built-ins

Unicode

Introduction to Unicode Encoding / Decoding

Benchmarking

Introduction to Benchmarking Use a decorator cProfile line_profiler memory_profiler profilehooks Quiz Yourself on Benchmarking

Encryption and Cryptography

Introduction to Encryption and Cryptography Key Derivation PyCrypto The Cryptography Package

Databases

Introduction to Databases in Python pyodbc and pypyodbc MySQL PostgreSQL Object Relational Mappers NoSQL and MongoDB Quiz Yourself on Databases

The super built-in

Introduction to Super Built-in Python 2 vs. Python 3 Method Resolution Order (MRO)Quiz Yourself on Super Built-in

Descriptors

Descriptors in Python and its Protocol Descriptor Examples

Scope

Scope in Python Local Scope Global Scope nonlocal Scope Quiz Yourself on Scope in Python

Web Scraping

Introduction to Web Scraping Preparing to Scrape BeautifulSoup Scrapy Quiz Yourself on Web Scraping

Web APIs

Introduction to APIs Twitter Reddit Wikipedia Other API Libraries

Working with FTP

Introduction to FTP and ftplib Downloading and Uploading a File via FTP

The urllib Module

urllib module in Python

urllib.robotparser

Quiz Yourself on urllib Module

The doctest Module

Introduction to doctest Module Working of doctest Module Check Your Test With a Text File Option Flags and Directives Quiz Yourself on doctest Module

The unittest Module

unittest Module in Python A Simple Example of unittest Module Command-Line Interface Creating a More Complex Test Creating Test Suites How to Skip Tests Integrating with doctest Quiz Yourself on unittest Module

The mock Module

mock module in Python Simple Examples of mock Module Side Effects of mock Module Autospeccing and Patch Quiz Yourself on mock Module

An Intro to coverage.py

Coverage.py Additional Information

The asyncio Module

Introduction to asyncio Module async and await A Bad Coroutine Example A Better Coroutine Example Scheduling Calls Tasks Quiz Yourself on asyncio Module

The threading Module

Introduction to Threading Module Locks and Synchronization Timers Other Thread Components Thread Communication Quiz Yourself on Threading Module

The multiprocessing Module

Multiprocessing in Python Locks and Logging The Pool Class Process Communication Quiz Yourself on Multiprocessing Module

The concurrent.futures Module

concurrent.futures module Creating a Pool Deadlocks

Conclusion

...

/

urllib.request

urllib.request

Let's explore urllib.request and its function.

The urllib.request module is primarily used for opening and fetching URLs. Let’s take a look at some of the things we can do with the urlopen function:

Press + to interact

import urllib.request
url = urllib.request.urlopen('https://www.google.com/')
print (url.geturl())
#'https://www.google.com/'
print (url.info())
#<http.client.HTTPMessage object at 0x7fddc2de04e0>
header = url.info()
print (header.as_string())
print (url.getcode())
#200

Here we import our module and ask it to open Google’s URL. Now we have an HTTPResponse object that we can interact with. The first thing we do is call the geturl method which will return the URL of the resource that was retrieved. This is useful for finding out if we followed a redirect.

Next we call info, which will return meta-data about the page, such as headers. Because of this, we assign that result to our ...