Name: Urllib - GET Requests || Python Tutorial || Learn Python Programming
Uploaded: 2020-03-06T17:42:17.000Z
Duration: 7 min 34 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

There are more pages on the web than people on Earth.

And while I haven’t checked, I am sure each one is full of original, high quality content

Most people access web pages through a browser, but as programmers we have other methods...

Today, we will learn how to use Python to send GET requests to web servers, and then

This way you can write software to read websites for you,

giving you more time to browse the internet.

In a browser, you access a web page by typing the URL in the address bar.

URL stands for “Uniform Resource Locator” and this string can hold a LOT of information.

At the beginning is the protocol, which is sometimes called the scheme.

Sometimes you will see a colon followed by a number.

If the port is not explicitly specified, you can determine it from the protocol.

HTTP uses port 80, while HTTPS uses port 443.

The text after the question mark is called the “query string”.

It holds a collection of key-value pairs separated by ampersands.

And lastly, you may see a hashtag at the end followed by a string.

This value is called a fragment and is used to jump to a section within the webpage.

Python 3 comes equipped with a package that simplifies the task of building, loading and

parsing URLs The “URL LIB” package...

This package contains five modules: request, response, error, parse and robotparser

The request module is used to open URLs The response module is used internally by

the request module - you will not work with this directly

The error module contains several error classes for use by the request module

The parse module has a variety of functions for breaking up a URL into meaningful pieces,

like the scheme, host, port, and query string data.

An exciting name, for a less than exciting module...

It is used to inspect robots.txt (“robots-dot-t-x-t”) files for what permissions are granted to

Today we will focus on the request module, since this is where the action lies.

Now use the “directory” function to see what is available.

This is because urllib is a package holding the modules that do the actual work.

So instead, import the module inside urllib that you want to use.

If you call the directory function on the request module, you will see a lot of classes

The function which enables you to easily open a specific URL is the “urlopen” function.

Just as the “open” function is used to open files,

As an example, let us open the home page for Wikipedia.

The function returns a “response” object.

If you look at the type, you will see it is NOT the response in the urllib package, but

a different type of response from a different package.

To see what you can do with the response, use the directory function.

First, let us check if the request was successful by looking at the response code.

A 200 response code means everything went OK.

You may ask why the number 200 was chosen.

Next, let us see how large the response is.

This is the size of the response in BYTES.

We can use the “peek” function to look at small part of the response, rather than

This most definitely looks like HTML, but notice that this is not a string.

The “b” at the beginning tells us this is a “bytes object”

The reason for this is that web servers can host binary data

If you look at the type, it is indeed a bytes object.

We can convert this to text by decoding it.

If you look at the peek value, the character set in the response is “UTF-8”

So to decode this bytes object, call the “decode” method and specify the encoding that was used.

And if you display the value, you can see all the HTML for the web page.

By the way, look what happens if you try to read the response a second time.

This is because once you read the response, Python closes the connection.

As a second example, let us send a search request to Google.

Earlier we said that a 200 response code meant everything was OK.

A 403 response code means that while our request was valid, the server is refusing to respond.

If they let anyone scrape their search results without restriction, then competitors would

We will now load the YouTube page for this incredible video on Black Holes.

Notice that this URL contains two parameters in the query string: V and T.

V is the video ID, and T is the time in the video to begin playback.

One way to construct the querystring is to append a lot of strings together.

To see this, import the “parse” module.

Looking at the directory, you can see a large collection of functions for working with URLs.

Here, we will use the “urlencode” function.

First, we create a dictionary containing the querystring parameters.

The result is a string that is suitable for use as the querystring.

Notice, however, that the question mark is NOT included.

Next, open the url using the “urlopen” method.

Subtitles ListPlay Video

Urllib - GET Requests || Python Tutorial || Learn Python Programming

subscribe

access

determine

character