The Evolution of the HTTP Protocol: From 1.0 to 2.0

HTTP (HyperText Transfer Protocol) is the most widely used network protocol on the Internet, and all WWW files need to follow this standard. Its original design goal was to provide a method for publishing and receiving HTML pages, which is used to transfer hypertext from a WWW server to a local browser, and it uses port 80 by default. When an HTTP client initiates a request, it will establish a TCP connection to the specified port of the server (port 80 by default).

It is worth noting that the HTTP protocol does not conflict with the TCP protocol. HTTP is at the application layer of the seven-layer protocol, while TCP is responsible for solving the logical problems of the transport layer. HTTP chooses TCP instead of UDP because a large amount of data needs to be transmitted when opening a web page, and the TCP protocol can provide transmission control to achieve the orderly organization and error correction of data. In addition, the bottlenecks and optimization techniques of the HTTP protocol are also based on the characteristics of the TCP protocol. For example, the three-way handshake when establishing a TCP connection will cause a delay of 1.5 RTTs (round-trip time). To avoid the handshake delay of each request, the application layer will adopt HTTP persistent connection schemes with different strategies; and because TCP has the slow start characteristic in the initial stage of the connection, the performance of connection reuse is usually better than that of a new connection.

The HTTP connection adopts the "request-response" mode. Not only does it need to establish a connection first when making a request, but also the server can only reply with data after the client sends a request to the server. With the development of the Internet, the HTTP protocol has also been continuously upgraded, and versions such as HTTP/1.0, HTTP/1.1, and HTTP/2.0 have emerged successively, and each version has been optimized and improved in terms of performance and functionality.

Comparison Table of HTTP Protocol Versions

Feature	HTTP/1.0	HTTP/1.1	HTTP/2.0
Connection method	Short connection, a new connection is established for each request	Persistent connection, `Connection: keep - alive` is enabled by default	Multiplexing, multiple requests on a single connection
Head of line blocking	Exists, affecting performance	Pipelining mechanism partially alleviates it, but browser support is limited	Solved through multiplexing
Request header support	Does not support the Host header, making it difficult to implement virtual hosts	Supports the Host header and can configure multiple virtual hosts	Supports richer functions
Cache control	`If - Modified - Since`, `Expires`	Adds `Entity tag`, `If - Unmodified - Since`, etc.	Further optimized
Resuming interrupted downloads	Does not support	Supported through `RANGE:bytes`	Supported
Header compression	Does not support	Does not support	Compressed by the HPACK algorithm
Request priority	Does not support	Does not support	Supports
Server push	Does not support	Does not support	Supports

HTTP/1.0

HTTP/1.0 is the first version of the HTTP protocol that specifies a version number in communication and is still widely used in scenarios such as proxy servers. This version stipulates that the browser and the server only maintain a short-term connection. The browser needs to establish a TCP connection with the server for each request, and the server will immediately disconnect the TCP connection after completing the request processing. It does not track each client or record past requests.

This mechanism leads to some performance flaws. Take a web page containing a large number of images as an example. The web page file itself does not contain image data but only specifies the URL address of the image. When the browser accesses this web page, it will first send a request for the web page file. When parsing the HTML content, if it finds an image tag, it will send a request to the server again to download the image data according to the URL specified in the src attribute. This means that accessing a web page with many images requires multiple requests and responses, and each time a separate connection needs to be established to transmit a single document or image. Even if the image files are small, the process of frequently establishing and closing connections is time-consuming and seriously affects the performance of both the client and the server. Similarly, when a web page contains content such as JavaScript files and CSS files, similar problems will also occur.

At the same time, bandwidth and latency are also important factors affecting network requests. With the significant improvement of bandwidth in the current network infrastructure, the latency problem of response speed has become more prominent. The two most criticized problems of HTTP/1.0 are the inability to reuse connections and head of line blocking.

To understand these two problems, it should be clear that the client establishes a connection to the server according to the domain name. Generally, a PC browser will establish 6-8 connections to the server of a single domain name at the same time, while a mobile browser controls it to be 4-6. The number of connections is not the more the better, as too many will increase resource overhead and overall latency. The inability to reuse connections results in a three-way handshake and slow start for each request. The three-way handshake has a significant impact in scenarios with high latency, and the slow start has a greater impact on requests for large files. Head of line blocking will lead to the inability to fully utilize the bandwidth, and subsequent healthy requests will be blocked.

Head of line blocking will cause healthy requests to be affected by unhealthy requests, and this loss is affected by the network environment, which is random and difficult to monitor. To solve the latency caused by head of line blocking, protocol designers introduced the pipelining mechanism, but this mechanism is only applicable to HTTP/1.1 and has strict usage conditions, and many browser vendors do not support it.

In HTTP/1.0, the request header is relatively simple. For example, when obtaining web page resources:

GET /index.html HTTP/1.0
User - Agent: Mozilla/5.0

HTTP/1.1

To overcome the deficiencies of HTTP/1.0, HTTP/1.1 supports persistent connections (the default mode is a persistent connection with pipelining). Multiple HTTP requests and responses can be transmitted over a single TCP connection, reducing the consumption and latency of connection establishment and closure. Take a web page containing a large number of images as an example. Multiple requests and responses can be transmitted in one connection, but the requests and responses for each individual web page file still need to use their own connections. In addition, HTTP/1.1 allows the client to send the next request without waiting for the result of the previous request to be returned, and the server needs to send back the response results in the order of receiving the requests to ensure that the client can distinguish the response content of each request, which significantly reduces the time required for the entire download process.

In HTTP/1.1, the connection header may appear in the request and response headers, which is used to indicate how the client and the server handle the persistent connection. By default, the client and the server assume that the other party supports persistent connections. If the client uses the HTTP/1.1 protocol but does not want to use a persistent connection, it needs to specify the value of connection as close in the header; if the server does not want to support a persistent connection, it also needs to clearly set the value of connection as close in the response. Whether the close value is included in the header of the request or the response, it indicates that the current TCP connection will be disconnected after the request processing is completed, and subsequent new requests from the client need to create a new TCP connection. For example:

# Request header
GET /index.html HTTP/1.1
User - Agent: Mozilla/5.0
Connection: close

# Response header
HTTP/1.1 200 OK
Content - Type: text/html
Connection: close

In addition, HTTP/1.1 has improved and expanded the functions of HTTP/1.0 by adding more request headers and response headers:

Host header support: HTTP/1.0 does not support the Host request header field, and the browser cannot specify the specific WEB site on the server to access through the host header name, which limits the configuration of multiple virtual WEB sites on the same IP address and port number. After HTTP/1.1 adds the Host request header field, the browser can clearly specify the site to access through the host header name, enabling the creation of multiple virtual WEB sites with different host names on the same IP address and port number. For example:

GET /index.html HTTP/1.1
Host: example.com

Persistent connection control: The persistent connection of HTTP/1.1 is implemented through new request headers. When the value of the Connection request header is Keep - Alive, the client notifies the server to keep the connection after returning the result of this request; when the value is close, it notifies the server to close the connection after returning the result.
Caching and resuming interrupted downloads: HTTP/1.0 does not support resuming interrupted downloads of files, and each file transfer starts from the 0-byte position. HTTP/1.1 adds a new RANGE:bytes field, and RANGE:bytes=XXXX means requesting the server to start the transfer from the XXXX-byte position of the file, enabling resuming interrupted downloads. For example:

GET /largefile.zip HTTP/1.1
Range: bytes=1000 -

In addition, HTTP/1.1 has also been improved in terms of cache processing, bandwidth optimization, error notification management, etc.:

Cache processing: HTTP/1.0 mainly uses If - Modified - Since and Expires as cache judgment criteria, and HTTP/1.1 has introduced more cache control strategies, such as Entity tag, If - Unmodified - Since, If - Match, If - None - Match, etc.
Bandwidth optimization: There is a phenomenon of bandwidth waste in HTTP/1.0, and it does not support resuming interrupted downloads. HTTP/1.1 introduces the range header field in the request header, allowing only part of the resource to be requested, and the return code is 206 (Partial Content), which improves the bandwidth utilization rate.
Error notification: HTTP/1.1 adds 24 new error status response codes, such as 409 (Conflict) indicating that the requested resource conflicts with the current state; 410 (Gone) indicating that the resource on the server has been permanently deleted.

SPDY: The Precursor of HTTP/2.0

SPDY is not a standard protocol. Its development team promoted it to become an Internet draft, but it ultimately failed to become an official standard on its own. However, the members of the SPDY development team participated in the entire process of formulating HTTP/2.0. The key functions of HTTP/2.0 mainly originated from SPDY technology, and it can be considered that the achievements of SPDY were adopted and evolved into HTTP/2.0. In September 2015, Google announced the removal of support for SPDY and instead embraced HTTP/2.0, which took effect in Chrome 51.

SPDY does not replace HTTP but modifies the way HTTP requests and responses are transmitted over the network. Just adding an SPDY transport layer, the existing server-side applications do not need to make any modifications. When using SPDY for transmission, HTTP requests will be processed, marked, simplified, and compressed.

SPDY has the following new features:

Multiplexing: Multiple request streams share a single TCP connection, solving the HOL blocking problem, reducing latency, and improving bandwidth utilization.
Request priority: Allows setting a priority for each request to ensure that important requests are responded to first. For example, when the browser loads the home page, it can give priority to displaying the HTML content and then load static resources and script files.
Header compression: The headers in HTTP/1.x often contain duplicate and redundant information. SPDY selects an appropriate compression algorithm to reduce the size and number of packets.
Encrypted protocol transmission based on HTTPS: Improves the reliability of the transmitted data.
Server push: For a web page using SPDY, when the client requests sytle.css, the server will push sytle.js to the client. When the client subsequently obtains sytle.js, it can directly retrieve it from the cache without making another request.

HTTP/2.0

Differences between HTTP/2.0 and SPDY

Transport protocol: HTTP/2.0 supports plaintext HTTP transmission, while SPDY enforces the use of HTTPS. Although HTTP/2.0 supports non-HTTPS, mainstream browsers such as Chrome and Firefox only support the HTTP/2.0 protocol deployed based on TLS. Therefore, upgrading to HTTP/2.0 usually requires upgrading HTTPS first.
Message header compression algorithm: HTTP/2.0 uses the HPACK algorithm to compress message headers, while SPDY uses the DEFLATE algorithm.

New Features of HTTP/2.0 Compared with HTTP/1.x

Multiplexing: HTTP/2.0 allows multiple request-response messages to be initiated simultaneously through a single HTTP/2 connection. In the HTTP/1.1 protocol, the browser client has limitations on the number of requests under the same domain name, and requests exceeding the limit will be blocked, which is one of the reasons why some websites use multiple static resource CDN domain names. The multiplexing of HTTP/2.0 allows messages to be exchanged in parallel on a single connection, reducing the basic unit of HTTP protocol communication to frames, and these frames correspond to messages in logical streams, achieving parallelism of multiple streams without relying on multiple TCP connections. For example, when requesting multiple resources simultaneously:

:method: GET
:scheme: https
:authority: example.com
:path: /image1.jpg

:method: GET
:scheme: https
:authority: example.com
:path: /image2.jpg

Compared with the reuse of persistent connections in HTTP/1.1, in HTTP/1.*, a connection is established for each request-response and closed after use, and each request needs to establish a connection; in the Pipeling mode of HTTP/1.1, several requests are queued for serial single-threaded processing, and subsequent requests need to wait for the previous request to return before they can be executed. Once a request times out, subsequent requests will be blocked; while in HTTP/2.0, multiple requests can be executed in parallel on one connection, and a request with serious latency will not affect the normal execution of other connections.

The multiplexing of HTTP/2.0 makes more effective use of TCP connections and improves HTTP performance. TCP connections will "self-tune", limiting the connection speed in the initial stage and gradually increasing the speed as data is successfully transmitted, that is, TCP slow start. HTTP/2.0 allows all data streams to share the same connection, reducing the inefficiency of HTTP connections, reducing congestion and packet loss recovery time, and increasing connection throughput.

Request priority: Multiplexing may cause key requests to be blocked. HTTP/2.0 allows setting a priority for each request, using a 31-bit priority value, where 0 represents the highest priority and 2(31) - 1 represents the lowest priority. The server can control resource allocation according to the priority and preferentially process and return high-priority request frames to the client.
Binary framing: HTTP/1.1 is based on text parsing, and the diversity of text formats requires considering multiple scenarios during parsing, resulting in poor robustness. HTTP/2.0 adopts binary format parsing and adds a binary framing layer between the application layer (HTTP/2.0) and the transport layer (TCP or UDP). This layer divides all transmitted information into smaller messages and frames (frame) and encodes them in binary format. The header information of HTTP/1.x is encapsulated into the HEADER frame, and the RequestBody is encapsulated into the DATA frame. All communication in HTTP/2.0 is completed on one connection, and this connection can carry any number of two-way data streams.
Header Compression: HTTP/1.1 does not support HTTP header compression, and SPDY and HTTP/2.0 emerged for this purpose. SPDY uses the DEFLATE algorithm, and HTTP/2.0 uses the HPACK algorithm. HTTP/2.0 maintains a dictionary and updates the HTTP headers incrementally to reduce the traffic generated by header transmission. Both communication parties cache a table of header fields respectively to avoid the transmission of duplicate headers and reduce the transmission size.
Server Push: In HTTP/2.0, the server can send multiple responses to a single request from the client. Server Push makes the optimization methods of using embedded resources in the HTTP/1.x era no longer necessary. If a request is initiated from the home page, the server may respond with the home page content, the logo, and the style sheet and other resources required by the client, and these resources can be cached, realizing the sharing of cached resources between different pages under the same origin.