The Magic Behind Typing An URL And Pressing ENTER
We do it dozens if not hundreds of times. We surf through the web and barely know what happens in the background so that when to type www.netflix.com you can watch your favorite movies and shows.
In this article, we will explore the main steps that occur behind the scenes, starting from the point we press Enter to the point the desired website is loaded and appears on our screen.
Before we even begin to scrape the surface of this topic, let’s quickly define four main concepts: servers, web servers, client-server models, and IP addresses.
What is a server?
A server is a special computer without a keyboard, mouse, or screen, that serves information to other computers. These computers, called “clients”, can connect to a server through either a local area network or a wide area network, such as the internet.
All websites are composed of different files like HTML documents, images, CSS stylesheets, and JavaScript files… Those are stored inside a server and delivered to the client by a web server.
What is a web server?
It’s a computer program that distributes web pages as they are requisitioned. Famous web servers today are Nginx and Apache. The basic objective is to store, process, and deliver web pages to the clients. This intercommunication is done using Hypertext Transfer Protocol (HTTP).
The client-server model
Whenever we enter an URL in a browser, we actually ask for specific files that are hosted on a server. The server hears the request, verifies credentials, and if everything checks out, serves the client the requested file. The communication between clients and servers is a two-way street. Finally, the browser interprets the files to make them readable for us.
Let’s take a look at the example above, the user types “www.foobar.com", the browser first, looks for the IP address of the server hosting the website files. when the browser find the IP address (which is 8.8.8.8 in the example), it sends a HTTP request to the server. The web server (nginx) that is installed on that server, listens to any HTTP requests, it accepts the request and responds with the files.
What is an IP address?
An IP address is a unique address that identifies a device on the internet or a local network. IP stands for “Internet Protocol,” which is the set of rules governing the format of data sent via the internet or local network.
There are 2 types of IP addresses: IP Version 4 (IPv4) and IP Version 6 (IPv6).
IPv4 (IP version 4) addresses are sequences of four numbers (from 0–255), separated by a dot (8.8.8.8).
Under IPv4, there are only 232 possible combinations, which offers just under 4.3 billion unique addresses. Due to the increase of the number of computers and devices on the Internet, we are running out of unique IPv4 addresses.
IPv6 came to the rescue by offering a much bigger number of unique IPs. An IPv6 address is a sequence of six segments of letters and/or numbers (0-F) separated by a semicolon.
What is the difference between an IP and an URL
When you type an URL (Uniform Ressource Locator) in the browser, you don’t send the request right away. This is not something that points to the server your looking for. Actually, URLs are made for humans. Since we are better at learning words than numbers, URLs are a sort of intermediate layer between humans and computers. As a result, your browser has to convert your URL into an IP address.
So, in order to get the IP address of a server, the web browser will check first if its cache contains the IP address of the typed domain name and if doesn't find the IP, it will next ask the operating system.
Note that, if the website was visited previously by the user, the browser will find the IP in the cache.
Let’s assume that the user visited the website for the first time and that the browser didn’t locally find the IP address of the website. In this case, the browser will make a query to a remote DNS server…
DNS request
DNS or Domain Name System is, in simple words, the technology that translates human-adapted, text-based domain names to machine-adapted, numerical-based IP.
The DNS request first goes through the resolver. The resolver is usually our Internet Service Provider. Most ISPs have servers dedicated to resolving domain names. If the resolver knows the IP, then, the resolution process ends and it will send it back to the browser.
If the resolver doesn’t know, the request will go to the root server. The root server doesn’t know the IP address of any website, instead, it knows where the TLD (Top-Level Domain) server is. In our examples, “foobar.com” and “netflix.com”, the top-level domain is “.com”.
If the TLD server doesn’t know the IP, it points the resolver to the Authoritative Name Servers of the domain name. These are the servers that will know the IP address of the domain name (if the website actually exists) and can send it back to the resolver then to the web browser.
If the website doesn’t exist, an error will be displayed on the screen.
After getting the IP address, it gets registered locally in the cache to avoid this long trip of DNS resolution process next time.
Now the browser has the IP address of the server so now it can process an HTTP request to it.
What is a network protocol?
A network protocol is an established set of rules that determine how data is transmitted between different devices in the same network. Essentially, it allows connected devices to communicate with each other, regardless of any differences in their internal processes, structure, or design.
Without protocols, devices wouldn’t be able to understand the electronic signals that they send to each other.
What is HTTP?
Stands for “Hypertext Transfer Protocol.” HTTP is the protocol used to transfer data over the web. It is part of the Internet protocol suite and defines commands and services used for transmitting webpage data
For example, when the browser sends a HTTP request, the HTTP verb/method is GET by default, that means, the browser tries to get data from a specified ressource in the server.
There are other HTTP verbs or methods, like POST, PUT, HEAD, DELETE. The method POST, for example, is used to send data to a server to create/update a resource.
What is TCP/IP protocol?
TCP/IP (Transmission Control Protocol/Internet Protocol), also referred to as the Internet Protocol Suite, is the World Wide Web’s core communication system that enables every Internet-connected device to communicate with every other such device simultaneously.
Although TCP and IP are two separate computer network protocols, they’re so often used together. As a result, the “TCP/IP” model is a recognized terminology
IP is the part that obtains the address to which data is sent while TCP is responsible for the way data is delivered, received, ordered, and error-checked over the network.
How TCP works
The Transmission Control Protocol (TCP) is a communications standard that enables application programs and computing devices to exchange messages over a network. It is designed to send packets across the internet and ensure the successful delivery of data and messages over networks.
When a client sends a request to a server, the data is broken into packets. A packet is a small parcel of information that gets transmitted over the network. In the same way, The web server responds by sending back other packets.
Using TCP, all packets that are sent, are tracked so no data is lost or corrupted in transit. This means TCP is reliable.
Difference between TCP and UDP
Just like TCP, UDP (User Datagram Protocol) is another widely used protocol for sending packets over the Internet. TCP is a connection-oriented protocol, whereas UDP is a connectionless protocol. The speed for TCP is slower while the speed of UDP is faster. It is used when you don’t care too much if you get all data always like:
- Tunneling/VPN (lost packets are ok — the tunneled protocol takes care of it)
- Media streaming (lost frames are ok)
- Games that don’t care if you get every update
Application server
Having only a web server in the server means that the delivered page will be only static. There won’t be any interaction happening between users and the website. In order to have a dynamic website, we need an application server.
Application servers are dedicated to take some parameters and return static content to the web server, then the webserver sends the content to the client.
Database and database server
In order to store, extract, manipulate information, we need to have a database and a database server on the server.
A database is an organized collection of data. There are a few types of databases. the most commonly used type is relational databases. A relational database stores data in the form of tables. These tables may or may not, be linked to each other through primary and foreign keys.
Now, what about database servers?
A database server is software that allows us to interact with the database. Using a database server, we will be able to perform many tasks such as data analysis, storage, data manipulation, archiving, and other non-user-specific tasks.
The load balancer
When a website has a large number of visitors, it would be impossible for one single server to handle all these requests. In order to have the website running all the time without any downtime, it would make sense to use many servers instead of one.
But how are client requests going to be sent to servers? which server will they be forwarded to first?
A load balancer is, as its name indicates, a software that will balance or distribute the load of traffic or requests accross the servers following a load-balancing algorithm. HAproxy is a very commonly used load-balancer.
There are several load-balancing algorithms, like: round-robin, weighted and least connections algorithm.
Round-robin alogrithm: Round Robin passes each new connection request to the next server in line, eventually distributing connections evenly across the array of machines being load balanced.
Weighted algorithm: The number of requests that each server/machine receives is proportionate to a ratio weight that we define for each machine. This weight can be defined based on each machine capabilities.
For example, we can say “Machine 3 can handle 2x the load of machine 1 or 2”, and the load balancer will send two requests to machine 3 for each request sent to the two others.
Least Connections algorithm: in this algorithm, Requests are served first to the server which is currently handling least number of persistent connections.
HTTPS/SSL
In this example of url “https://www.holbertonschool.com", we can see that the protocol is https, Not http. So, what is HTTPS?
Hyper Text Transfer Protocol Secure (HTTPS) is the secure version of HTTP. The ‘S’ at the end of HTTPS stands for ‘Secure’. It means all communications between the web browser and the wesbite are encrypted. HTTPS is often used to protect highly confidential online transactions like online banking and online shopping order forms.
With regular HTTP connections, all communications are in ‘plain text’ and can be read by any hacker that manages to break into the connection between the browser and the website. This presents a clear danger if the communication includes sensitive information like credit card details or social security number. With a HTTPS connection, all communications are securely encrypted. This means that even if somebody managed to break into the connection, they would not be able decrypt any of the data which passes between you and the website.
When a website uses HTTPS protocol, we can see a padlock icon in the address bar.
How Does HTTPS Work?
HTTPS websites typically use one of two secure protocols to encrypt communications — SSL (Secure Sockets Layer) or TLS (Transport Layer Security). Both the TLS and SSL protocols use what is known as an asymmetric Public Key Infrastructure system.
An asymmetric system uses two keys to encrypt communications, a public key and a private key. Anything encrypted with the public key can only be decrypted by the private key.
So, when we request a HTTPS connection to a website that uses HTTPS protocol, the website first sends its SSL certificate to the browser. This certificate contains the public key of the server. That means, if we encrypt any data using that public key, only the server that has the corresponding private key can decrypt and read it. After receiving the SSL certificate, the “SSL handshake” occurs and a secure connection between the two machines is established.
Firewall
A firewall is a network security system designed to prevent unauthorized access to or from a private network. Firewalls can be hardware or software.
There is two categories of firewalls: network firewalls or host-based firewalls:
Network firewalls are frequently used to prevent unauthorized Internet users from accessing private networks connected to the Internet, especially intranets. That means, All messages entering or leaving the intranet pass through the firewall, which examines each message and blocks those that do not meet the specified security criterias. Host-based firewalls run on host computers and control network traffic in and out of those machines.
In order to be protected from hackers and attacks, servers and load balancers are often equipped with firewalls.
For example, We can configure a firewall to accept connections coming from only port 22 (ssh port), 80 (http port) and 443 (https port). In that way, if an attacker tries to connect on another port, the firewall doesn’t allow his request. we can also configure the firewall to accept only a certain list or range of IP addresses…