Introduction to OSI model and TCP/IP for Testers

Most applications out there run on the HTTP protocol, so having a solid understanding of this protocol will make your testing work much more manageable. We explored this in a previous post: What is HTTP protocol – introduction to HTTP for Testers. But there’s more to networks than just HTTP. In this post, we are going to dive deeper into networks by exploring the OSI model.

My main goal in this article is to show you the OSI model and explain how data flows in a network. Then I will go through the differences between the OSI model and TCP/IP. At the end of the article, I will also mention a few protocols used in networks.

But before we get into the details, I should explain some basic terminology.

Terminology

LAN (Local Area Network) and WLAN (Wireless Local Area Network) 

Networking basics LAN WLAN

LAN is a local network that consists of a group of computers and devices connected via a single physical network (cables). It is limited to a specific geographic area/location.

An excellent example of this kind of network would be a library, office, or home. I don’t think most of us use a LAN in our homes these days, because a LAN connects devices via cables.  Nowadays, our devices are connected wirelessly via WIFI, so we’re talking about WLAN.

WAN (Wide Area Network)

WAN combines numerous sites and covers large geographic regions (connecting physically distant locations). The best example of this is the internet itself – that is, thousands of local networks (LAN / WLAN) connected. 

Another example would be connecting three company offices in different cities. Each office has its LAN. By combining them, we could create the company’s own internal network – WAN.

Networking basics WAN

Differences between IP and MAC address

You have probably already heard of and know something about what an IP is. However, you may not have met the concept of a MAC address. So, let me explain in a few words what an IP is, and then a MAC address, to illustrate the key differences between them.

IP (internet protocol) 

We use IP for communication between different networks (to address and transport data from one network to another). It performs the role of routing, i.e., searches for the fastest route to pass a data packet. An IP address is a logical address – this means that it is allocated depending on which network the device has been connected to. If a device is in two networks, it will have two IP addresses.

MAC address (Media Access Control)

MAC is a physical address with a unique identifier burned out on the network card. It identifies specific devices and is assigned by the manufacturer. MAC addresses are used for communication within one network, e.g., in a home network, if you want to connect a computer to a printer or other devices, it will use MAC addresses to do that.

Key differences to remember

IP

MAC

Logical address

Physical address

Identifies connection with a device in the network

Identifies device in the network

Assigned by the network administrator or ISP (internet service provider)

Assigned by the manufacturer

Used in WAN communication

Used in LAN/WLAN communication

OSI model

The OSI model has never been directly implemented as it’s mostly a reference architecture on how data should flow from one application to another through a network. TCP/IP is used, and these days it’s the most popular. After the OSI model, I will say more about TCP/IP. But it’s good to start with the OSI because it’s easier to understand some of the concepts. 

Networking basics OSI model

The OSI model consists of 7 layers divided into two groups:

  • Host layers (happening on the computer side. Responsible for accurate data delivery between devices)
  • Media layers (happening on the network side. Responsible for making sure that the data has arrived at its destination)

7. Application layer

In this layer, the user directly interacts with applications. Here is decided which interfaces are used to interact with the network through the corresponding protocols in this layer. 

Examples of such applications are chrome or Gmail:

  • Chrome uses the HTTP / HTTPS protocol
  • Gmail uses email protocols like SMTP, IMAP.

The applications themselves are not in the application layer – in this layer, there are only the protocols or services that the applications use.

6. Presentation layer

The task of this layer is proper data representation, compression/decompression, encryption/decryption. This ensures that the data sent from the X system application layer can be read by the Y system application layer.

5. Session layer

This layer is responsible for creating, managing, and then closing sessions between two applications that want to communicate with each other. 

4. Transport layer

The task of this layer is to make sure that the data has arrived safely from the sender to the recipient. When it sends data, it breaks it into segments. When it accepts data, it puts it back into a stream of data.

Networking basics Transport Layer

In this layer  two protocols are used: TCP and UDP (later on in the article I’ll be saying more about these)

3. Network layer

Provides addressing and routing services. It defines which routes connect individual computers and decides how much information to send using one connection or another. Data transferred through this layer are called packets.

Places two addresses in the packet sent:

  • Source address
  • Destination address

This layer is based on IP (internet protocol).

2. Data-link layer

This layer deals with packing data into frames and sending them to the physical layer. It also oversees the quality of the information provided by the physical layer. It recognizes errors related to losing packages and damaging frames and deals with their repair. 

1. Physical layer

This is the physical aspect of the network. This applies to cables, network cards, WIFI, etc. It is only used to send logical zeros and ones (bits). It determines how fast the data flows. When this layer receives frames from the data link layer, it changes them to a bitstream.

Encapsulation and decapsulation of data

Encapsulation and decapsulation of data

Encapsulation adds pieces of information to data sent over the network. This occurs when we send data. At each layer, some information is added to our data. We combine the address of the sender and recipient, the encryption method, data format, how the data will be divided, sent, etc.

Decapsulation occurs when we receive information. It consists of removing pieces of information collected from the network. At each layer, some info disappears. In the end, the user only gets what he should receive without the IP, MAC address, etc.

Differences between the OSI model and TCP/IP

The TCP/IP model has a similar organization of layers to the OSI model. However, TCP/IP is not as rigorously divided and better reflects the actual structure of the Internet.

Networking basics Differences between OSI model and TCP:IP

In TCP/IP, there are only four layers:

  • Application layer
  • Transport layer
  • Internet layer
  • Network interface layer

The OSI model makes a clear distinction between layers and some concepts. In TCP/IP, it is harder to make this clear distinction and explain these concepts. Now you can see why I introduced to you the OSI model before the TCP/IP.

The TCP/IP application layer contains three layers from the OSI model:

  • Application layer
  • Presentation layer
  • Session layer 

The working of the application layer in the TCP/IP is a combination of these three layers from the OSI model. In this layer, we have various protocols such as HTTP, DNS, SMTP, FTP. 

The transport and internet layers in TCP/IP work, as I described in the OSI model. But in the next section, I will be revealing more details on how the transport layer protocols (TCP and UDP) work.

The network interface layer in TCP/IP is a combination of two layers form the OSI model (data link and physical layer). I’m not going to go into the details of this layer. But in the OSI model, I described the critical functions of these two last layers. Here in TCP/IP, these functions are realized in one layer.

Protocols in the TCP/IP model

Internet layer protocols

ARP (Address Resolution Protocol)

Used to identify the MAC address. If the device knows the IP address of the target device, then ARP sends a request to all of the devices in the LAN to search for the MAC address of the device with the given IP. Then the device with that IP sends an ARP response with its MAC address. 

This information will be saved in the ARP table. In windows or macOS, open terminal and type arp -a. Then you should see the ARP table.

In the image below, you can see how this process works when an ARP request matches the IP of the device.

Networking basics

The RARP protocol performs the reverse operation.

IP (Internet protocol)

I explained at the beginning of this article what IP is. But I want to make clear that the IP in the TCP/IP model is in the internet layer. It is also good to add that IP has two versions.

  • IPv4
  • IPv6

The second one has been introduced because IPv4 addresses are ending. IPv6 is more efficient, has better routing, and is safer. 

ICMP (Internet Control Message Protocol)

This acts as a tool for solving problems. The ICMP reports any communication errors between hosts. ICMP messages can help to diagnose a problem. For example, if the router or host is overloaded, ICMP can send a message to slow down the transfer rate.

ICMP is used in the ping program, which allows the diagnosis of network connections. Ping lets you check if there is a connection between the hosts. It also allows you to measure the number of packets lost and delays in their transmission.

In the terminal, type ping www.scalac.io. After ping, you need to provide the host. You can choose any website. I’m going to check my connection with the scalac site. To exit ping, use CTRL + C.

Ping sends an ICMP packet to the host provided. In my case, I sent 17 packets and received back 17 packets. In this short connection, I didn’t lose any packets. The program also counts the time gap between sending and receiving packets. In the end, the program summarizes the connection and shows us the minimal/ average / maximum time gap between sending and receiving packets.

Transport layer protocols

TCP (Transmission Control Protocol)

TCP is a highly reliable and connection-oriented protocol. It applies the 3-way handshake principle. Before it sends any data, it will first establish a connection.

Networking basics - Transmission Control Protocol

This rule consists of three steps, made to establish a connection.

  1. SYN – The device sends a message to the server, “I want to connect with you.”
  2. SYN / ACK – When the server receives the message, it will reply that it is ready for communication.
  3. ACK – The device sends confirmation of receiving the response from the server and that it is ready for communication.

The high reliability of TCP is due to the device, making sure that the data sent has been received by the server. Then the server makes sure that the data sent to you have been collected by you. If the server sends 10 data packets, and for some reason, you do not receive one of them, and you do not confirm the receipt, this server will try to send the lost package again. 

TCP also provides data delivery in order. Each sent packet is numbered. Although packets may still arrive out of order, TCP will arrange them in order before sending them to the application.

To summarize the advantages of TCP:

  • Set up a connection before sending any data
  • Data delivery acknowledgment
  • Retransmission of lost data
  • Deliver data in order

UDP (User Datagram Protocol)

UDP sends data and doesn’t care if the device has received it or not. It also doesn’t care if some packets are lost. But the significant advantage of the User Datagram Protocol is that the packet sizes are smaller than TCP (about 60% lighter). 

Networking basics connection

UDP is an economical version of TCP. 

  • Connectionless and unreliable.
  • No data retransmission
  • No data delivery acknowledgment
  • Data can arrive out of order

You may ask the question, then why use UDP? It’s such an unreliable protocol!

In some cases, UDP is better because TCP has significant overheads (data retransmission, delivery acknowledgment, etc.) UDP is often used to transmit data in real-time: video streaming or audio such as Skype calls.

Application layer protocols

Network management protocols

DNS (Domain Name Services) – Changes the domain name to an IP address. The domain name is used because it’s human-friendly. It’s easier to remember a domain name (www.google.com) than an IP address (172.217.16.4). When you type any website address into a browser, then the browser sends a request to the DNS for the IP address of that domain.

Networking basics management protocols

If you type into a browser IP 172.217.16.4, then you should see the Google page because this is Google’s IP address. I can get it directly by requesting the DNS in the terminal. Type in terminal: nslookup www.google.com. 

NTP (Network Time Protocol) – This is an uncomplicated and straightforward protocol. It is used for automatic time synchronization in devices connected to a network. Imagine now manually synchronizing time for 10 or 50 devices. This would be ineffective.

Some devices, procedures, or safety mechanisms require accurate time synchronization for proper operation. Also, thanks to the NTP, finding the causes of any network or device errors is easier. Because using the logs, we will be able to find out what the order of events was that caused the failures.

SNMP (Simple Network Management Protocol) – This is used for monitoring, management of updates, and diagnostics of networks and network devices.

Remote authentication protocols

SSH (Secure Shell) – This allows you to remotely log in to the terminal in network devices and administer them (e.,g. router, firewall, a remote server). SSH is secure because communication is encrypted. SSH uses the TCP protocol.

File transfer protocols

FTP (File Transfer Protocol) –  The purpose of this protocol is to display a list of files/folders, adding, deleting, or downloading them from the server. A good example is sending website files to a server. To do this, you need to use an FTP client with which you can authenticate yourself and get access to the FTP server. A popular FTP client is FileZilla. FTP uses TCP. 

A significant flaw of FTP is the lack of data encryption. Therefore, to ensure secure authentication and transfer of files, it is worth using FTPs (FTP Secure and FTP-SSL) or SFTP (SSH File Transfer Protocol). They work in the same way as FTP but extend its functionality by encrypting the transmitted data.

Email protocols

SMTP (Simple Mail Transfer Protocol) and IMAP (Internet Access Message Protocol) are two protocols used in sending and receiving emails. SMTP’s task is to send email messages from a client to an email server or between email servers. IMAP is used to manage and retrieve email messages from an email server.

SMTP IMAP

This image shows an example when a sender (hubert@gmail.com) and a recipient (jacek@wp.pl) have different email service providers.

  1. In the beginning, the email message is sent to the sender’s email server (Gmail)
  2. Then the Gmail email server sends an email message to the recipient’s email server  (WP)
  3. Finally,  IMAP retrieves the email message from the wp email server to our client.

When the sender and recipient have the same email service provider (Gmail), step 2 will be skipped.

Browser protocols

HTTP/HTTPS – I have written a separate article on HTTP. You can read it here: What is HTTP protocol – introduction to HTTP for Testers. I explain there exactly how HTTP works. HTTPS extends HTTP functionality with data encryption protocols.

VoIP protocols (Voice over IP)

SIP (Session Initiation Protocol) – This performs an administrative function (using TCP). It is used only to set and close an audio or video connection.

RTP (Real-Time Transport Protocol) – This is used to transfer data during audio or video calls (using UDP).

For example, let’s say you want to call someone on Skype. SIP will be used to establish the connection. When the connection is established, the RTP springs into action and transmits the data. When you end the conversation, SIP will close the connection.

Summarize

You have come to know many new concepts today. You now know how data flows in networks. They go through a rather complicated process. All of the topics I have touched are so extensive; they could easily have a separate article for themselves. However, I have tried to present them to you at a fairly general level, easy to understand. Without going too deeply into the more technical aspects.

If you think I have managed to explain things understandably and interestingly, please share this article on social media. And if you have any questions, also feel free to ask them in the comments below.

How to upgrade Angular (JS) to Angular 7?

You must have been asleep (for a few years at least) because Angular 7.x is already here and you’re still stuck on AngularJS. Or maybe it’s just that your codebase is so damn large you can’t face wasting years of your life rewriting it all at once? 

Fear not! It’s not that bad. You can (and should!) upgrade to Angular 7 step by step, or you can do it while leaving your old code in place, working with the latest Angular shipping new features.

But how? Simple. By choosing the Hybrid Application path. This means we will be able to expand our old app with the new Angular, running both frameworks at the same time.

In my opinion, the best way to understand any tool is to in fact dig into the code itself.  So I have prepared a small (really the bare minimum!) repository to use as an Angular upgrade example to show off the process of upgrading.

Prerequisites

Well, actually there aren’t that many prerequisites that your project must follow to be able to upgrade.  All the in-depth details are, as always, in Angular Docs.

Also, these are the things you can do if you like but you are not really obliged to do; some are just for helping you get through the upgrading process (like sticking to the style guide).

To name a few:

  1. Following the AngularJS Style Guide 
  2. Using a Module Bundler: but who doesn’t these days?
  3. TypeScript: this is a must, but you probably already know that
  4. Component Directives: just a good step forward you should already be using if you’re on AngularJS >= 1.5

UpgradeModule to the rescue

Have I mentioned the hybrid app already? Because this seems like a good moment to put in a few more words on it.

The Angular Hybrid app is just like any other, but it comes packed with two versions of Angular; the good ol AngularJS and one of the latest/newest Angular versions (you’re probably reading this with somewhere around Angular 7 / 8 out there, or maybe even 9? 10…?).

Angular comes with a handy UpgradeModule which will help us bootstrap the hybrid application.

The next part of this post will cover the tools inside this module to help you bootstrap, upgrade/downgrade the components.

Bootstrapping

While we usually/sometimes bootstrap AngularJS app automagically from HTML, like this:

<!doctype html>
<html ng-app="app">
<head>
  <meta charset="UTF-8">
  <title>ng Hybrid App</title>
</head>
<body>
  <app></app>
</body>
</html>

working with the hybrid application requires bootstrapping manually two modules, one for each ng version (more on this below). So, it’s necessary to replace the auto bootstrap with manual in your ng1.x app:

import * as angular from 'angular';

import { AppComponent } from './components/app/app.component';
import { WhatAmIComponent } from './components/whatAmI/whatAmI.component';

angular
  .module("app",[])
  .component(AppComponent.selector, new AppComponent())
  .component(WhatAmIComponent.selector, new WhatAmIComponent());

angular
  .bootstrap(document, ['app']);

Who doesn’t like more npm dependencies?

First of all, let’s introduce some Angular dependencies into our project, namely:

  • @angular/core: 
  • @angular/common: 
  • @angular/compiler: 
  • @angular/platform-browser: 
  • @angular/platform-browser-dynamic: 
  • @angular/upgrade: 
  • rxjs

These aren’t all Angular comes with, but they’re enough to do an upgrade. You’ll probably install some more while working with the code e.g. @angular/router, @angular/forms and so on.

Besides Angular itself, we’ll need some polyfills:

  • core-js
  • zone.js

You’ll find the current state of the package.json here: https://github.com/kamil-maslowski/ng-hybrid-app/blob/upgrade-packages/package.json.

{
  "name": "hybrid-app",
  "version": "0.0.1",
  "description": "",
  "scripts": {
    "build": "webpack --config webpack.config.js"
  },
  "author": "Kamil Maslowski",
  "license": "ISC",
  "dependencies": {
    "@angular/common": "^7.2.12",
    "@angular/compiler": "^7.2.12",
    "@angular/core": "^7.2.12",
    "@angular/platform-browser": "^7.2.12",
    "@angular/platform-browser-dynamic": "^7.2.12",
    "@angular/upgrade": "^7.2.12",
    "angular": "^1.7.8",
    "core-js": "^3.0.0",
    "rxjs": "^6.4.0",
    "zone.js": "^0.8.29"
  },
  "devDependencies": {
    "@types/angular": "^1.6.54",
    "html-webpack-plugin": "^3.2.0",
    "ts-loader": "^5.3.3",
    "typescript": "^3.4.1",
    "webpack": "^4.29.6",
    "webpack-cli": "^3.3.0"
  }
}

The order of things to upgrade Angular

AJS module

Now we need to make some changes in the project structure. Our old place-where-it-all-started was here, let’s rename it as ajs.module.ts. 

We also need to remove the line:

angular
  .bootstrap(document, ['app']);

Why do we do this? As I’ve already mentioned, we need two modules, AJS will act as our AngularJS module, and the bootstrapping will be handled by the Angular UpgradeModule tools.

App module

Our app.module:

import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';
import { UpgradeModule } from '@angular/upgrade/static';

@NgModule({
  imports: [
    BrowserModule,
  ]
})
export class AppModule {
  constructor(private upgrade: UpgradeModule) { }
  ngDoBootstrap() {
    this.upgrade.bootstrap(document.documentElement, ['app']);
  }
}

namely the place where we will bootstrap the Angular app (now along with AngularJS) is really simple. We just need to import the bare minimum and export a module, overriding the ngDoBootstrap.

Index.ts reinvented

So the main entry point of our app, index.ts, now looks like this

import 'core-js/proposals/reflect-metadata';
import 'zone.js';

import { platformBrowserDynamic } from '@angular/platform-browser-dynamic';

import './ajs.module';
import { AppModule } from './app.module';

platformBrowserDynamic().bootstrapModule(AppModule);

First, we’ll import some polyfills Angular will need to support all the latest mojos not yet in some of the browsers.  For overall info on that check out the Angular Browser Docs

Then platform-browser-dynamic, already covered above, our AngularJS and Angular modules, and well – that’s all we need here.

The last step now is just to bootstrap the app!

platformBrowserDynamic().bootstrapModule(AppModule);

Hybrid app ready to run!

Now we already have a working, hybrid angular application. But no need to take my word for it.  Just run:

{ "build": "webpack --config webpack.config.js" }

and check out yourself the dist folder. So why almost? Because I’d like to also show you how to take your first baby-steps in your fancy new hybrid app. We’re going to write an Angular component, and downgrade it to make it usable inside the AngularJS code. Let’s do it!

First steps in your hybrid app

Let’s create a new component under the src/components. We’ll call it whatIdLikeToBe.component.ts. We’ll create this component in Angular style, exporting the component class annotated with Component from the @angular/core package. As it’s a template (remember we can also use templateUrl instead of the inline template!) we just have to put in a simple string.

You’ll find the source below

import { Component } from '@angular/core';

@Component({
  selector: 'what-id-like-to-be',
  template: `
    <h2>yay! I'm an ng6 component!</h2>
  `
})
export default class WhatIdLikeToBeComponent { }

What next? Next, we want to be able to use our brand new Angular component in our Angular Hybrid App. To be able to do that we have to tell Angular(JS) that we have some component that it needs to downgrade to 1.x format, so…

import * as angular from 'angular';

import { AppComponent } from './components/app/app.component';
import { WhatAmIComponent } from './components/whatAmI/whatAmI.component';
import { downgradeComponent } from '@angular/upgrade/static';
import WhatIdLikeToBeComponent from './components/whatIdLikeToBe/whatIdLikeToBe.component';

angular
  .module("app",[])
  .component(AppComponent.selector, new AppComponent())
  .component(WhatAmIComponent.selector, new WhatAmIComponent())
  .directive(
    'whatIdLikeToBe',
    downgradeComponent({ component: WhatIdLikeToBeComponent })
  );

See what we did there? We downgraded our component to the AngularJS directive specifying a directive name (spot the camelCase here, it’s easy to mix up where to use camelCase vs kebab-case). And that’s all we need! Our Angular component can now be used anywhere in the AngularJS codebase, like this:

export class AppComponent implements ng.IComponentController, ng.IComponentOptions {
  static selector = 'app';

  controller: ng.Injectable = AppComponent;
  template: string = `
    <h1>Upgrade me!</h1>
    <div>
      <what-am-i></what-am-i>
      <what-id-like-to-be></what-id-like-to-be>
    </div>
  `;
}

Once more, spot how we specified the camelCase directive name vs how we use it with kebab-case here!

Conclusions on Angular upgrade

Reading the intro you were probably thinking:  “Wait a moment! How to upgrade to Angular 7? But we’re already on v. 8/9/2802 ”. And yes, you’d be completely right, which shows how often Angular is updated (in fact there’s a major new version every six months). So there’s no time to lose to start upgrading!

You may, of course, be happy with the current source code that backs your projects, but as time passes, and literally everyday tech stacks develop more and more, it’s worth checking out what’s out there. Not only for the sake of developers, but also for the sake of your end-users, whose user experience will be improved for numerous reasons, not least because of the performance boosts you can achieve and all the mojos made available to you by the Angular team.

Useful links