As an IT security professional, one of the areas where I gained a significant amount of knowledge and experience in is network security.  Working for a telecommunications company, it’s pretty much a given that if you are working in security, you are dealing with network security protocols such as IPsec, SSL/TLS and SSH.   I started off primarily working with IPsec and eventually was exposed to TLS and finally, SSH.  One of the real issues that I first encountered was differentiating between these network security protocols. It took me time to be able to understand and explain the following:

  • “What is the difference between these network security protocols?”
  • “Which one shall I use?”

I worked primarily within the Voice over IP (VoIP) product line. Some of the comparisons that I make are based on standards and best practices found within both VoIP and Wireless industries such as IETF3GPPTiSPAN and PacketCable. For more information on any of these standard bodies, please click on their respective links.


Why do we need network security protocols anyways? What services are they providing? To understand this and how these protocols work, you first need to understand how the OSI stack works. The OSI model is an open standard that is used so any vendor can implement a solution that can interop with one another. This is achievable because it follows specified guidelines which are defined in standards known as Request For Comments (RFCs).  These RFCs are published by the IETF.  There are seven different layers in the OSI model:

  1. Application Layer
  2. Presentation Layer
  3. Session Layer
  4. Transport Layer
  5. Network Layer
  6. Data Link Layer
  7. Physical Layer

Some people will start with the Physical Layer and work their way up the stack to the Application Layer but I like to look at it starting from the Application Layer down to the Physical Layer as this is how a packet is constructed.  In this model, each layer is encapsulated within the next layer (starting from the Application) until finally a packet is sent from a host to a destination endpoint. Once the destination endpoint receives the packet, it processes the packet starting at the Physical Layer  and working its way up to the Application Layer.  In order for these protocols to communicate properly, each layer contains a header and a payload.  When we talk of encapsulation it means that when each layer constructs its own header and payload, this payload will also contain the header and payload of the layer above it. Here is an example and to assist in understanding:

Let’s say that we open our web browser and want to connect to This is an HTTP request and we are going to make a connection to an HTTP server that is hosting Since HTTP uses TCP for the transport layer, a full duplex communication channel is established via a 3 way TCP handshake (TCP is described in more detail below). I’m going to ignore the DNS name resolution here (assume we already have the IP address). HTTP operates at the Application Layer. It’ll construct its packet header and payload information (containing an HTTP request). Once it’s completed it’ll call the next layer in the OSI model which is the Presentation Layer. The Presentation Layers purpose is to transform the data into a standard format that can be easily interpreted. It will add its header information and the payload will contain the entire application packet. This goes on and on until the packet can be sent out the physical link and to its destination.  Looking at the illustration below, you can see how each layer encapsulates the previous layer (from left to right).

To correlate to the network security protocols we will be discussing, we will focus on three layers of the OSI model: Application Layer, Transport Layer and Network Layer.

  • Application Layer is where services such as HTTP, FTP, telnet, SMTP, and SSL reside. Each application serves it’s own specific purpose.  SSH is an application protocol
  • Transport Layer is where TCP and UDP reside. It contains the port information. This will tell us which service is being used and if the connection is connection-oriented (TCP) or connectionless (UDP).  See the next paragraph for a more detailed description of TCP and UDP.  TLS is a transport protocol
  • Network Layer  is where IP and ICMP reside.  The Network Layer cares only about the IP address associated with the packet.  Any decision we make regarding who to send the packet to and how, is based off of the Network Layer. If Host X wanted to Communicate with Host Y, it could potentially take many paths (known as hops) before it arrives from Host X to Y.  This layer will tell us who the next device in the path is (say Host A) and send the packet to this device. This would continue on and on until the packet arrives at its destination (Host Y). IPsec is a Network Layer protocol


TCP is a connection oriented transport protocol. Some of the benefits that it provides are:

  • Packet sequencing so packets aren’t received out of sequence
  • Flow and congestion control. It can detect when congestion is occurring on the link and reduce it’s transfer rate
  • Error detection and correction. Retransmit packets if required
  • TCP uses a handshaking scheme to establish a connection. A host sends a TCP SYN packet to the receiver. The receiver then sends a TCP SYN/ACK packet back to the originator. The originator sends a final TCP ACK packet back to the receiver. At this point a full duplex channel is established between the two peers
  • More system overhead with TCP due to the features it provides

UDP is connectionless. What this means is that it doesn’t establish a bi-directional session with a remote peer. It will send a stream of packets to a remote peer and close the connection. It’s a best effort protocol. The two main benefits are:

  • Speed. Doesn’t require a handshake to occur before data can be transmitted
  • Less CPU-intensive. Requires little resources

Why Security?

Now that you have some background information, you can really understand the network security protocols.  IPsec, TLS and SSH share a common goal; provide a secure connection between two peers/devices/endpoints.  Why do we care about providing security services anyways? What is the benefit?

Security can be defined as protecting the confidentiality, integrity and availability (CIA) of assets. It achieves this by providing both integrity protection (ensuring that the message hasn’t been altered) and confidentiality protection (mitigation against eavesdropping by encrypting the connection). All of these protocols provide a standard set of encryption algorithms (both symmetric and asymmetric) and hashing algorithms. Where the differences reside is where they actually perform these secure services (ie. layers)

  • Provide authentication mechanisms in order to mitigate against identity theft or masquerading attacks. This can be provided via user authentication (i.e. certificates) or data-origin authentication. Mitigating against identity theft attacks is important as it can threaten a service providers revenues, damage reputation
  • Provide message integrity (ie. message contents haven’t been altered) in order to mitigate against theft of service. This is provided by message signing (i.e. digital signatures, HMACs).
  • Provide confidentiality protection (via encryption) in order to mitigate against eavesdropping.  To do this, we use a cryptographic algorithm
  • Provide non-repudiation in order to ensure accountability (i.e. the person cannot later deny sending the message)

Asymmetric (public key) vs. Symmetric (secret key) cryptography

Symmetric cryptography is extremely fast as it uses the same secret key to encrypt and decrypt the packet by taking into account two distinct attributes: substitution (replace a character with a different character) and transposition (move the character around). The more substitution and transposition that the cipher goes through, the harder it is to reverse engineer to crack the cipher. If you want to encrypt bulk data or perform continuous encryption/decryption you would use symmetric cryptography. Some examples of symmetric ciphers are AES, 3DES and DES.

Symmetric cryptography has two distinct problems:

  • How do you ensure that the two parties have the same secret key? You need some sort of key management to securely deliver the keys properly
  • Symmetric cryptography also doesn’t scale well.  The equation ( N(N-1)/2 = number of keys ) shows how many keys are required for N parties.

Asymmetric cryptography uses two keys that are mathematically related (public and private). The public key is made public to all users and the private key is kept confidential. Only the owner should have access to the private key.  When we want to encrypt a message between a local and remote host, we use the public key of the remote host to encrypt the message and the remote host will use its own private key to decrypt the message. Asymmetric cryptography scales more efficiently and handles key distribution better. The problem with asymmetric is that it is significantly slower due to mathematically intensive computations. Some asymmetric ciphers include RSA and Diffie-Hellman. Public Key Infrastructure (PKI) is based on asymmetric cryptography.   One of the advantages of asymmetric cryptography is that it can be used for user authentication instead of data-origin authentication (authenticating a user and not a device)

Ideally we want the best of both worlds. This is known as a hybrid approach; we use the asymmetric algorithm ONLY to encrypt and exchange a symmetric secret key. This symmetric key will now be used to encrypt/decrypt all subsequent messages

Now that you have a good understanding on the protocol stack, why we need security and how we provide it, I will discuss the network security protocols that are available.  This will all be covered in Part II.