Decoding the Encryption Mystery

Decoding the Encryption Mystery

Blog Post 5: From Handshake to Payload

Welcome again to 100daysofapisecurity. Today, we will explore the world of encryption, whereby you will be taken through several encryption nuances and potential vulnerabilities.

End-to-End Encryption (E2EE)

End-to-End Encryption, or E2EE, is the digital equivalent of whispering a message in a secret language into a friend's ear in the middle of a noisy environment. Only the sender and receiver can decrypt and understand the message. For APIs, this means that even if data transmission is intercepted, all that's captured is unclear chatter, ensuring confidentiality in sensitive data transfers such as health records or financial transactions.


The SSL (Secure Sockets Layer) era was the first attempt at encrypting data traffic between two systems. But as vulnerabilities crept in, the world transitioned to the more secure TLS (Transport Layer Security). Think of SSL as a wax-sealed letter and TLS as a more sophisticated locking mechanism. When you see a URL beginning with https://, you're experiencing this security layer.


While the communication channel's encryption is pivotal, often the data itself demands an added layer of security. This process sees data being encrypted using complex algorithms, like the Advanced Encryption Standard (AES), before transmission, and decrypted upon receipt.

Deep Dive into TLS

TLS operates through multiple protocols:

  1. Handshake Protocol: It establishes connections, negotiates encryption algorithms, and exchanges keys.

  2. Record Protocol: Upon a successful handshake, this protocol ensures secure data transmission.

  3. Alert Protocol: It alerts parties about potential issues during the encrypted session.

Role of an API Gateway

TLS is a crucial protocol that as explained ensures data confidentiality and integrity as it's transmitted over networks. When an API message, often referred to as a payload, is sent securely using TLS, it undergoes a series of steps before it can be processed by devices, such as an API Gateway. Here's a breakdown of this process.

Step One: The Handshake

When a client (it could be a browser, a mobile app, or any other client application) wants to send a secured API request, it initiates a TLS handshake with the server where the API resides, or in this case, with the API Gateway.

  • The handshake begins with the client sending a ClientHello message indicating supported cryptographic algorithms, a randomly generated value, and other settings.

  • The server responds with a ServerHello, selecting the mutual cryptographic algorithms and sending its random value.

  • The server then sends its digital certificate to the client. This certificate is validated by the client, often using a trusted Certificate Authority (CA). More about this step further below.

  • Using information from the certificate and the random values, both the client and the server generate a shared secret, called the "pre-master secret". Note that this secret isn't transmitted. Instead, techniques like Diffie-Hellman key exchange or RSA is used to ensure both parties can compute this shared secret independently.

  • Using this pre-master secret, both parties generate encryption keys for the session.

Step Two: Encrypting the API Payload

Once the handshake is completed, the client can start sending the API payload. This payload is encrypted using the session keys derived from the shared pre-master secret.

  • The payload is segmented into manageable chunks.

  • Each chunk is encrypted using the session key and the agreed-upon encryption algorithm.

  • The encrypted data is packaged with a message authentication code (MAC) to ensure data integrity.

Step Three: Reception by the API Gateway

The decrypted payload can now be processed. This might involve rate limiting, payload inspection, authentication, and routing the request to the appropriate backend service.

Step Four: Responding to the Client

When the backend service processes the request and sends a response, the API Gateway steps in again:

  • The response payload is encrypted using the same session keys.

  • The encrypted response is sent back to the client over the established TLS connection.

  • The client decrypts the response using its session key, verifies the data's integrity, and then processes the decrypted payload.

TLS Certificates and CAs

A TLS Certificate 'package' acts as the server's public-facing identity, underpinned by cryptographic guarantees. At its core, a TLS certificate contains:

  • The domain name for which the certificate was issued.

  • The entity that it belongs to (e.g. a company name).

  • The certificate's public key.

  • Its validity period.

  • The Certificate Authority (CA) that issued it.

  • And a digital signature from the CA.

Compare it with an ambassador's seal which signifies trust, authentication, and the imprimatur of their homeland. TLS certificates take on this ambassadorial role, becoming the foundation of trust in our digital interactions. Much like a diplomat's credentials guarantee their identity, a TLS certificate verifies the identity of a server. When you connect to a website or an API endpoint, the presented certificate tells you, "You can trust me; I am who I say I am." This assurance is essential in scenarios like online banking, where trust is key.

Just as countries authorize their diplomats, trusted entities called Certificate Authorities (CAs) issue TLS certificates. CAs are organizations that are trusted to verify and vouch for the identity of servers. When a server presents its certificate, the client checks if it's signed by a CA they trust. If so, a layer of trust is established.

So what is the role of a TLS certificate in light of TLS encryption? Beyond identity validation, TLS certificates play a crucial role in data encryption. The public key within the certificate is used to establish encrypted sessions. When a client wants to start a secure session, it uses this public key to encrypt a shared secret, which only the server can decrypt using its private key. This shared secret becomes the basis for encrypted communication.

API Vulnerabilities & Exploitation Tactics

APIs, if not protected properly, can fall victim to multiple types of encryption-related attacks:

  • Man-in-the-Middle (MitM) Attacks: This vulnerability can allow a hacker to capture and potentially modify data in transit, similar to a bad actor intercepting a confidential letter. Think about a replay attack whereby attackers can capture encrypted requests and resend them, initiating unauthorized actions without proper mechanisms like unique timestamps or tokens.

  • Downgrade Attacks: Attackers can force victims to use outdated encryption protocols, making breaches easier.

In their continued search of breaches, attackers might:

  1. Crack Encryption: With tools like John the Ripper or Hashcat, they attempt to break weak encryption or poorly set passwords.

  2. Session Hijacking: They can exploit vulnerabilities to take over a session, granting unauthorized access to protected data.

  3. Leverage Protocol Vulnerabilities: Older encryption methods might have known issues. For instance, the BEAST attack exploited flaws in SSL/TLS.

A very common situation is that an application server has an expired SSL certificate. Defining the expiration of SSL certificates serves several purposes such as:

  • Security: The longer a key exists, the more susceptible it becomes to potential breaches or cryptographic vulnerabilities.

  • Identity Verification: The digital world is dynamic. Domains change owners, companies undergo restructuring, and services evolve. Regularly renewing certificates ensures that the entity you're communicating with is still the one it claims to be.

  • Key Rotation: Renewing a certificate generally means generating a new key pair, which refreshes the encryption and makes any potential compromise of the old key irrelevant.

But, there are also consequences of an Expired SSL Certificate:

  • Trust Erosion: Clients or browsers connecting to a server with an expired certificate will receive a warning or error message. This can annoy users, as it signals a potential security risk.

  • Service Disruption: Many applications and API clients might refuse connections to servers with expired certificates, leading to service downtimes or failures.

  • Security Vulnerabilities: While the encryption provided by the expired certificate might technically remain intact, the trustworthiness is compromised, making man-in-the-middle attacks more feasible.

  • Reputation Damage: Trust is hard to gain but easy to lose. Operating with an expired certificate can damage a company's reputation, making users think twice before using the service again.