Encrypted Traffic Overview - How Public Key Infrastructure (PKI) works
RSA is a lynchpin of Public Key Cryptography. The more I learn about Public Key Cryptography the more I am amazed by the intelligence and thought processes put into it to ensure data can be encrypted. Public Key Cryptography is asymmetric, so it is mainly used as a way for systems to securely communicate symmetric encryption keys between systems. The symmetric keys are then used to encrypt all the data transferred between the systems. The data transferred between systems is not encrypted with Public Key Cryptography because it would be too slow to transfer data. I put together graphics that outline a quick recap of Public Key Cryptography. I may be missing some steps, and there are also some variations but here is a very simple explanation.
Two systems want to communicate securely. The client contacts the server. It could be an online ecommerce company.
The server sends back its public key along with other data related to encrypting the session.
When the client receives the servers public certificate it can verify if it can trust the server through a system of 3rd party trust. Most ecommerce companies get their certificates signed by Certificate Authorities (CA)s who are trusted by most browsers. On a side note, there can be compromises here that can break the system of trust. There is a famous case where the Certificate Authority Diginotar certificate authority was hacked https://slate.com/technology/2016/12/how-the-2011-hack-of-diginotar-changed-the-internets-infrastructure.html
This enabled hackers (Cyber security researchers point to Iran) to falsely sign certificates as Diginotar and compromise systems.
After the certificate is checked and some data about the session is exchanged; the session will prepare to securely exchange symmetric encryption keys.
A pre-master secret is sent between systems. It is data encrypted with the server’s public key.
The server decrypts the pre-master secret with its private key.
Now both sides have the same sets of data. They then calculate the symmetric keys from the same sets of data.
Data is then encrypted using the symmetric encryption key.
What is the role of Hashing in encrypted communications?
The heart of what hashing does for users is integrity. Hashing is especially critical in software distribution over the Internet like Git Hub. The software vendor/producer generates a hash of the file. The user who downloads the file can run the same hash and compare the values. This ensures the integrity of the downloaded file from tampering or corruption.
Integrity does not include authenticity. Because anyone could take data or a file, hash it and send it to its destination. With integrity all you know is the data/file wasn’t manipulated. You can’t verify the authenticity of who sent it. This is where Message Authentication Codes (MAC)s come into play. Specifically, there is Hash-Based Message Authentication Codes (HMAC)s. Within systems you will see HMACs listed like, HMAC-SHA1, or HMAC-SHA2-512. The HMAC provides authenticity by taking a symmetric key, adding it to the data to calculate a total hash. On the other receiving end, the system has the same data and key. The receiver calculates the total hash using the same data and key, and ensures it matches from the sender. In this way the authenticity of the sender and integrity of the data is verified because only the sender should have access to the key.
The next level beyond authenticity is can you have a mechanism where if someone sends data, they cannot deny sending it. This is called non-repudiation. Non-repudiation s accomplished through the usage of digital signatures created from hashes. The sender calculates a hash of some data and then encrypts the hash with their private key and sends it as a digital signature. The receiver decrypts the hash with the sender’s public key, and then compares it to a hash taken of the data.
Because integrity, authenticity, and non-repudiation all rely on hashing they are all in susceptible if hashes are compromised. The main attacks against hashes are called collision attacks. A program called SHAttered broke SHA-1. “Cryptographers predicted SHA-1 would be broken in a real-world scenario, but the SHAttered research came three years earlier than they expected, and also cost only $110,000 to execute using cloud-rented computing power, far less than what people thought it might cost.” (Cimpanu, 2019). The SHAttered web site is here https://shattered.it/ . The web site also has a link to their research paper where the successfully collided SHA-1 hashed pdf documents.
The next level of SHA-1 attacks beyond collision attacks are called chosen prefix collision attacks. This was researched in the paper, From Collisions to Chosen-Prefix Collisions Application to Full SHA-1. A good explanation of the impact of chosen-prefix collision attacks, "A much more interesting attack is to find a so-called 'chosen-prefix collision,' where the attacker can freely choose the prefix for the two colliding messages. Such collisions change everything in terms of threat because you can now consider having collisions with meaningful data inside (like names or identities in a digital certificate, etc)." (Cimpanu, 2019)
Many vendors like Chrome, and Firefox have deprecated SHA-1 and no longer support it. It still shows up in some places but is slowly being phased out for stringer hashing algorithms.
References
Cimpanu, C. (2019, May 12). SHA-1 collision attacks are now actually practical and a looming danger. ZDNet. Retrieved September 21, 2021 from https://www.zdnet.com/article/sha-1-collision-attacks-are-now-actually-practical-and-a-looming-danger/
Leurent, G., Peyrin, T. From Collisions to Chosen-Prefix Collisions Application to Full SHA-1. Nanyang Technological University, Singapore. Retrieved September 21, 2021 from https://eprint.iacr.org/2019/459.pdf
Stevens, M., Bursztein, E., Karpman, P., Albertini, A., Markov, Y. The first collision for full SHA-1. Google research. Retrieved September 21, 2021 from https://shattered.it/static/shattered.pdf
The Case for Examining Encrypted Traffic
Analyzing Encrypted traffic through meta data
I recently watched a SANS webcast by network encryption security expert Richard Bejtlich. He used an interesting metaphor when discussing investigating network encryption. He compared encryption to a wrapping paper around a gift. Many observations about the gift can be made before actually opening the present.
Does the gift exist?
How much does the gift weigh?
How big is it?
Are there any other like it?
Did I get a gift like this last year?
Where was it?
When did I get it?
How was it delivered?
All these observations can help someone determine the nature of the gift. His analogy is that in encryption there are also observations that an investigator can use to determine the nature of the encrypted communications. Returning back to the start of an encrypted session much of the initial communications between client and server is unencrypted.
If an analyst has access to this network data, they can make some observations about the encrypted present. For example, with TLS 1.2 encryption standard (which is still very common) investigators can determine where the encrypted traffic is going.
A popular network log data used for this type of high level encryption analysis is Zeek log data https://docs.zeek.org/en/master/logs/ssl.html
But in newer TLS 1.3 ESNI “Encrypted Server Name Indication” the remote server name is also encrypted. This feature actually prompted China to recently block access to all TLS 1.3 sites because they could not censor that traffic with their Greta Firewall. “While TLS hides the content of a user's communication, it cannot always hide the server they are communicating with because its handshake optionally contains a Server Name Indication (SNI) field designed to explain where traffic is going. China and other nations have used this info to block their users from accessing particular websites.” (Harb. China now blocking ESNI-enabled TLS 1.3 connections, say Great-Firewall-watchers)
References
Bejtlich. R. What can you see when you're blindfolded? Network Security Monitoring vs. Encryption. Corelight. (Sept, 2020). Retrieved September 25, 2020 from https://www.sans.org/webcasts/network-security-monitoring-encryption-116695 (Links to an external site.)
Harb. R. China now blocking ESNI-enabled TLS 1.3 connections, say Great-Firewall-watchers. The Register. (Aug, 2020). Retrieved September 26, 2020 from https://www.theregister.com/2020/08/11/china_blocking_tls_1_3_esni/
Fully Analyzing Encrypted Traffic
Fully analyzing encrypted traffic is accomplished through a Man-in-the-Middle (MitM) function. The diagram below explain this. The reference diagram of a PKI Infrastructure below, helps illustrate a few points. Issuing CAs issue certificates to subjects. In web browsing, Relying Parties are essentially users operating browsers like Chrome and Firefox. They are relying on the integrity of the entire to trust who the subject says they are.
Diagram of PKI Infrastructure (Turner, 2012)
A big reason for impersonation by governments like China is to do something called Man-in-the-Middle (MitM). The solution to inspecting encrypted traffic is something called Man-in-the-Middle (MitM). The diagram below from a NIST Venafi presentation is an excellent example of MitM. The MitM machine decrypts all communications from one side and re-encrypts it onto the other side. The two end systems believe they have an encrypted session, but the system in the middle can see everything.
Man-in-the-Middle (Turner, 2012)
The main damage to this malicious certificate scenario is impersonation or MitM here is a discussion on how MitM is done. Believe it or not there is legitimate MitM (although most would argue China’s Great Firewall MitM is not legitimate). Your organization is probably trying to accomplish this same MitM for corporate traffic.
“Many organizations, in order to provide gateway antivirus protection and protect against the theft of intellectual property, perform deep packet inspection. If you see that certificates your browser trusts on a company owned device are signed not by the site you’re visiting, but by your company’s firewall instead, this means that the company is using this kind of technology.” (Baka, 2019)
Organizations have a different reason for traffic interception than China. First, they are only doing the inspection on organization traffic, and they should have some controls in place around it. For example, organizations don’t inspect encrypted traffic to healthcare or banking sites. Personal email, like gmail, Hotmail, etc is a gray area for organizations. Personal email is encrypted, and also not healthcare or banking, but it can contain a lot of personal information. Because organizations don’t want to inspect personal email, many organizations opted to block access to it all together. If you worked for an organization that once allowed personal email access and stopped a few years back, I am willing to bet this is the reason why. If your organization still allows personal email they are probably not doing encrypted traffic inspection at all, or are treating personal email like banking and healthcare sites and not inspecting it. For organizations to inspect traffic and perform the MitM there are two types:
Outbound encrypted traffic inspection – Called SSL “Forward Proxy”
Inbound encrypted traffic inspection – Called SSL “Inbound Inspection”
Forward-Proxy (PaloAlto, 2021)
Inbound Inspection (PaloAlto, 2021)
In the Great Firewall or corporate MitM scenario is illustrated below. The firewall or inspection devices acts as a MitM where the relying party trusts it.
Diagram of PKI Infrastructure with Man-in-the-Middle (Turner, 2012)
Now onto some fraudulent ways MitM can occur.
A. Impersonator to obtain a fraudulent certificate
B. Compromising a registry Authority
C. Fraudulently signing certificates
D. Stealing CA keys
CA Compromise and Fraudulent Certificate Scenarios (Turner, 2012)
References
Baka, P. (2019, May 13). Protecting against Man-In-The-Middle Attacks. The SSL store. Retrieved September 28, 2021 from https://www.thesslstore.com/blog/protecting-against-man-in-the-middle-attacks/
Google. (2021). HTTPS encryption on the web. Google Transparency report. Retrieved September 28, 2021 from HTTPS encryption on the web – Google Transparency Report
Nohe, P. (2018, August 8). Let’s Talk About HTTPS, Google and China. The SSL store. Retrieved September 28, 2021 from https://www.thesslstore.com/blog/https-google-china/
PaloAlto. (2021, January 30). Difference Between SSL Forward-Proxy and Inbound Inspection Decryption Mode. PaloAlto Customer support Portal. Retrieved September 28, 2021 from https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClV8CAK
Turner, P. (2012). NIST ITL July 2012 CA Compromise. Venafi. Retrieved September 28, 2021 from https://csrc.nist.gov/csrc/media/projects/forum/documents/2012/october-2012_fcsm_pturner.pdf
Engineering SSL Interception
When organizations perform MitM on encrypted traffic it is called SSL Interception. When evaluating SSL interception vendor solutions, it is important to understand that SSL interception is a network function rather than a solution into itself. SSL Interception developed as an ancillary function two several types of network products. The main products that developed SSL interception as an ancillary function were:
Application Delivery Controllers
Web Proxies
Next Generation Firewalls (NGFW)s
Traffic Brokers
It is important to understand the base network functions because they can helped dictate what the best SSL Interception solution is for your organization.
Application Delivery Controllers
F5
A10
For large scale web services to have redundancy and support high traffic loads they are usually fronted by a device called a load balancer.
Web services started to require encryption (think online banking, etc), and these web servers had to support the encryption of large amounts of user sessions. which was extremely resource intensive for the servers. Specialized SSL hardware was used and/or integrated with load balancers to provide this function SSL processing function. This is the "inbound inspection" referenced in the earlier section that discussed the types of MitM.
If you are an organization and want to inspect encrypted traffic leaving your organization for Data Loss Prevention (DLP) and encrypted malware traffic these SSL inspection devices can decrypt the traffic. But think back to their original use case, high volume web traffic. These SSL termination/inspection devices often run on specialized hardware and are very expensive. They are absolutely necessary if your terminating large volumes of SSL traffic but are overkill for a regular business that wants to inspect outbound SSL traffic.
Web Proxies
Blue Coat
ZScaler
The history of the next set of SSL inspection devices comes from a web proxy lineage. If an organization had a perimeter network security device they either had a web proxy, a firewall or both (or nothing at all).
The firewall and the proxy served two separate purposes. The firewall protected the internal corporate resources from the Interne, and the proxy acted as a MitM for internal user web traffic. This protected end user computers from interacting with the remote web servers. Then web proxy vendors started offering web categorization features and organizations used web proxies to regulate and monitor Internet web surfing in their networks.
Encrypted web traffic began to grow and web proxy vendors needed to get out ahead of it. The majority of encrypted traffic was encrypted web traffic, and the proxy was already receiving this traffic through "explicit" configuration in the client browsers or transparent in the organizations web traffic flow. So these proxy vendors started to offer SSL inspection functions, most commonly through a dedicated hardware. The proxy would the traffic from common encrypted ports (i.e., 443, 8443, etc) to the specialized SSL interception device/service.
A reason why an organization may use an SSL inspection function like this is because the are already using a web proxy which their encrypted web traffic is traversing.
Next Generation Firewalls (NGFW)s
Palo Alto
Cisco
Juniper
Fortinet
Firewalls sat in this position where all the network traffic flow traversed them, yet organizations still had separate web proxies to proxy web traffic. The firewall market got very competitive as different people left firewall companies to start new/better firewall companies. As part of this competition more capabilities were developed in firewalls. First, firewalls started to do the web proxy function. Many did it poorly or could not support large amounts of users so their remained a strong market for separate web proxies. But over the years the firewall vendors got better at web proxying and now many firewalls work fine as both a firewall and web proxy. Along with the web proxy capability comes the encrypted web traffic heading the firewalls way; so the firewalls also developed SSL interception capabilities.
If you have an existing firewall that supports SSL interception that is a good place to start developing your SSL interception solution. You don't have to manage a separate vendor product, and integrate it in your environmet.
Traffic Brokers
Gigamon
References
Certificate Transparency
Comments