Certificates, SSL / TLS, & al. - Because the web requires identity, privacy and integrity

mail

Glossary of certificates

Certificate
Electronic document used to prove the ownership of a public key by the named subject of the certificate. This testifies of trust relationship between public and private keys.
Synonyms : (public key|digital|identity) certificate
Certificate Chains
A Certificate Authority may issue a certificate for another Certificate Authority. When examining a certificate, Alice may need to examine the certificate of the issuer, for each parent Certificate Authority, until reaching one which she has confidence in. She may decide to trust only certificates with a limited chain of issuers, to reduce her risk of a "bad" certificate in the chain.
Certificate Management
Managing certificates implies :
  • creating new certificates
  • revoking obsolete certificates
  • updating a list of obsolete certificates : the CRL.
It is impossible to tell from the certificate alone whether it has been revoked or not. Therefore when examining certificates for validity it is necessary to contact the issuing CA to check the CRL.
Certificate Signing Request (CSR) aka certificate request
Message sent from an applicant to a Certification Authority when applying for a certificate. It contains :
  • the public key for which the certificate should be issued
  • ID information (domain name)
  • integrity protection (a digital signature)
The most common format of these messages is PKCS#10. (see also).
Certification Authority (CA) aka certificate authority
  • Trusted 3rd-party entity that issues certificates
  • By verifying the information in a certificate request before granting the certificate, the CA assures itself of the identity of the private key owner of a key pair. For instance, if Alice requests a personal certificate, the CA must first make sure that Alice really is the person the certificate request claims she is.
cipher suite (en.wikipedia.org, httpd.apache.org)
Common Name (CN)
The common name is typically composed of host + domain Name (i.e. www.example.com or mysite.com). SSL server certificates are specific to the common name that they have been issued to, at the host level. Which means the common name must be the same as the web address you will be accessing when connecting to a secure site. For example, a SSL server certificate for the domain domain.com will receive a warning if accessing a site named www.domain.com or secure.domain.com, as www.domain.com and secure.domain.com are different from domain.com. (see also)
Distinguished Name (DN)
read this. see also.
PKCS#10
Format of messages sent to a CA to request certification of a public key. This is the most common format used to write CSR.
Root-level CA
Each certificate requires an issuer to assert the validity of the identity of the certificate subject, up to the top-level CA. This presents a problem: who can vouch for the certificate of the top-level authority, which has no issuer ? In this unique case, the certificate is self-signed, so the issuer of the certificate is the same as the subject. Browsers are preconfigured to trust well-known CA, but it is important to exercise extra care in trusting a self-signed certificate. The wide publication of a public key by the root authority reduces the risk in trusting this key —it would be obvious if someone else publicized a key claiming to be the authority.
Well-known CA :
Server Name Indication (SNI)
See my dedicated article about SNI
Subject Alternative Name (SAN)
Such certificates can hold extra key/value pairs for extended functionality. Such functionality includes allowing the certificate to match a list of domains :
  • example.com
  • www.example.com
  • www.example2.com
  • dev.example2.com
  • www.example3.net
  • mail.example.net
Such certificates must be re-issued to add / remove supported names.
TLS
  • the successor of the now-deprecated SSL
  • a cryptographic protocol designed to provide communications security over a computer network
Different versions :
  • TLSv1.0 :
    • an upgrade of SSLv3.0, but not backward-compatible
    • "security standards" recommended upgrading to TLSv1.1 or higher before June 2018
    • deprecated by industry majors (as well as TLSv1.1) since march 2020
  • TLSv1.1 : deprecated by industry majors since march 2020
  • TLSv1.2
  • TLSv1.3
Wildcard Certificate
Certificate that can be used on all level-1 subdomains of a domain.
According to the RFC 2818, the * matches 1 or more non-dot characters (but some implementations allow a dot. source)
*.example.com :
  • matches :
    • www.example.com
    • ftp.example.com
    • test.example.com
  • does not match :
    • example.com : * must match at least 1 character
    • dev.test.example.com : this is 2 levels deep
Pro's of wildcard certificates vs conventional certificates :
  • 1 wildcard certificate is cheaper than n conventional certificates
  • more convenient when it comes to server configuration
  • only 1 validity period for all : renewing the certificate renews all
  • applies instantly to new level-1 subdomains, even though they were unknown when ordering the certificate

What's the difference between wildcard and SAN certificates ?

wildcard certificates SAN certificates
apply to... all level-1 subdomains of a single domain all listed domains
apply to the naked domain example.com ? No Yes, if explicitly listed in the supported names
action to add / remove a supported domain nothing (provided this is a level-1 subdomain of the registered domain) re-issue the certificate

Fields of X.509 certificates :

issuer
holds the DN of the CA that issued the public-key certificate (example, source : see p24 of this PDF)
subject
the entity for which the public-key certificate has been issued, typically the name of the company / organization running a website (example, source : see p18 of this PDF)
mail

SNI

One option for dealing with HTTPS virtual hosting is a modification to the SSL protocol known as SNI. This modification includes the host name the client is attempting to connect to in the SSL connection request, allowing the server to know which SSL certificate to use for the connection.
mail

How to read metadata of a certificate (Common Names, Subject Alternative Names, dates, ) ?

Different methods are available to read metadata whether the certificate is

For local certificates, there are quick'n'dirty methods which work fairly well :

Bare metal :
strings whatever.der
This will output a screenful of crap and somewhere in the middle, the common names.
With gnutls-bin (source) :
certtool -i < /usr/share/ca-certificates/mozilla/DigiCert_Global_Root_CA.crt | grep 'Subject:' 
Subject: CN=DigiCert Global Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US

Other solutions, both for local and remote certificates :

Use OpenSSL.
mail

How to create a CSR ?

Create a CSR :

  1. Define the CN
  2. Create a passwordless private key
  3. Generate the CSR itself :
    openssl req -new -key "$CN".key -out "$CN".csr -sha256 -subj "/C=US/ST=New Jersey/L=Fairfield/O=ACME Corporation/OU=Acme Rocket-Powered Products, Inc./CN=$CN"

Try it :

tmpDir=$(mktemp -d); cd "$tmpDir"; export CN='www.acme.com'; openssl genrsa -out "$CN".key 2048; openssl req -new -key "$CN".key -out "$CN".csr -sha256 -subj "/C=US/ST=New Jersey/L=Fairfield/O=ACME Corporation/OU=Acme Rocket-Powered Products, Inc./CN=$CN"; ls -l; rm "$CN".{key,csr}; cd -; rmdir "$tmpDir"
-rw-r--r-- 1 bob users 1066 Mar 10 14:39 www.acme.com.csr
-rw------- 1 bob users 1675 Mar 10 14:39 www.acme.com.key

Check the contents of a CSR :

openssl req -in "$CN".csr -noout -text

Try it :

tmpDir=$(mktemp -d); cd "$tmpDir"; export CN='www.acme.com'; openssl genrsa -out "$CN".key 2048; openssl req -new -key "$CN".key -out "$CN".csr -sha256 -subj "/C=US/ST=New Jersey/L=Fairfield/O=ACME Corporation/OU=Acme Rocket-Powered Products, Inc./CN=$CN"; openssl req -in "$CN".csr -noout -text; rm "$CN".{key,csr}; cd -; rmdir "$tmpDir"
Certificate Request:
	Data:
		Version: 1 (0x0)
		Subject: C = US, ST = New Jersey, L = Fairfield, O = ACME Corporation, OU = "Acme Rocket-Powered Products, Inc.", CN = www.acme.com
		Subject Public Key Info:
			Public Key Algorithm: rsaEncryption
				Public-Key: (2048 bit)
				Modulus:
					5c:d0:52:6b:e6:8d:82:ba:50:ed:3f:ea:24:42:81:
					(encoded junk)
					9f:ce:50:4e:9c:7f:01:91:46:14:18:66:e2:a7:41:
					bd:8b
				Exponent: 65537 (0x10001)
		Attributes:
			a0:00
	Signature Algorithm: sha256WithRSAEncryption
		8e:7a:33:c2:c3:ec:21:e3:09:9b:2b:dc:46:41:a3:3d:65:15:
		(encoded junk)
		88:91:c9:d4:c3:6d:86:9d:90:e8:da:26:d0:f2:5f:aa:7c:3f:
		e3:70:0e:a5

Next step :

The purpose of a CSR is to apply for a new certificate to a CA. So you'll have to transmit the CSR to the CA and ask a client / server certificate :

X509v3 Extended Key Usage:
				TLS Web Server Authentication, TLS Web Client Authentication

Make sure the private key matches the certificate :

$ openssl x509 -noout -modulus -in "$CN".crt | openssl md5
15c5f1ca9276647b3d4da491b2d48482

$ openssl rsa -noout -modulus -in "$CN".key | openssl md5
Enter pass phrase for "$CN".key:
15c5f1ca9276647b3d4da491b2d48482
mail

How to create a self-signed certificate ?

To do so, we'll need to generate a private key, which will be used to sign our certificate.
  1. Define the CN. This can be the web server hostname, or the virtualhost name :
    export CN='commonName'
  2. Create a passwordless private key :
    openssl genrsa -out "$CN".key 2048
  3. Create the certificate :
    openssl req -new -x509 -days 365 -sha256 -key "$CN".key -out "$CN".crt -subj "/C=US/ST=New Jersey/L=Fairfield/O=ACME Corporation/OU=Acme Rocket-Powered Products, Inc./CN=$CN"
  4. Concatenate the key and the certificate :
    cat "$CN".crt "$CN".key < "$CN".pem
mail

SSL / TLS certificates : format, file types, metadata

Format :

  • The binary format of a certificate (contents format and encoding rules) is defined using the ASN.1 notation (X208) PKCS.
  • When a certificate can not be sent in binary form, it may be translated into ASCII with Base64 encoding (See also MIME-encoding), which gives a PEM-encoded certificate.

File types :

file extension contains format description usage context origin
  • .csr
  • .p10
 pkcs10 Certificate Signing Request (details). .csr extension is Apache mod_ssl practice.
  • .key
  • .p8
 private key pkcs8 private key (details). .key is Apache mod_ssl practice.
  • .cer
  • .crt
  • .der
 CA certificate binary CA X.509 certificate.
The DER format is simply a binary form of a certificate instead of the ASCII PEM format. The only way to tell the difference between a DER .cer file and a PEM .cer file is to open it in a text editor and look for the BEGIN / END statements. All types of certificates and private keys can be encoded in DER format.		 DER is typically used with Java platforms. .crt was introduced by Netscape.
  • .p7b
  • .p7c
  • .spc
  • certificates
  • chain certificates
  • no private key
 base64 ASCII The PKCS#7 and P7B certificates contain -----BEGIN PKCS7----- and -----END PKCS7----- statements. Several platforms support P7B files including Microsoft Windows and Java Tomcat.
x-pkcs7-certificates, a "certs-only PKCS#7 bundle" (???). These extensions were introduced by Microsoft.
  • .cer
  • .crt
  • .key
  • .pem
  • server certificate
  • intermediate certificate
  • private key
 base64 ASCII
  • The PEM format is the most common format that CA issue certificates in. This format contains -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- statements.
  • The .pem extension stands for Privacy Enhanced Mail. Since OpenSSL defaults to PEM encoding, almost all open-source software use PEM formatted .crt files locally.
  • .pem files can be generated by concatenating in this specific order :
    1. the server certificate itself : server.crt
    2. the server private key : server.key
    3. the 1st intermediate CA certificate (who signed server.crt) : intermediate_CA_1.pem
    4. the 2nd intermediate CA certificate (who signed intermediate_CA_1.pem) : intermediate_CA_2.pem
    cat server.crt server.key intermediate_CA_1.pem intermediate_CA_2.pem > server.pem

Apache configuration:

In the virtualhost configuration file : 
SSLCertificateFile	/appli/projects/tri/apache_2.2.24/conf/ssl/appName_environment.pem
SSLCertificateKeyFile	/appli/projects/tri/apache_2.2.24/conf/ssl/appName_environment.key
SSLCertificateChainFile	/appli/projects/tri/apache_2.2.24/conf/ssl/appName_environment-bundle.pem
mail

Theory of SSL / TLS

The HTTPS frenzy...

Over the last few years, several initiatives such as :

  • HTTPS Everywhere
  • Google better ranking pages served via HTTPS rather than those served by HTTP
lead to the current "HTTPS frenzy" : even though you don't know what it is, what it is for, whether this is "good" or not, you have to have it.

In order to make enlightened decisions —since there's definitely no magic involved— here is what HTTPS offers :

  1. cryptography : make sure private information (like your credit card number) stays private. Information must be readable only by the recipient.
  2. integrity : make sure the message was not altered on-the-fly during transmission
  3. authentication : make sure you're actually talking to who you think you're talking to
In practice, these (+ key exchange methods) are gathered into cipher suites.

Keep private things private : cryptography :

2 "flavors" of cryptography :

  • Conventional cryptography (symmetric cryptography) : Bob and Alice share a secret key, which is used to encrypt/decrypt messages. The problem with this method is sharing the key securely.
  • Public-key cryptography : Bob has a private and a public key. So does Alice. Anyone can read Bob's public key, as it's public, and anyone can encrypt messages with it. But as it takes Bob's private key to decrypt them, only Bob can. This method solves the key exchange problem.

Ensure the message is not altered on-the-fly : Message digests :

Although Alice may encrypt her message to make it private, there is still a concern that someone might modify her original message or substitute it with a different one, in order to change its contents, or its recipient. One way of guaranteeing the integrity of Alice's message is for her to create a concise summary of her message and send this to Bob as well. Upon receipt of the message, Bob creates its own summary and compares it with the one Alice sent. If the summaries are the same then the message has been received intact.

A summary such as this is called a message digest, one-way function or hash function. Message digests are used to create a unique, short, fixed-length representation of a longer, variable-length message.

The message digest must be sent securely by Alice to Bob. Otherwise, it will be impossible for Bob to determine the integrity of the message he receives. This is where digital signatures join the game.

Ensure the message was sent by whom claims sending it : digital signatures (details) :

Upon receiving a message like :
Dear Bob,
 ...
 ...
	XOXOX
	Alice
Bob needs to check it was really sent to him by Alice, thanks to a digital signature created by Alice and included with the message. 
Digital signature = ( digest of the message + sequence number ) x encrypt with sender's private key
  • Anyone can decrypt the digital signature using the public key, which means (in case of success) the corresponding private key (that is known by the sender only) was used to encrypt the signature : the sender is authenticated.
  • Including the digest in the signature means the signature is only good for that message. It also ensures the integrity of the message since no one can change the digest and still sign it (because they don't have the private key).
  • To guard against interception and reuse of the signature by an intruder at a later date, the signature contains a unique sequence number. This protects Bob from a fraudulent claim from Alice that she did not send the message -- only she could have signed it (non-repudiation).

Validate the peer's identity : certificates (details) :

Even though Alice is capable of sending a private message to Bob, ensure its integrity and sign it, she must also check she's really sending it to Bob :
  • Alice has to make sure that the public key she's using is Bob's
  • Bob needs to check the signatures he'll get were made from Alice's private key
A certificate (basic example) :
  • validates the other's identity (called the subject). The subject can be a person, a server, or any entity. A distinguished name is used to provide an identity in a specific context. For example, Alice may have a private certificate to discuss with Bob, and another one she uses at work. Distinguished names are defined by the X.509 standard, which defines the fields, field names and abbreviations used to refer to the fields (details).
  • confirms the public key
  • is signed by a trusted agency known as Certificate Authority.
See also : binary / ASCII format of a certificate.

Glossary :

Please refer to the glossary for definitions.

SSL :

SSL is a protocol layer which may be placed between a reliable connection-oriented network layer protocol (e.g. TCP/IP) and the application protocol layer (e.g. HTTP). SSL provides for secure communication between client and server by allowing :
  • mutual authentication
  • use of digital signatures for integrity
  • encryption for privacy
It supports a range of choices for specific algorithms used for cryptography, digests and signatures. This allows algorithm selection for specific servers to be made based on legal, export or other concerns and also enables the protocol to take advantage of new algorithms. Choices are negotiated between client and server when establishing a protocol session, this is called the Cipher Suite Negotiation. This defines :
  • Key Exchange Method : how the shared secret symmetric cryptography key used for application data transfer will be agreed upon by client and server
  • Cipher for Data Transfer. SSL uses conventional symmetric cryptography for encrypting messages in a session. There are 9 choices of how to encrypt, including the option not to encrypt:
    • No encryption
    • Stream Ciphers :
      • RC4 with 40-bit keys
      • RC4 with 128-bit keys
    • CBC Block Ciphers :
      • RC2 with 40 bit key (Details : 1)
      • DES with 40 bit key (Details : 1)
      • DES with 56 bit key
      • Triple-DES with 168 bit key (Details : 1)
      • Idea (128 bit key) (Details : 1, 2)
      • Fortezza (96 bit key)
    Current common algorithms are : AES, Blowfish, DES, Triple DES, Serpent, Twofish
  • Message Digest for creating the MAC : none / MD5 (128-bit hash) / SHA-1 (160-bit hash)

In Apache, SSL is provided by mod_ssl, which implements the OpenSSL cryptography engine.

TLS :

TLS is the successor of SSL. After SSL 3.0, the next version emerged as TLS 1.0 in 1999. Just as SSL, TLS also provides security services such as confidentiality, integrity, and end point authentication. Similarly encryption, message authentication code, and digital certificates are used to provide these security services. TLS is immune to attacks such as POODLE attack, which has compromised the security of SSL 3.0.

What is the difference between SSL and TLS ?

Read this, and this.

SSL and TLS are NOT encryption mechanisms. They are options that dictate how the secure connection will be initiated.
No matter which method you choose for initiating the connection, TLS or SSL, the same level of encryption will be obtained when talking to the server and that level is determined by the software installed on the server, how that is configured, and what your program actually supports.

Additional details :

When making HTTPS queries, everything apart from the hostname is encrypted (source). Actually, after the SSL handshake is complete, an encrypted session is started to exchange data between client and server. The hostname stays unencrypted just because this is the destination of IP packets (?).