Skip to content

Latest commit

 

History

History
322 lines (266 loc) · 14.4 KB

certificate.md

File metadata and controls

322 lines (266 loc) · 14.4 KB
Raw
title
Certificate

APISIX supports to load multiple SSL certificates by TLS extension Server Name Indication (SNI).

Single SNI

It is most common for an SSL certificate to contain only one domain. We can create an ssl object. Here is a simple case, creates a ssl object and route object.

  • cert: PEM-encoded public certificate of the SSL key pair.
  • key: PEM-encoded private key of the SSL key pair.
  • snis: Hostname(s) to associate with this certificate as SNIs. To set this attribute this certificate must have a valid private key associated with it.

We will use the Python script below to simplify the example:

#!/usr/bin/env python
# coding: utf-8
import sys
# sudo pip install requests
import requests

if len(sys.argv) <= 3:
    print("bad argument")
    sys.exit(1)
with open(sys.argv[1]) as f:
    cert = f.read()
with open(sys.argv[2]) as f:
    key = f.read()
sni = sys.argv[3]
api_key = "edd1c9f034335f136f87ad84b625c8f1"
resp = requests.put("http://127.0.0.1:9180/apisix/admin/ssls/1", json={
    "cert": cert,
    "key": key,
    "snis": [sni],
}, headers={
    "X-API-KEY": api_key,
})
print(resp.status_code)
print(resp.text)
# create SSL object
./create-ssl.py t.crt t.key test.com

# create Router object
curl http://127.0.0.1:9180/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -i -d '
{
    "uri": "/hello",
    "hosts": ["test.com"],
    "methods": ["GET"],
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "127.0.0.1:1980": 1
        }
    }
}'

# make a test

curl --resolve 'test.com:9443:127.0.0.1' https://test.com:9443/hello  -vvv
* Added test.com:9443:127.0.0.1 to DNS cache
* About to connect() to test.com port 9443 (#0)
*   Trying 127.0.0.1...
* Connected to test.com (127.0.0.1) port 9443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* 	subject: CN=test.com,O=iresty,L=ZhuHai,ST=GuangDong,C=CN
* 	start date: Jun 24 22:18:05 2019 GMT
* 	expire date: May 31 22:18:05 2119 GMT
* 	common name: test.com
* 	issuer: CN=test.com,O=iresty,L=ZhuHai,ST=GuangDong,C=CN
> GET /hello HTTP/1.1
> User-Agent: curl/7.29.0
> Host: test.com:9443
> Accept: */*

wildcard SNI

Sometimes, one SSL certificate may contain a wildcard domain like *.test.com, that means it can accept more than one domain, eg: www.test.com or mail.test.com.

Here is an example, note that the value we pass as sni is *.test.com.

./create-ssl.py t.crt t.key '*.test.com'

curl http://127.0.0.1:9180/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -i -d '
{
    "uri": "/hello",
    "hosts": ["*.test.com"],
    "methods": ["GET"],
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "127.0.0.1:1980": 1
        }
    }
}'

# make a test

curl --resolve 'www.test.com:9443:127.0.0.1' https://www.test.com:9443/hello  -vvv
* Added test.com:9443:127.0.0.1 to DNS cache
* About to connect() to test.com port 9443 (#0)
*   Trying 127.0.0.1...
* Connected to test.com (127.0.0.1) port 9443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* 	subject: CN=test.com,O=iresty,L=ZhuHai,ST=GuangDong,C=CN
* 	start date: Jun 24 22:18:05 2019 GMT
* 	expire date: May 31 22:18:05 2119 GMT
* 	common name: test.com
* 	issuer: CN=test.com,O=iresty,L=ZhuHai,ST=GuangDong,C=CN
> GET /hello HTTP/1.1
> User-Agent: curl/7.29.0
> Host: test.com:9443
> Accept: */*

multiple domain

If your SSL certificate may contain more than one domain, like www.test.com and mail.test.com, then you can add them into the snis array. For example:

{
    "snis": ["www.test.com", "mail.test.com"]
}

multiple certificates for a single domain

If you want to configure multiple certificate for a single domain, for instance, supporting both the ECC and RSA key-exchange algorithm, then just configure the extra certificates (the first certificate and private key should be still put in cert and key) and private keys by certs and keys.

  • certs: PEM-encoded certificate array.
  • keys: PEM-encoded private key array.

APISIX will pair certificate and private key with the same indice as a SSL key pair. So the length of certs and keys must be same.

set up multiple CA certificates

APISIX currently uses CA certificates in several places, such as Protect Admin API, etcd with mTLS, and Deployment Modes.

In these places, ssl_trusted_certificate or trusted_ca_cert will be used to set up the CA certificate, but these configurations will eventually be translated into lua_ssl_trusted_certificate directive in OpenResty.

If you need to set up different CA certificates in different places, then you can package these CA certificates into a CA bundle file and point to this file when you need to set up CAs. This will avoid the problem that the generated lua_ssl_trusted_certificate has multiple locations and overwrites each other.

The following is a complete example to show how to set up multiple CA certificates in APISIX.

Suppose we let client and APISIX Admin API, APISIX and ETCD communicate with each other using mTLS protocol, and currently there are two CA certificates, foo_ca.crt and bar_ca.crt, and use each of these two CA certificates to issue client and server certificate pairs, foo_ca.crt and its issued certificate pair are used to protect Admin API, and bar_ca.crt and its issued certificate pair are used to protect ETCD.

The following table details the configurations involved in this example and what they do:

Configuration Type Description
foo_ca.crt CA cert Issues the secondary certificate required for the client to communicate with the APISIX Admin API over mTLS.
foo_client.crt cert A certificate issued by foo_ca.crt and used by the client to prove its identity when accessing the APISIX Admin API.
foo_client.key key Issued by foo_ca.crt, used by the client, the key file required to access the APISIX Admin API.
foo_server.crt cert Issued by foo_ca.crt, used by APISIX, corresponding to the admin_api_mtls.admin_ssl_cert configuration entry.
foo_server.key key Issued by foo_ca.crt, used by APISIX, corresponding to the admin_api_mtls.admin_ssl_cert_key configuration entry.
admin.apisix.dev doname Common Name used in issuing foo_server.crt certificate, through which the client accesses APISIX Admin API
bar_ca.crt CA cert Issues the secondary certificate required for APISIX to communicate with ETCD over mTLS.
bar_etcd.crt cert Issued by bar_ca.crt and used by ETCD, corresponding to the -cert-file option in the ETCD startup command.
bar_etcd.key key Issued by bar_ca.crt and used by ETCD, corresponding to the --key-file option in the ETCD startup command.
bar_apisix.crt cert Issued by bar_ca.crt, used by APISIX, corresponding to the etcd.tls.cert configuration entry.
bar_apisix.key key Issued by bar_ca.crt, used by APISIX, corresponding to the etcd.tls.key configuration entry.
etcd.cluster.dev key Common Name used in issuing bar_etcd.crt certificate, which is used as SNI when APISIX communicates with ETCD over mTLS. corresponds to etcd.tls.sni configuration item.
apisix.ca-bundle CA bundle Merged from foo_ca.crt and bar_ca.crt, replacing foo_ca.crt and bar_ca.crt.
  1. Create CA bundle files
cat /path/to/foo_ca.crt /path/to/bar_ca.crt > apisix.ca-bundle
  1. Start the ETCD cluster and enable client authentication

Start by writing a goreman configuration named Procfile-single-enable-mtls, the content as:

# Use goreman to run `go get github.com/mattn/goreman`
etcd1: etcd --name infra1 --listen-client-urls https://127.0.0.1:12379 --advertise-client-urls https://127.0.0.1:12379 --listen-peer-urls http://127.0.0.1:12380 --initial-advertise-peer-urls http://127.0.0.1:12380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --cert-file /path/to/bar_etcd.crt --key-file /path/to/bar_etcd.key --client-cert-auth --trusted-ca-file /path/to/apisix.ca-bundle
etcd2: etcd --name infra2 --listen-client-urls https://127.0.0.1:22379 --advertise-client-urls https://127.0.0.1:22379 --listen-peer-urls http://127.0.0.1:22380 --initial-advertise-peer-urls http://127.0.0.1:22380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --cert-file /path/to/bar_etcd.crt --key-file /path/to/bar_etcd.key --client-cert-auth --trusted-ca-file /path/to/apisix.ca-bundle
etcd3: etcd --name infra3 --listen-client-urls https://127.0.0.1:32379 --advertise-client-urls https://127.0.0.1:32379 --listen-peer-urls http://127.0.0.1:32380 --initial-advertise-peer-urls http://127.0.0.1:32380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --cert-file /path/to/bar_etcd.crt --key-file /path/to/bar_etcd.key --client-cert-auth --trusted-ca-file /path/to/apisix.ca-bundle

Use goreman to start the ETCD cluster:

goreman -f Procfile-single-enable-mtls start > goreman.log 2>&1 &
  1. Update config.yaml
deployment:
  admin:
    admin_key
      - name: admin
        key: edd1c9f034335f136f87ad84b625c8f1
        role: admin
    admin_listen:
      ip: 127.0.0.1
      port: 9180
    https_admin: true
    admin_api_mtls:
      admin_ssl_ca_cert: /path/to/apisix.ca-bundle
      admin_ssl_cert: /path/to/foo_server.crt
      admin_ssl_cert_key: /path/to/foo_server.key

apisix:
  ssl:
    ssl_trusted_certificate: /path/to/apisix.ca-bundle

deployment:
  role: traditional
  role_traditional:
    config_provider: etcd
  etcd:
    host:
      - "https://127.0.0.1:12379"
      - "https://127.0.0.1:22379"
      - "https://127.0.0.1:32379"
    tls:
      cert: /path/to/bar_apisix.crt
      key: /path/to/bar_apisix.key
      sni: etcd.cluster.dev
  1. Test APISIX Admin API

Start APISIX, if APISIX starts successfully and there is no abnormal output in logs/error.log, it means that mTLS communication between APISIX and ETCD is normal.

Use curl to simulate a client, communicate with APISIX Admin API with mTLS, and create a route:

curl -vvv \
    --resolve 'admin.apisix.dev:9180:127.0.0.1' https://admin.apisix.dev:9180/apisix/admin/routes/1 \
    --cert /path/to/foo_client.crt \
    --key /path/to/foo_client.key \
    --cacert /path/to/apisix.ca-bundle \
    -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -i -d '
{
    "uri": "/get",
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "httpbin.org:80": 1
        }
    }
}'

A successful mTLS communication between curl and the APISIX Admin API is indicated if the following SSL handshake process is output:

* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, CERT verify (15):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
  1. Verify APISIX proxy
curl http://127.0.0.1:9080/get -i

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 298
Connection: keep-alive
Date: Tue, 26 Jul 2022 16:31:00 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Server: APISIX/2.14.1

...

APISIX proxied the request to the /get path of the upstream httpbin.org and returned HTTP/1.1 200 OK. The whole process is working fine using CA bundle instead of CA certificate.