How the ACME protocol works with a private CA: RFC 8555 flow, challenges, ACME vs EST vs SCEP vs CMP, and certbot, acme.sh and cert-manager setup with DuoKey.
ACME with a private CA means running the Automatic Certificate Management Environment protocol (RFC 8555), the same protocol Let's Encrypt uses, in front of your own internal certificate authority. Servers, Kubernetes clusters and load balancers request, prove control of, receive and renew their internal TLS certificates automatically, using standard clients such as certbot, acme.sh or cert-manager, with no tickets and no manual CSR handling.
Certificate outages are rarely cryptographic failures. They are calendar failures: a certificate nobody tracked expires on a Saturday. The CA/Browser Forum has adopted a schedule that cuts the maximum validity of public TLS certificates from 200 days today to 47 days in 2029. Internal certificates aren't bound by it, but the same teams run both, and a manual process built for yearly renewals won't survive monthly ones.
This guide explains how ACME works, which challenge types fit an internal network, how ACME compares with EST, SCEP and CMP, and how to point certbot, acme.sh and Kubernetes cert-manager at a private CA in DuoKey Cockpit, including revocation, post-quantum options and troubleshooting.
The pressure comes from public TLS. CA/Browser Forum ballot SC-081v3 introduced shorter maximum validity periods for publicly trusted TLS certificates. From the Baseline Requirements' Relevant Dates table:
| Certificates issued on or after | Maximum validity |
|---|---|
| 15 March 2026 | 200 days |
| 15 March 2027 | 100 days |
| 15 March 2029 | 47 days |
The full timeline, including domain validation reuse periods, is covered in our 47-day TLS certificate lifetime guide.
These rules apply to public TLS only; a certificate from your own internal CA isn't covered. Three things carry over anyway:
ACME is defined in RFC 8555 (March 2019, Proposed Standard). It's a JSON-over-HTTPS protocol in which every request is signed with the client's account key (JWS). A server publishes a single directory URL, and clients discover every other endpoint (newNonce, newAccount, newOrder, revokeCert) from it.
| Step | RFC 8555 section | What happens |
|---|---|---|
| 1. Account | 7.3 | The client generates an account key pair and sends a signed newAccount request. The server returns an account URL. |
| 2. Order | 7.4 | The client submits a newOrder listing the identifiers (DNS names) it wants in the certificate. |
| 3. Authorization | 7.5 | The server returns one authorization per identifier, each with a set of possible challenges. |
| 4. Challenge | 8 | The client provisions one challenge response (an HTTP file, a DNS TXT record or a TLS-ALPN certificate) and tells the server it is ready. The server validates it. |
| 5. Finalize | 7.4 | Once all authorizations are valid, the client sends a PKCS#10 CSR to the order's finalize URL. |
| 6. Download | 7.4.2 | The server issues the certificate; the client fetches the chain with a POST-as-GET request. |
Revocation (section 7.6) is a separate request to revokeCert, carrying the certificate and an optional reason code taken from RFC 5280.
None of this is specific to Let's Encrypt: any RFC 8555 client works with any RFC 8555 server. For a private CA you change one setting, the directory URL.
A challenge proves that the requester controls the name it is asking for. Three types are standardized:
| Challenge | Defined in | How it is validated | Fits internal networks when |
|---|---|---|---|
| http-01 | RFC 8555, 8.3 | The ACME server fetches http://<name>/.well-known/acme-challenge/<token> on TCP port 80 | The CA can reach the host over HTTP (same network, or routed); typical for ingress controllers and web servers |
| dns-01 | RFC 8555, 8.4 | The ACME server looks up a TXT record at _acme-challenge.<name> containing a digest of the key authorization | The host is not reachable from the CA and your internal DNS has an API the client can update |
| tls-alpn-01 | RFC 8737 (Feb. 2020) | The ACME server connects on TCP port 443 with ALPN acme-tls/1 and checks a self-signed certificate carrying the acmeIdentifier extension | The client controls the TLS listener on 443 (for example a reverse proxy) and port 80 is closed |
For internal PKI the deciding factor is reachability: the validation request travels from your CA to your host, so firewall rules between the two matter as much as the challenge type.
DuoKey's ACME server supports http-01 and dns-01. http-01 is validated for real: DuoKey fetches the token from the requester's domain. For lab or trusted-network CAs an administrator can opt into skipping that check; it is off by default.
RFC 8555 section 7.3.4 defines External Account Binding (EAB). It links a new ACME account to an account the CA already knows, such as a customer record. The CA hands out a key identifier and a MAC key out of band; the client includes an externalAccountBinding JWS, signed with that MAC key, in its newAccount request. A server that requires it sets externalAccountRequired: true in the directory's meta object.
cert-manager supports it through spec.acme.externalAccountBinding (keyID and keySecretRef); check your client's documentation for its EAB options.
DuoKey's documentation does not describe EAB for its ACME server, so don't plan a design around it; contact DuoKey if EAB is a requirement.
ACME fits servers and Kubernetes. Most estates also need at least one other enrollment protocol.
| Protocol | Standard | Transport and auth | Typical use | Strength | Limitation |
|---|---|---|---|---|---|
| ACME | RFC 8555 (Proposed Standard, 2019) | JSON over HTTPS, JWS-signed requests, challenge-based proof of control | Web servers, ingress, Kubernetes, load balancers | Large client ecosystem (certbot, acme.sh, cert-manager, Caddy); renewal built in | Built around DNS names; challenges need network reachability |
| EST | RFC 7030 (Proposed Standard, 2013) | HTTPS (TLS) under /.well-known/est/ | Routers, gateways, IoT devices | Simple re-enrollment with an existing certificate; optional server-side key generation | Smaller client ecosystem than ACME |
| SCEP | RFC 8894 (Informational, 2020) | CMS messages over HTTP, shared challengePassword | MDM-managed endpoints, network devices | Simple flow; pending requests can wait for operator approval | Informational only; shared secret for first enrollment |
| CMP | RFC 4210, now obsoleted by RFC 9810 (2025); RFC 9483 Lightweight CMP Profile (2023) | CMP messages over HTTP or CoAP | Industrial, operator and machine-to-machine PKI | Rich lifecycle operations; profile designed for industrial and IoT | More complex to implement |
Rule of thumb: ACME for anything with a DNS name that runs a web or API listener; EST for devices that can do TLS but not ACME; SCEP where an MDM or a legacy device leaves you no choice; CMP for industrial PKIs. DuoKey Cockpit exposes all four (plus a KMIP 2.1 endpoint), so the protocol choice doesn't force a second PKI.
The steps below follow DuoKey's enrollment protocols documentation. The client commands are generic usage of each client with a custom ACME directory URL, not DuoKey-specific tooling.
In DuoKey certificates management, create a root CA and an issuing (intermediate) CA, or import an external CA. Choose the algorithm family:
Cockpit enforces parent-algorithm compatibility so PQC and hybrid chains verify. PQC and hybrid CAs are vault-signed: their signing keys live in the vault or HSM and are not generated locally.
Enable ACME on the issuing CA (Cockpit → PKI → ACME → Configure). The CA's directory URL follows this format:
https://{cockpit-url}/api/pki/acme/{ca-id}/directory
There's one directory per CA, so you can run separate CAs (for example production, lab, PQC pilot) with separate directories.
Distribute your root CA certificate to the hosts and clients that will validate the issued certificates. If the Cockpit endpoint itself is served with a certificate from a private CA, the ACME client must trust that too: certbot honours the REQUESTS_CA_BUNDLE environment variable (set it globally, or scheduled renewals will fail), and acme.sh accepts --ca-bundle <file>.
# http-01 with an existing web server (webroot)
sudo certbot certonly \
--server https://{cockpit-url}/api/pki/acme/{ca-id}/directory \
--webroot -w /var/www/html \
-d app.corp.example.com
# or let certbot answer http-01 itself on port 80
sudo certbot certonly --standalone \
--server https://{cockpit-url}/api/pki/acme/{ca-id}/directory \
-d app.corp.example.com
# renewal: run on a schedule (timer or cron)
sudo certbot renew
acme.sh --register-account \
--server https://{cockpit-url}/api/pki/acme/{ca-id}/directory \
-m [email protected]
acme.sh --issue \
--server https://{cockpit-url}/api/pki/acme/{ca-id}/directory \
-d app.corp.example.com -w /var/www/html
acme.sh --install-cert -d app.corp.example.com \
--key-file /etc/nginx/tls/app.key \
--fullchain-file /etc/nginx/tls/app.fullchain.pem \
--reloadcmd "systemctl reload nginx"
acme.sh's default CA is ZeroSSL, so pass --server on each command or set it once with acme.sh --set-default-ca --server <directory-url>. For dns-01, replace -w with --dns <your_dns_hook>.
cert-manager with ACME is the path DuoKey has verified end to end, and the one its documentation recommends for Kubernetes today (a native cert-manager external issuer is still rolling out). Prerequisites: ACME enabled on the CA, cert-manager installed, an ingress controller for the http-01 solver, and a domain Cockpit can reach. DuoKey rejects loopback and link-local targets.
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: duokey-private-acme
spec:
acme:
# ACME directory of the DuoKey issuing CA
server: https://{cockpit-url}/api/pki/acme/{ca-id}/directory
email: [email protected]
# Secret where cert-manager stores the ACME account private key
privateKeySecretRef:
name: duokey-private-acme-account-key
# Only needed if the Cockpit endpoint uses a certificate from a private CA
# (cert-manager 1.11+): base64-encoded PEM bundle
# caBundle: <base64-encoded-PEM-CA-bundle>
solvers:
- http01:
ingress:
ingressClassName: nginx
To use dns-01 instead, replace the http01 solver with one of cert-manager's dns01 providers, configured per the cert-manager DNS01 documentation (for example a provider for your internal DNS server).
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: app-tls
namespace: default
spec:
secretName: app-tls
dnsNames:
- app.corp.example.com
issuerRef:
name: duokey-private-acme
kind: ClusterIssuer
group: cert-manager.io
cert-manager's default spec.duration is 90 days, and by default it renews at two-thirds of the certificate's lifetime. You can change that with spec.renewBefore (an absolute duration before expiry) or spec.renewBeforePercentage, which cert-manager recommends because it's computed from the certificate's actual lifetime and avoids renewal loops when the CA issues a shorter certificate than requested.
Some targets don't run an ACME client at all. For these, DuoKey Cockpit has deployment connectors with per-job validation and rollback, so a bad deployment can be reverted instead of leaving a service with a broken chain. All Cockpit enrollment endpoints (ACME, EST, SCEP, CMP, KMIP) share the same lifecycle: deploy, start, pause, stop, with health and metrics.
ACME clients can revoke through the revokeCert endpoint, and Cockpit can revoke directly. The reason codes come from RFC 5280 section 5.3.1:
| Code | Reason | Code | Reason |
|---|---|---|---|
| 0 | unspecified | 5 | cessationOfOperation |
| 1 | keyCompromise | 6 | certificateHold |
| 2 | cACompromise | 8 | removeFromCRL (delta CRLs only) |
| 3 | affiliationChanged | 9 | privilegeWithdrawn |
| 4 | superseded | 10 | aACompromise |
In practice, keyCompromise, superseded and cessationOfOperation cover most internal cases.
Each DuoKey CA publishes a CRL and runs an OCSP responder, so relying parties can check status. CAs themselves can be suspended and reactivated, revoked (simple or cascade) and archived. For requests that need a human decision, Cockpit also has a CSR approval workflow (request, approve or reject, issue) alongside ACME.
Automation only covers the certificates you know about. Pair private ACME with discovery: DuoKey's cryptographic posture management scans filesystems, keystores and cloud KMS for certificates and keys, and keeps a living CBOM.
ACME is algorithm-agnostic: the client sends a CSR, the CA signs it. A private ACME CA is a practical place to pilot ML-DSA certificates.
DuoKey supports ML-DSA and SLH-DSA CAs and a hybrid composite ML-DSA-65 + ECDSA-P256. Its documentation is direct about the constraint: "Not every browser, server or device understands ML-DSA / SLH-DSA yet." The recommendation is to use pure PQC where you control both ends, and the hybrid composite where classical relying parties must still verify the chain. Client support varies too: an ACME client may not yet generate ML-DSA keys or CSRs, so test each client and TLS stack first. For issued PQC certificates in DuoKey, the leaf key is vault-resident.
Because each CA has its own directory URL, moving a workload from a classical to a hybrid CA is a change of server in the issuer, followed by normal renewal.
Client can't reach the directory (TLS error). The client doesn't trust the certificate on the Cockpit endpoint. Set REQUESTS_CA_BUNDLE for certbot, --ca-bundle for acme.sh, or caBundle on the cert-manager issuer.
http-01 challenge fails. The CA must reach http://<name>/.well-known/acme-challenge/<token> on port 80. Check DNS resolution from the CA's network, firewalls between Cockpit and the host, and that the ingress or web server routes that path. Loopback and link-local names are rejected by DuoKey.
cert-manager Order stuck pending. Run kubectl describe certificate, then follow the CertificateRequest, Order and Challenge resources; the Challenge status shows the validation error.
dns-01 fails intermittently. The TXT record at _acme-challenge.<name> may not have propagated to the resolver the CA uses. Check split-horizon DNS: internal and external views can answer differently.
acme.sh issues from the wrong CA. acme.sh defaults to ZeroSSL. Pass --server or set the default CA.
Renewal loops in cert-manager. The issued certificate is shorter than renewBefore. Use renewBeforePercentage or lower renewBefore.
Client rejects a PQC or hybrid certificate. The relying party's TLS library doesn't support the algorithm yet. Issue from a classical or hybrid CA for that workload.
Q: What is the ACME protocol?
ACME (Automatic Certificate Management Environment, RFC 8555) is an IETF standard that lets a client request, validate, receive, renew and revoke X.509 certificates automatically over HTTPS. The client creates an account, places an order, proves control of each name through a challenge, sends a CSR and downloads the certificate.
Q: Can I use ACME with a private CA?
Yes. ACME isn't tied to public CAs. Any RFC 8555 server can front a private CA, and standard clients only need its directory URL. In DuoKey Cockpit each CA has its own directory at {cockpit-url}/api/pki/acme/{ca-id}/directory.
Q: Does cert-manager work with a private ACME server?
Yes. Set spec.acme.server on an Issuer or ClusterIssuer to the private directory URL, and add caBundle (cert-manager 1.11+) if the ACME endpoint uses a privately issued TLS certificate. cert-manager over ACME is the path DuoKey has verified end to end.
Q: ACME vs EST: which should I use?
Use ACME for servers, ingress and Kubernetes, where challenges prove control of DNS names. Use EST (RFC 7030) for routers, gateways and IoT devices, especially when they re-enroll with an existing certificate. Both can run against the same CA.
Q: Which ACME challenge works best on an internal network?
http-01 when the CA can reach the host on port 80; dns-01 when it can't and your internal DNS has an API. tls-alpn-01 (RFC 8737) needs control of the listener on port 443. DuoKey supports http-01 and dns-01.
Q: Do the 47-day certificate rules apply to internal certificates?
No. Ballot SC-081 changes the Baseline Requirements for publicly trusted TLS certificates. Internal CAs set their own policy, but short lifetimes and automated renewal are good practice for both.
Q: What is External Account Binding in ACME?
EAB (RFC 8555, section 7.3.4) ties a new ACME account to an account the CA already knows, using a key ID and MAC key handed out out of band. DuoKey's documentation doesn't describe EAB for its ACME server.
Q: Can ACME issue post-quantum certificates?
The protocol is algorithm-agnostic, so an ACME CA can issue ML-DSA or hybrid certificates if both the client's CSR and the CA support them. DuoKey offers ML-DSA, SLH-DSA and hybrid CAs; client and relying-party support still varies.
ACME turns internal certificate renewal from a calendar task into a background job, using clients your teams already know. With a private CA in DuoKey Cockpit, the same CAs serve ACME, EST, SCEP and CMP, and classical, hybrid and post-quantum chains sit side by side. To see where your current certificates and algorithms stand first, start with a free Quantum Risk Score.
Written by
DuoKey Team
Related Resources
Tell us where control is difficult today. We will help you identify a practical next step.
