Secure Edge Infrastructure with Consul, Nomad & Vault
Building a secure edge infrastructure requires careful integration of multiple components, especially when working with HashiCorp’s stack. While official documentation covers individual steps, combining Consul, Nomad, and Vault into a functional, secure platform often reveals hidden challenges. This guide provides a detailed, practical workflow to deploy these tools efficiently and securely.
Moreover, we will integrate ZippyOPS expertise, providing consulting, implementation, and managed services in areas such as DevOps, DevSecOps, DataOps, Cloud, Automated Ops, AIOps, MLOps, Microservices, Infrastructure, and Security. Learn how ZippyOPS can help streamline deployment and maintenance: ZippyOPS Services.
Physical Architecture for Edge Deployments
To achieve a resilient setup, we propose a cluster architecture consisting of five nodes:
- Three control plane nodes for Consul and Nomad configured with High Availability (HA).
- Two data plane nodes located in edge environments for user workloads.
Placement is critical: control plane nodes may reside in a main building or centralized cabinet, while client nodes are deployed at each edge site. Servers maintain cluster state, whereas clients handle user requests. This separation ensures performance, security, and high availability.
Preparing Nodes and Environment for Secure Edge Infrastructure
Before deploying on physical hardware, create a virtualized test environment. Using tools like Ansible, Terraform, and Git, you can iterate on configuration safely. Ansible Vault protects sensitive data, including passwords, keys, and TLS certificates, ensuring security without exposing secrets in your repositories.
Minimum virtual machine specifications for testing:
- OS: Ubuntu 22.04 LTS
- Disk: 20 GB
- Memory: 2 GB
- vCPU: 2
You may deploy these VMs via any public cloud provider or locally with VirtualBox and Vagrant. The Ansible inventory should differentiate server and client roles for seamless automation.
Deploying the Consul and Nomad Cluster
Using Ansible Galaxy, pull tested roles for Consul and Nomad. Configure a dummy network interface (dummy0) with a reserved IP (169.254.1.1) to resolve *.consul domains.
The playbook should also handle:
- OS updates
- User creation and SSH key deployment
- Container runtime installation
- Debugging tools
Set the appropriate role variables (consul_node_role, nomad_node_role) and enable Nomad CNI for network configuration. This approach supports rapid provisioning on production servers, edge devices, or development VMs.
For more automation and operational insights, ZippyOPS provides end-to-end consulting and managed services in cloud and microservices infrastructure: ZippyOPS Solutions.
Provisioning the Vault Cluster
Once Consul and Nomad are operational, deploy Vault in HA mode on the same control plane nodes. Start by creating a Root CA for TLS certificates:
openssl genrsa -aes256 -out rootCA.key 4096
openssl req -x509 -new -key rootCA.key -sha256 -days 1024 -out rootCA.crt
Next, generate Vault server certificates and include subject alternative names (SANs) for all nodes. Use Ansible to distribute certificates and configure the Vault role:
vault_backend: consul
vault_consul: 169.254.1.1:8500
vault_tls_cert_file: "vault.crt"
vault_tls_key_file: "vault.key"
vault_tls_ca_file: "rootCA.crt"
Finally, initialize and unseal Vault, verifying the cluster status.
For detailed guidance on certificate management and automation, explore resources from HashiCorp.
Enabling TLS for Consul and Nomad
TLS ensures secure communication for both infrastructure components and application workloads. Use Vault to issue certificates, distribute them with Consul Template, and enable mTLS authentication.
Consul TLS Setup
- Configure Vault PKI secrets engine.
- Issue server and client certificates via Vault.
- Enable Consul Connect for service mesh security.
- Deploy certificates to
/opt/consul/agent-certs/and enable TLS on all nodes.
Example TLS configuration for Consul server:
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
ca_file = "/opt/consul/agent-certs/ca.crt"
cert_file = "/opt/consul/agent-certs/agent.crt"
key_file = "/opt/consul/agent-certs/agent.key"
auto_encrypt { allow_tls = true }
Consul Template automates certificate renewal, preventing downtime due to expired certificates.
Nomad TLS Setup
Nomad TLS mirrors Consul steps but includes 127.0.0.1 in SANs for local service communication. Update Consul Template to pull certificates from multiple Vault paths and ensure sidecars use TLS in the service mesh. Finally, validate Nomad’s Web UI and API using Vault-issued certificates.
Configuring ACLs
Enable Consul Access Control Lists (ACLs) to secure cluster communications. Temporarily shrink the cluster to a single node for bootstrapping. After enabling ACLs, distribute policies and tokens for Vault, Nomad, and Consul agents. Enforce a default deny policy for maximum security.
This process ensures a fully secure environment where only authorized services and users can access sensitive resources.
Conclusion: Building a Secure Edge Infrastructure
Congratulations! Your secure edge infrastructure is now functional, supporting TLS-protected Consul, Nomad, and Vault clusters. You can deploy middleware such as Traefik, Prometheus, and Nomad Autoscaler for load balancing, monitoring, and autoscaling.
Looking ahead, consider a hybrid architecture where control plane nodes reside in the cloud while data plane servers operate at the edge. This approach enables centralized management, multi-site control, and cost efficiency.
ZippyOPS can assist in consulting, implementation, and ongoing management across DevOps, Cloud, and Security:
For a professional consultation, reach out to sales@zippyops.com.
