Consul Demystified: Service Discovery vs Service Mesh Explained

Consul is one of those tools that everyone mentions but few explain clearly. Is it a service discovery tool? A service mesh? A key-value store? A DNS server?

The answer is "yes to all of the above" - and that's exactly why it's confusing.

This guide breaks down what Consul actually does, when you need each capability, and how the pieces fit together. By the end, you'll understand when Consul is the right choice and when alternatives make more sense.

______

The Core Problem

In traditional infrastructure, services found each other through static configuration. Your app knew the database lived at 10.0.1.50:5432 because someone hardcoded it in a config file. The load balancer sat at a known IP. Life was simple.

This breaks down with modern infrastructure:

• Containers spin up and down with dynamic IPs
• Auto-scaling changes instance counts constantly
• Multi-region deployments need location-aware routing
• Microservices multiply the number of service-to-service connections exponentially

Suddenly "where is the payment service?" becomes a hard question. And "is it healthy?" becomes even harder.

Consul solves this with two main capabilities: Service Discovery and Service Mesh. Let's break each one down.

______

Service Discovery: Finding Services

Service discovery answers a simple question: "Where is the payment service running right now?"

How It Works

• Consul Servers store the service catalog and maintain consensus via Raft protocol. You run 3 or 5 for high availability.
• Consul Agents run alongside each service. They register services, perform health checks, and cache catalog data locally.

______

Service Registration

Services register themselves with metadata:

1service {
2  name = "payment-api"
3  port = 8080
4  
5  tags = ["v2", "production"]
6  
7  meta = {
8    version = "2.1.0"
9    team    = "payments"
10  }
11  
12  check {
13    http = "http://localhost:8080/health"
14    interval = "10s"
15    timeout = "2s"
16  }
17}

Service Lookup

Other services query Consul to find healthy instances:

# DNS interface - the elegant approach
dig @127.0.0.1 -p 8600 payment-api.service.consul

# HTTP API - for programmatic access
curl http://localhost:8500/v1/health/service/payment-api?passing=true

The DNS interface is particularly elegant. Configure your resolver to forward .consul queries to Consul, and services resolve like any hostname. Your application doesn't need to know Consul exists.

Health Checking

Consul continuously monitors service health. Unhealthy instances are automatically removed from query results. No traffic goes to broken services.

Health check types include:

• HTTP - Hit an endpoint, expect 2xx response
• TCP - Verify port is accepting connections
• Script - Run a custom script, check exit code
• TTL - Service must actively report health within a time window
• gRPC - Native gRPC health checking

______

Service Mesh: Securing and Managing Traffic

Service discovery tells you where services are. Service mesh controls how they communicate.

The Shift to Zero Trust

Traditional security used perimeter-based trust. If you're inside the network, you're trusted. Modern architectures assume the network is compromised - every service-to-service call needs authentication and authorization.

This is where Consul Connect (Consul's service mesh) comes in.

How Service Mesh Works

A sidecar proxy (typically Envoy) runs alongside each service. All traffic flows through the proxy, which handles:

• mTLS encryption - Every connection is encrypted with mutual TLS. Services authenticate each other using certificates issued by Consul's built-in CA. No plaintext traffic on the network.
• Authorization - Intentions define which services can talk to which. "Payment service can call database, but not user service."
• Observability - Proxies emit metrics, traces, and logs for all traffic automatically.

Intentions: Service-to-Service Authorization

1Kind = "service-intentions"
2Name = "payment-api"
3
4Sources = [
5  {
6    Name   = "web-frontend"
7    Action = "allow"
8  },
9  {
10    Name   = "*"
11    Action = "deny"  # Default deny all others
12  }
13]

Intentions are identity-based, not IP-based. Even if an attacker compromises a container and gets on your network, they can't impersonate another service without its certificate.

Traffic Management

Beyond security, the mesh handles traffic routing:

1Kind = "service-splitter"
2Name = "payment-api"
3
4Splits = [
5  {
6    Weight        = 90
7    ServiceSubset = "v1"
8  },
9  {
10    Weight        = 10
11    ServiceSubset = "v2"
12  }
13]

Use cases include canary deployments, blue-green releases, and A/B testing - all without changing application code.

______

Consul vs Alternatives

For Service Discovery

• Consul - Distributed, multi-datacenter, works across platforms. Best for hybrid/multi-cloud.
• AWS Cloud Map - AWS-native, simple setup. Best for pure AWS environments.
• Kubernetes DNS - Built into K8s, zero setup. Best for single K8s cluster.
• etcd - Key-value based, requires more work. Best if you're already using etcd.

For Service Mesh

• Consul Connect - Works across Kubernetes, VMs, ECS, bare metal. Best for multi-platform environments.
• Istio - Kubernetes-focused, massive feature set. Best for pure Kubernetes.
• Linkerd - Simple, performant, Kubernetes-native. Best when you want simplicity.
• AWS App Mesh - AWS-native, Envoy-based. Best for pure AWS.

Consul's strength is working across platforms. If you have Kubernetes, VMs, ECS, and bare metal all needing to communicate - Consul handles that. If you're pure Kubernetes, Istio or Linkerd might be simpler.

______

When to Use Consul

Use Consul when:

• Running microservices across multiple platforms (K8s + VMs + ECS)
• You need multi-datacenter service discovery
• You want service mesh without Kubernetes dependency
• You're already in the HashiCorp ecosystem (Vault, Nomad integrate tightly)

Consider alternatives when:

• Pure Kubernetes environment (Istio or Linkerd may be simpler)
• Pure AWS (Cloud Map + App Mesh are native options)
• Simple architecture with few services (might be overkill)

_______

Wrapping Up

Consul does two related but distinct things:

• Service Discovery - A distributed registry that tracks where services are running, with health checking to route only to healthy instances. Use this when you need dynamic service location.
• Service Mesh - A security and traffic management layer using sidecar proxies, providing mTLS, authorization, and traffic control. Use this when you need zero-trust security or advanced traffic management.

You can use either capability independently. Start with service discovery if you just need dynamic service location. Add the mesh when zero-trust security or canary deployments become necessary.

For hybrid environments spanning Kubernetes, VMs, and multiple clouds - Consul remains one of the most flexible options available. It's not always the simplest choice, but when your infrastructure spans multiple platforms, that flexibility becomes essential.