Lecture 170: Main calls of microservices: data consistency, monitoring complexity, fault tolerance

Today we're diving deeper to talk about the main challenges of microservice architecture: how to handle data consistency, monitoring, and fault tolerance.

Data consistency in microservices

Ah, data consistency... Monolith developers usually take it for granted: there's a single database and changes happen inside one transaction — everything's logical and transparent. But once you move to microservices, that "simplicity illusion" disappears.

Why does the problem appear? The thing is, in a microservice architecture every service usually has its own database (yeah, data isolation is everything). But that means operations that touch multiple services become "tricky". For example, what if you have an Order service and a Payment service and you need to make sure both the order and the payment succeeded?

You can't just say: "Okay, I'll do it all in one transaction." No, my friend, distributed systems demand a more sophisticated approach!

The fix

Distributed transactions can be implemented with the Sagas pattern (Sagas). The idea is simple: each service does its part and notifies others about success. If something goes wrong, you run a compensating operation (rollback).

Saga workflow diagram:

+-------------------+             +-------------------+
| Order Service     |  -------->  | Payment Service   |
+-------------------+             +-------------------+
        |                               |
        |        [Success]              |       [Success]
        |                               |
+-------------------+          +-------------------+
| Inventory Service |  <-------| Notification Svc |
+-------------------+          +-------------------+

In practice:

1. Orchestration — one central component drives the process (for example, a Saga Orchestrator).
2. Choreography — services exchange events and react to others' success/failure.

Example implementation:

// Example of saga orchestration in Spring
public class OrderService {
    @Autowired
    private EventPublisher eventPublisher;

    public void createOrder(Order order) {
        eventPublisher.publishEvent(new OrderCreatedEvent(order));
    }

    @EventListener
    public void onPaymentSuccessful(PaymentCompletedEvent e) {
        eventPublisher.publishEvent(new InventoryReservedEvent(e.getOrderId()));
    }

    @EventListener
    public void onInventoryFailure(InventoryFailureEvent e) {
        // Compensation: cancel the order
        eventPublisher.publishEvent(new OrderCancelledEvent(e.getOrderId()));
    }
}

Another approach to consistency is the principle of eventual consistency (Eventual Consistency). It works like this: changes propagate through the system via events, and everything eventually converges to a consistent state (kind of "almost immediately").

You can use message brokers like Kafka to pass those events between microservices.

Monitoring complexity

If you thought monitoring a monolith is hard, listen up: in a system with dozens (or hundreds!) of microservices things get way messier. Figuring out exactly where it "hurts" can be a real puzzle.

Why is monitoring hard?

1. Lots of failure points. In a monolith you often have one database, one server. In a microservice system there can be hundreds of those points.
2. Distributed architecture. Services talk over the network, and debugging failures gets harder: "Is this a service bug or the network?".
3. Asynchrony. You can get processing delays and unexpected behavior.

The fix

Use tools like Spring Cloud Sleuth and Zipkin to trace request chains across services. They add a unique ID to each request and propagate it through the entire call chain.

Sleuth and Zipkin setup:

spring:
  sleuth:
    sampler:
      probability: 1.0
zipkin:
  base-url: http://localhost:9411

This lets you see which services are "slowing down" the chain and pinpoint bottlenecks.

Tools like Prometheus and Grafana help track metrics (e.g., throughput, memory usage) in real time. You can also add custom metrics via Spring Actuator.

Example custom metric:

@Component
public class CustomMetrics {
    private final Counter myCounter;

    public CustomMetrics(MeterRegistry registry) {
        this.myCounter = Counter.builder("custom.metric")
            .description("Example custom metric")
            .register(registry);
    }

    public void increment() {
        myCounter.increment();
    }
}

Fault tolerance in microservices

If you're building a distributed system, keep one life truth in mind: everything will fail. The network might drop, a server might hang, a database might stop answering. It's not "if" but "when". The goal is simple: minimize the blast radius.

Failure problems

1. Cascading failures. One service goes down, and the whole system follows.
2. Slow responses. If a service is "lagging", it can slow the entire chain.

The fix

The Circuit Breaker pattern prevents cascading failures. If a service stops responding properly, the Circuit Breaker opens and returns a predefined response (or nothing). After a short period it probes the service again.

Implementation with Resilience4j:

@Retry(name = "myService", fallbackMethod = "fallback")
public String callExternalService() {
    // Logic for calling an external service
}

public String fallback(Exception e) {
    return "Fallback response";
}

Another technique is Fallback: if a service is unavailable, return a cached response. Also, set timeouts for requests.

Spring Cloud Retry:

spring:
  cloud:
    loadbalancer:
      retry:
        enabled: true

In real life

Everything we've discussed today is crucial for real-world development. Data consistency issues show up when you want to save an order and a payment at the same time, and distributed tracing becomes your best friend when something breaks. Circuit Breaker is your savior from cascading failures, and Fallbacks and Retry help avoid catastrophic fallout.