Gain insights into real-world microservice implementation, exploring tech stacks, Kafka, Docker, Kubernetes, and more. Learn about synchronous and asynchronous integrations with hands-on coding demonstrations.

docker_ms2_v3.tar.gz

scs-esi

kafka

atom

links

netflix

consul

atom-kafka-scs-netflix-up

links-up

Microservices are one of the most important software architecture trends, but it’s one thing to define an architecture and quite another to implement it. 

This course focuses on the nitty-gritty details of real-world implementation. You’ll learn recipes for tech stacks that can be used to implement microservices, as well as the pros and cons of each. 

You’ll start by exploring some fundamental concepts for implementing microservices. Within each concept, you’ll learn about the different technologies used to implement it. The technologies include: Frontend Integration with Edge Side Includes (ESI), asynchronous microservices with Kafka and REST feeds, synchronous microservices with the Netflix stack and Consul, Docker, Kubernetes, Cloud Foundry. Each technology you learn about is described and then demonstrated with real code. 

By the end of this course, you’ll be a microservice pro. Whether you’re a software engineer or an engineering manager, this course will prove useful throughout your career.

Microservice Architecture: Practical Implementation

# Consumer groups

An event like `neworder42` should probably be processed only once by one instance of the invoicing microservice. Only one instance of a microservice should receive it, ensuring that only one invoice is written for this order. However, another instance of a microservice might work on `neworder21` in parallel.

Consumers are organized in consumer groups where each partition sends records to exactly one consumer in the consumer group.  One consumer can be responsible for several partitions.




Thus, a consumer receives the messages of one or multiple partitions. 
# Example

The drawing above shows an example. Consumer 1 receives the messages of partitions 1 and 2 and consumer 2 receives the messages of partition 3.


The invoicing microservice instances could be organized in a consumer group, ensuring that only one instance of the invoicing microservice processes each record.


When a consumer receives a message from a partition, it will also later receive all messages from the same partition. The order of messages per partition is also preserved meaning that records in different partitions can be handled in parallel, and at the same time the sequence of records in a single partition is guaranteed.

Therefore, the instance of the invoicing microservice that receives `neworder42` would also receive `updated42` if those records are sent to the same partition. So, the instance would be responsible for all events about the order 42.

Of course, this applies only if the mapping of consumers to partitions remains stable. For example, if new consumers are added to the consumer group for scaling, the mapping can change.

The new consumer would need to handle at least one partition that was previously handled by a different consumer.


The maximum number of consumers in a consumer group is equal to the number of partitions, because each consumer must be responsible for at least one partition. Ideally, there are more partitions than consumers so that we can add more consumers when scaling.


Consumers are always members of a consumer group. Therefore, they receive records sent only to their partitions. If each consumer is to receive all records from all partitions, then there must be a separate consumer group for each consumer with only one member.


In this lesson, we'll continue learning about the architecture of Kafka.

Preface

Self-contained Systems

Concept: Frontend Integration

Recipe: Links and Client-side Integration

Recipe: Server-side Integration using Edge Side Includes (ESI)

Concept: Asynchronous Microservices

Recipe: Messaging and Kafka

Recipe: Asynchronous Communication with Atom and REST

Concept: Synchronous Microservices

Recipe: REST with the Netflix Stack

Recipe: REST with Consul and Apache httpd

Concept: Microservices Platforms

Recipe: Docker Containers with Kubernetes

Recipe: PaaS with Cloud Foundry

Appendix

Even More on The Architecture of Kafka

Consumer groups #