4

I am currently working on an architecture see below. First I'm not sure if this kind of architecture is called an event-driven or a data-driven architecture or maybe both.

There some input messages are sent from the Frontend to T1. These messages are first validated, then collected and in the end evaluated.

My current approach is to persist the raw messages with all meta information in MS A, the sorted collections in MS B and the evaluations in MS C. This separates the data to the appropriately concerned microservices.

In T2 I only produce the messages which MS B requires.
In T3 I only produce the messages which MS C requires.
But when evaluation the collections all meta information from MS A is required. So how to proceed with this kind?

  1. Should I send only the minimum of data to the queue and provide an API?
  2. Should I send all data to the queues (forward data for following services)?
  3. Should I send all information for the next service to the queue and provide an API?
  4. Something else?

Or did I misunderstand the approach "Communicating microservices through Kafka"?

Please feel free to offer criticism!
Thanks for advice!

2
  • Does this help answer your question? confluent.io/blog/… Jan 11, 2018 at 7:22
  • @RobinMoffatt yes! In my opinion, this is a big question, therefore, I had to think about it some days. So excuse me for this delay. So as I understood it right. I use kafka for a global event triggering which keeps scaling in focus and handles this pretty. And the kafka consumers fetch the needed data via a API call and publish the results to another queue. Jan 17, 2018 at 15:44

1 Answer 1

3

I believe that this is a message-driven and a data-driven architecture, but this should not be important. What it is more important is that the microservices use Choreography (as opposed to Orchestration). This question could help.

The cleanest architecture would be to put all the data in the messages, in this way the number dependencies is limited to 2. Also, the resilience of the system is increased: if the microservice A is down, the other downstream microservices could continue to work.

Every microservice consume only the part of the message that interests it and ignores the other. This creates a nice and extendable pipeline of stream-like processing. If, however, the message is too big, you should use the Microservice A (or any other microservice) as the reference for more data.

6
  • The last phrase I don't understand right. So if the messages are too big (yes they are) then I should use an API call or what do you mean? Jan 17, 2018 at 15:45
  • @DanielEisenreich I mean that the event only references the data that is hold somewhere else. For example in a distributed file system. This data can be accesible by an API exposed by Microservice A. This is for the case when the events are too big to be moved around, i.e. > 1 GB. Jan 17, 2018 at 15:50
  • I understand. And would you prefer to put the business logic and the API into one microservice? Jan 17, 2018 at 15:54
  • @DanielEisenreich quite the opposite. The API in ms A is only for events that are too big (I shouldn't have added the last paragraph as it confuses you) Jan 17, 2018 at 16:05
  • So if the data is not too big, then you recommend the 2. solution of my post, right? Forward all data for the next services and avoid API calls? Jan 17, 2018 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.