Using StreamRefs when the Source and Sink are not both part of the cluster
StreamRefs are thought to be used within a cluster. But at least in theory, StreamRefs should work over plain Akka Remoting as well. That stated, we don't recommend using Cluster Client (or any remoting from outside the cluster) any more because it has several drawbacks:
- it's less stable than having all nodes in the same cluster and you can more easily get into situations where nodes have to be restarted because of quarantining when messages are lost (e.g. because of intermittent network problems)
- it introduces tight coupling between the external actor system and the cluster because you need to make sure that you run exactly the same versions of akka / your application, so that serialization is compatible between all parts of your system
Therefore, we now recommend using either Akka gRPC or Akka HTTP to interface with a cluster. Both encourage specifying well-defined interfaces that are easier to keep compatible between versions. And also, both, gRPC and HTTP, allow streaming use cases.
Related Articles
How to do Rate Limiting in Akka Streams
In certain scenarios it is important to limit the number of concurrent requests to other services. For example, to avoid overwhelming the services and avoid performance degradation, or to maintain service level agreements. This is particularly ...
Error handling and recovery in Akka Streams
When developing applications you should assume that there will be unexpected issues. For this, Akka provides a set of supervision strategies to deal with errors within your actors. Akka streams is no different, in fact its error handling strategies ...
How to implement batching logic in Akka Streams
A common request we see with streaming data is the need to take the stream of elements and group them together (i.e. committing data to a database, a message queue or disk). Batching is usually a more efficient and performant solution than writing a ...
How to do Throttling in Akka Streams
When building a streaming application you may find the need to throttle the upstream so as to avoid exceeding a specified rate. Akka Stream's provides the capability to either fail the stream or shape it by applying back pressure. This is simply done ...
How to stream multiple stream actions concurrently
To construct efficient, scalable and low-latency data streams, it is often important to perform tasks concurrently. However Akka Streams executes stages sequentially on a single thread, so you must explicitly request this concurrency. Inserting these ...