Streaming Data
We'll cover the following...
A common pattern with any HTTP API is the need to receive events. In a lot of cases, there is no way to achieve this other than regularly polling the API. That can cause a lot of stress on the HTTP endpoint, as it requires setting up a new connection, which means a lot of overload with TCP and SSL.
Streaming basics
A more efficient way is to use streaming. An adequate technology here is ServerSent Events message protocol defined by HTML5. Alternatively, it would be possible to use Transfer-Encoding:chunked defined by HTTP/1.1 or even the WebSocket protocol. However, chunked encoding is more complicated, and the WebSocket is a little bit overkill for the simple use case presented here.
To implement any streaming mechanism in a scalable and efficient manner, you need to be sure that your backend offers that feature. It could be a messaging queue, a database or any other software that provides a stream of events that the application can subscribe to.
If the offered API has to poll its backend regularly to know about new events, it is simply moving the problem from one ...