Data Synchronisation

The need to store data in multiple places

There are cases where we need to store the same data in multiple places and potentially different forms. These are also referred to as materialized views. Below are some examples of such cases:

• Data that resides in a persistent datastore also needs to be cached in a separate in-memory store so that read operations can be processed from the cache with lower latency. Write operations need to update both the persistent datastore and the in-memory store.

• Data stored in a distributed key-value store must also be stored in a separate datastore that provides efficient full-text search, such as ElasticSearch or Solr. Depending on the form of read operations, the appropriate datastore can be used for optimal performance.

• Data stored in a relational database also needs to be stored in a graph database so that graph queries can be performed in a more efficient way.