Evaluation of Distributed Task Scheduler Design

Availability

The first component in our design was a rate limiter that is appropriately replicated and ensures availability. Task submission is done by several nodes. If a node that is submitting a task (send the task to a unique ID generator for storing task ID along with the task attributes into the database) fails, the other nodes take its place. The queue in which we push the task is also distributed in nature, ensuring availability. We always have resources available as we continuously monitor if we need to add or remove resources. Each component in the design is ...