Using Serverless Deployments with Pull Requests

So far, all the changes we made were pushed to a branch. That was on purpose since one of my goals was to show you the benefits of using serverless deployments with pull requests.

Benefits of using serverless deployments with pull requests

The percentage of the apps running as serverless inevitably varies. Some might have all the stateless applications running as serverless, while others might have none. Between those two extremes can be all shades of gray. While I cannot tell you how many apps you should run as serverless deployments, what I can tell you with a high level of certainty is that you’ll use Knative much more in temporary environments like those created for pull requests than in permanent ones like staging and production. The reason for such a bold statement lies in the differences in purpose for running applications.

Resource usage in preview environments

An application running in a preview environment is to validate (automatically and/or manually) that a change we’d like to merge to the master branch is a good one and that it is relatively safe to merge it with production code. However, the validations we’re running against a release in a preview environment are often not the final ones. A preview environment is often not the same as production.

• It might differ in size.
• It might not be integrated with all the other applications.
• It might not have all the data we need, and so on and so forth.

That’s why we have the staging and other non-production permanent environments. Their primary purpose is often to provide a production-like environment where we can run the final set of validations before promoting to production. If we’d do that for each preview environment, our resource usage would go through the roof.

Imagine having hundreds or even thousands of open pull requests and that each is deployed to a preview environment. Now, imagine that each pull request is a full-blown system. How much CPU and memory would that require?

For that reason, our preview environments used with pull requests usually contain only the application in question. We might add one or two essential applications it integrates with, but we seldom deploy the full system there. Instead, we use mocks and stubs to test applications in those environments. That, by itself, should save a lot of resources, while still maintaining the advantages of preview environments. Nevertheless, we can do better than that.

Preview environments are one of the best use-cases for serverless loads. We might choose to run our application as serverless in production, or we might decide not to do so. The decision will depend on many factors, user experience being one of the most important ones. If our application scales to zero replicas due to lack of traffic when a request does come, it might take a while until the process in the newly spun replica is fully initialized. It should be obvious why forcing our users to wait for more than a second or two before receiving the first response is a bad idea. They are impatient and are likely to go somewhere else if they are unhappy with us. Also, our production traffic is likely going to be more demanding and less volatile than the one exercised over deployments in preview environments.

When a preview environment is created as a result of creating a pull request, our application is deployed to a unique Namespace. Typically, we would run some automated tests right after the deployment. But what happens after that? The answer is “mostly nothing”. A while later (a minute, an hour, a day, or even a week), someone might open the deployment associated with the pull request and do some manual validations. Those might result in the need for new issues or validations. As a result, that pull request is likely going to be idle for a while, before it is used again. So, we have mostly unused deployments wasting our memory and CPU.

Deployments initiated by pull requests

Given that we established how the preview environments represent a colossal waste in resources, we can easily conclude that deployments initiated by pull requests are one of the best candidates for serverless computing.

But, that would also assume that we do not mind waiting until an application is scaled from zero to one or more replicas. In my experience, that is rarely a problem. We cannot put users of our production releases and pull request reviewers on the same level. It should not be a problem if a person who decides to review a pull request and validate the release candidate manually has to wait for a second or even a minute until the application is scaled up. That loss in time is more than justified with better usage of resources.

Before and after the review, our app would use no resources unless it has dependencies (e.g., database) that cannot be converted into serverless deployments. We can gain a lot, even in those cases, when only some of the deployments are serverless. A hundred percent of deployments in preview environments running as serverless is better than fifty percent. Still, fifty percent is better than nothing.

🔍 Databases in preview environments

Databases in preview or even in permanent environments can be serverless as well. As long as their state is safely stored in a network drive, that should be able to continue operating when scaled from zero to one or more replicas. Nevertheless, databases tend to be slow to boot, especially when having a lot of data. Even though they could be serverless, they are probably not the right candidate. The fact that we can do something does not mean that we should.

Creating a pull request

Now, let’s create a pull request and see how the jx-knative behaves.