Run async tasks application starts in ASP.NET Core

How to run async tasks application starts in ASP.NET Core? This article is a short follow-up to two of my previous articles about running asynchronous tasks in ASP.NET Core. In this article, I will provide some feedback on the example I used to demonstrate the problem and the approach I took. Previous posts can be found here and here.

How to run async tasks in ASP.NET Core?

How to run asynchronous tasks when the application starts in ASP.NET Core

Run async tasks application starts

The most common feedback on the post is the example I use to describe the problem. In my first article, I suggested three possible scenarios in which you might want to run tasks before the application starts:

• Check if your strongly typed configuration is valid.
• Start/fill the cache with data from the database or API
• Run the database migration before starting the application.

The first two options are fine, but several people have questions about running the database migration example. In both articles, I pointed out that it may not be a good idea to use database migration as the startup task of the application, but I still use it as an example anyway. In hindsight, this was a bad choice for me…

Database migration is a bad choice

So, what makes database migration a problem? After all, you absolutely need to migrate the database before the application starts processing requests! There seem to be three problems:

1. Only one process should run database migration
2. Migration usually requires more permissions than a typical web application should have
3. People are reluctant to run EF Core migration directly

I will discuss each one in turn below. Run async tasks application starts in ASP.NET Core

1. Only one process should run database migration

One of the most common ways to scale a web application is to run multiple instances on its scale and use a load balancer to distribute requests among them.

This “web farm” approach works well, especially when the application is stateless-requests are distributed among the various applications, and if one application crashes for some reason, other applications can still handle the request . Run async tasks application starts in ASP.NET Core

Unfortunately, if you try to run a database migration during the deployment of your application, you may encounter problems. If you start multiple instances of an application at approximately the same time, you can start multiple database migration tasks at once . There is no guarantee that this will cause you problems, but unless you are very careful to ensure idempotent updates and error handling, you are likely to get stuck.

You really don’t want to deal with the kind of database integrity issues that might be caused by using this method. A better option is to run a single migration process before (or after, depending on your requirements) deploying the web application . Using a single process to perform database migration, you can naturally avoid the most serious dangers. Run async tasks application starts in ASP.NET Core.

Compared to running the database migration as a startup task, this method seems to require more work, but is safer and easier to implement. Run async tasks application starts in ASP.NET Core.

Of course, this is not the only option. If you are firm about starting a task migration, you can use some kind of distributed lock to ensure that only one application instance can run its migration at the same time. But this does not solve the second problem…

2. Migration usually requires more permissions than a typical web application should have

The best practice is to restrict your application so that it only has permission to access and modify the resources it needs. If the reporting application only needs to read sales data, it cannot modify them or change the database schema! If your application has security issues, locking down the permissions associated with a given connection string can prevent great repercussions. Run async tasks application starts in ASP.NET Core.

If you use the web application itself to run a database migration, the web application naturally requires database permissions to perform high-risk activities, such as modifying the database schema, changing permissions, or updating/deleting data. Do you really want the web application to be able to delete the studentstable (thanks to XKCD)?

Similarly, you can perform some implementation-specific operations, such as using a different connection string for migration compared to normal database access. But fundamentally, you do not like to use external locking migration process as a Web application program process.

3. People are reluctant to run EF Core migration directly

This is not obvious, but many people say that using EF Core migration tools in production may not be a good idea. Run async tasks application starts in ASP.NET Core

I think your article should clarify that running EF migration in production is not a good idea. The migration framework is a good idea to run at startup. 😊— Khalid 👨‍🚀 (@buhakmeh)

January 9, 2019

Personally, I haven’t tried to use EF Core migration in production for more than a year, and the tools have undoubtedly improved since then. Having said that, I still see some problems:

• When using the EF Core global tool for migration, you need to install the .NET Core SDK, and you don’t need this tool on the production server. You certainly don’t have to use it (for example, I didn’t use it in my post).
• The last time I tried to run other SQL scripts and scripts generated by EF Core was tricky, and EF Core would feel uneasy. However, this situation can be improved well now, otherwise I might have done something wrong before!
• If you want to update the database safely , you may have to make some edits to the generated script anyway. Migrating the database schema should be compatible with existing (running) applications, in order to avoid downtime.
• The Microsoft documentation itself hints that running EF Core migration at application startup is not a good idea!

I personally did not use EF Core migration, but used DbUp and FluentMigrator, and found that they all work normally.

So, if the database migration task is not ideal for the application startup task example, what is it? Run async tasks application starts in ASP.NET Core

Better start task example

I mentioned two other examples in the post, but describe them again below. There are some interesting additions (provided by Ruben Bartelink): Run async tasks application starts in ASP.NET Core.

• Check if your strongly typed configuration is valid . ASP.NET Core 2.2 introduced Options validation, but IOptions<T>this is only performed when the class is accessed for the first time . As mentioned in the previous article, you may want to perform eager verification at application startup to confirm that your environment and configuration are valid.
• Fill the cache . Your application may require data from the file system or remote service. This data only needs to be loaded once, but it is relatively expensive to load. Loading this data before the application starts can avoid the request burdening the delay.
• Pre-connect to databases and/or external services . In a similar way, you can populate the database connection pool by connecting to a database or other external service. These are usually relatively expensive operations, so the use cases are also large.
• Pre-JIT and load assembly by instantiating all singletons in the application . I think this is a very interesting idea-by loading all the Singletons registered in the DI container when the application starts, it is possible to potentially avoid a lot of expensive JIT compilation and assembly loading in the context of the request.

As with all these examples, the benefits you will see will largely depend on the characteristics of the particular application. That is to say, the last suggestion is to pre-instantiate all singletons in the application, which particularly aroused my interest, so I will introduce how to achieve this goal in a future article. Run async tasks application starts in ASP.NET Core

Alternative methods of using health checks

I totally agree with the feedback regarding the database migration. When this is not a good idea, using them as examples of starting tasks can be misleading, especially because I personally don’t use the methods I describe! Run async tasks application starts in ASP.NET Core

However, many people agree with the overall approach I described . In the start Kestrel server before running the task and start processing the request. Run async tasks application starts in ASP.NET Core

Damian Hickey is an exception, he suggested that he prefer to start Kestrel as soon as possible. He recommends using the health check endpoint to signal the load balancer that the application is ready to start receiving requests after all startup tasks are completed. During this period, all non-health check traffic (if the load balancer is doing this, there should be no traffic) will be received 503 Service Unavailable.

The main advantage of this method is to avoid network timeouts. Generally speaking, it is better to request an error code to return quickly than not responding at all and causing the client to time out. This reduces the amount of resources required by the client. By starting Kestrel earlier, the application can start responding to requests earlier even if the response is a “not ready” response.

In fact, this is very similar to the method described in my first article, but because it is too complicated to achieve the goal I set, I discard it. Technically, in that article, I specifically studied in Kestrel start before the method to run the task, and health check method can not.

However, Damian particularly likes to run Kestrel first, and this fact makes me reconsider. I can’t think of any actual task examples or other reasons why the task needs to be completed before Kestrel starts. I can only think of tasks that need to be run before regular traffic reaches the application. Both methods can be implemented.

Therefore, in my next article, I will show an example of how to take this approach and use ASP.NET Core 2.2 health checks to signal the load balancer after the application has completed all startup tasks. Run async tasks application starts in ASP.NET Core

Conclusion

In this article, I shared the feedback from my previous article about running asynchronous tasks when the application starts. The biggest problem is that I chose to use EF Core database migration as an example to start the task. Database migrations are not suitable for running when the application starts, because they usually need to be run by a single process and require more database permissions than ordinary web applications.

I have provided some better examples of starting tasks and suggested another method that will start Kestrel as soon as possible and use health checks to notify when the task is complete. I will show an example in the next article. Run async tasks application starts in ASP.NET Core