.NET: Tools for working with multithreading and asynchrony. Part 1

.NET: Tools for working with multithreading and asynchrony. Part 1


Publish to Habr the original article, the translation of which is posted on the blog Codingsight .

The need to do something asynchronously, without waiting for the result here and now, or to divide a lot of work between several units performing it was before the advent of computers. With their appearance, such a need has become very tangible. Now, in 2019, typing this article on a laptop with an 8 core Intel Core processor, on which not one hundred processes work in parallel, but threads and more. Nearby, there is already a bit shabby, a phone bought a couple of years ago, he has an 8 core processor on board. The thematic resources are full of articles and videos where their authors admire the flagship smartphones of this year where they put 16-core processors. MS Azure provides in less than $ 20/hour a virtual machine with a 128 core processor and 2 TB RAM. Unfortunately, it is impossible to extract the maximum and curb this power without knowing how to manage the interaction of flows.

Terminology


Process (Process) - an OS object, isolated address space, contains threads.
Thread - the OS object, the smallest unit of execution, part of the process, the threads share memory and other resources among themselves within the framework of the process.
Multitasking - the property of the OS, the ability to perform multiple processes at the same time
Multi-core - a feature of the processor, the ability to use multiple cores to process data
Multiprocessing is a property of a computer, the ability to simultaneously work with several processors physically
Multi-threading is a property of the process, the ability to distribute data processing across multiple threads.
Parallelism - performing several actions physically at the same time per unit of time
Asynchrony - performing an operation without waiting for the completion of this processing, the result of the execution can be processed later.

Metaphor


Not all definitions are good and some need additional explanation, so I’ll add a breakfast metaphor to the formally introduced terminology. Cooking breakfast in this metaphor - process.

Preparing breakfast in the morning I ( CPU ) come into the kitchen ( Computer ). I have 2 hands ( Cores ). The kitchen has a number of devices ( IO ): oven, kettle, toaster, refrigerator. I turn on the gas, put a griddle on it and pour oil there, without waiting until it warms up ( asynchronously, Non-Blocking-IO-Wait ), I take the eggs out of the fridge and break them into a plate, after which I beat it with one hand ( Thread # 1 ), and with the other ( Thread # 2 ) I hold the plate (Shared Resource). Now I would like to turn on the kettle, but I don’t have enough hands ( Thread Starvation ). During this time, the frying pan is heated (processing the result) where I poured what I whipped. I reach for the teapot and turn it on and stupidly watch how the water in it boils ( Blocking-IO-Wait ), although I could have washed the plate where I was beating the omelet during this time.

I cooked an omelet using only 2 hands, and I don’t have it anymore, but at the same time, at the moment of beating the omelet, 3 operations occurred at once: beating the omelet, holding the plate, warming the pan. CPU is the fastest part of the computer, IO is something that It only slows down, because it is often an effective solution to occupy something with a CPU while data is being received from IO.

Continuing the metaphor:

  • If in the process of cooking an omelet, I would also try to change clothes, it would be an example of multitasking. An important caveat: computers with this are much better than people.
  • A kitchen with several cooks, for example, in a restaurant is a multi-core computer.
  • Many food court restaurants in the mall - data center

Tools.NET


In working with streams, as in many other things, .NET is good. With each new version, it presents more and more new tools for working with them, new layers of abstraction on OS threads. In working with the construction of abstractions, the framework developers use an approach that leaves it possible to use a high-level abstraction, going down one or more levels below. Most often this is not necessary, moreover it opens up the opportunity to shoot yourself in the foot of a shotgun, but sometimes, in rare cases, it may be the only way to solve a problem that does not solve at the current level of abstraction.

By tools, I mean both the software interfaces (APIs) provided by the framework and third-party packages, and the whole software solutions that simplify the search for any problems related to the multithreaded code.

Starting a Stream


Thread class, the most basic in .NET for working with threads. The constructor accepts one of two delegates:

  • ThreadStart - No Parameters
  • ParametrizedThreadStart - with one parameter of type object.

The delegate will be executed in the newly created thread after calling the Start method, if a delegate of the type ParametrizedThreadStart was passed to the constructor, then an object must be passed to the Start method. This mechanism is needed to transfer any local information to the stream. It is worth noting that creating a thread is an expensive operation, and the thread itself is a heavy object, at least because 1MB of memory is allocated to the stack, and it requires interaction with the OS API.

  new Thread (...). Start (...);
  

The ThreadPool class represents the concept of a pool. In .NET, the thread pool is a work of engineering and Microsoft developers have put a lot of effort into working optimally in a variety of scenarios.

General concept:

From the moment of launch, the application in the background creates several streams for emergency and provides an opportunity to take them for use. If threads are used frequently and in large quantities, then the pool expands to meet the need of the calling code. When there are no free threads in the pool at the right time, it will either wait for the return of one of the threads, or it will create a new one. From this it follows that the thread pool is great for some short actions and is bad for operations running as a service throughout the entire operation of applications.

To use a stream from the pool, there is a QueueUserWorkItem method that accepts a WaitCallback delegate, which is the same as ParametrizedThreadStart, and the parameter passed to it performs the same function.


  ThreadPool.QueueUserWorkItem (...);
  

The less well-known method of the pool of threads RegisterWaitForSingleObject serves to organize non-blocking IO operations. The delegate passed to this method will be called when the WaitHandle passed to the method is “released” (Released).

  ThreadPool.RegisterWaitForSingleObject (...)
  

In .NET there is a stream timer and it differs from the WinForms/WPF timers in that its handler will be called in a stream taken from the pool.

  System.Threading.Timer
  

There is also a rather exotic way to send a delegate to a stream from a pool — the BeginInvoke method.

  DelegateInstance.BeginInvoke
  

I also want to casually dwell on the function, to the call of which many of the above methods are reduced - CreateThread from Kernel32.dll Win32 API. There is a way, thanks to the mechanism of extern methods to call this function. I saw such a call only once in the worst example of legacy code, and the motivation of the author who did this is still a mystery to me.

  Kernel32.dll CreateThread
  

View and debug threads


Created by you personally, by all third party components and the .NET pool of threads, you can view it in the Threads window of Visual Studio. This window will display information about streams only when the application is under debugging and in break mode. Here you can conveniently view the stack names and priorities of each thread, switch debugging to a specific thread. The Priority property of the Thread class can set the priority of a thread, which the OC and CLR will perceive as a recommendation when splitting processor time between threads.



Task Parallel Library


Task Parallel Library (TPL) appeared in .NET 4.0. Now it is the standard and the main tool for working with asynchrony. Any code using an older approach is considered legacy. The basic unit of the TPL is the Task class from the System.Threading.Tasks namespace. Task is an abstraction over the stream. With the new version of the C # language, we have an elegant way of working with Task `s - async/await operators. These concepts allowed to write asynchronous code as if it were simple and synchronous, it enabled even people with little understanding of the internal kitchen of threads to write applications using them, applications that do not hang when performing long operations. Using async/await theme for one or even several articles, but I will try to put the essence in several sentences:

  • async is a modifier of the method that returns Task or void
  • and await, Task’s non-blocking operator.

Once again: the await operator, in general (there are exceptions), releases the current execution thread further, and when Task finishes its execution, and the flow (in fact, it is more correct to say the context, but more on that later) will continue to continue the method execution. Inside .NET, this mechanism is implemented as well as the yield return, when the written method turns into a whole class, which is a state machine and can be executed in separate chunks depending on these states. Anyone interested can write any simple code using asynс/await, compile and view the assembly using JetBrains dotPeek with the Compiler Generated Code enabled.

Consider options for starting and using Task’s. In the example code below, we create a new task that does nothing useful ( Thread.Sleep (10000) ), but in real life it must be some kind of complex CPU-working work.

  using TCO = System.Threading.Tasks.TaskCreationOptions;

 public static async void VoidAsyncMethod () {
  var cancellationSource = new CancellationTokenSource ();

  await Task.Factory.StartNew (
//Code of action
  () = & gt;  Thread.Sleep (10000),
  cancellationSource.Token,
  TCO.LongRunning |  TCO.AttachedToParent |  TCO.PreferFairness,
  scheduler
  );

//Code after await
 }
  

Task is created with a number of options:

  • LongRunning - a hint that the task will not be completed quickly, which means that it may be worth considering not to take a stream from the pool, but create a separate task for this task so as not to harm the rest.
  • AttachedToParent - Tasks can be arranged in a hierarchy. If this option has been used, then Task can be in a state when it has run and is waiting for the children to run.
  • PreferFairness - means that it would be good to execute the Task and sent for execution earlier before those that were sent later. But this is just a recommendation and the result is not guaranteed.

The second parameter to the method is the CancellationToken. To correctly handle the cancellation of an operation after it is started, the executable code must be filled with checks of the CancellationToken state.If there are no checks, the Cancel method called on the CancellationTokenSource object will be able to stop the Task’s execution only before it is started.

The last parameter is the TaskScheduler type scheduler object. This class and its descendants are designed to manage the strategies for allocating Task'ov to threads; by default, Task will be executed on a random thread from the pool.

The await operator is applied to the created Task, which means the code written after it, if such is, will be executed in the same context (often this means that on the same thread) as the code before await.

The method is marked as async void, which means that it can use the await operator, but the calling code cannot wait for execution. If such an opportunity is necessary, the method should return Task. Methods labeled async void are quite common: as a rule, these are event handlers or other methods that work on the principle of execution and forgetting (fire and forget). If it is necessary not only to give the opportunity to wait for the completion of the execution, but also to return the result, then it is necessary to use the Task.

On the Task’s that I returned to the StartNew method, however, like on any other, you can call the ConfigureAwait method with the false parameter, then the execution after await will continue not on the captured context, but on an arbitrary one. This must be done always when the execution context is not critical for the code after await. It is also a recommendation from MS when writing code that will be delivered packaged in a library.

Let's dwell on how you can wait for the completion of the task. Below is a sample code, with comments, when the wait is done conditionally well and when it is conditionally bad.

  public static async void AnotherMethod () {

  int result = await AsyncMethod ();//good

  result = AsyncMethod (). Result;//bad

  AsyncMethod (). Wait ();//bad

  IEnumerable & lt; Task & gt;  tasks = new Task [] {
  AsyncMethod (), OtherAsyncMethod ()
  };

  await Task.WhenAll (tasks);//good
  await Task.WhenAny (tasks);//good

  Task.WaitAll (tasks.ToArray ());//bad
 }
  

In the first example, we wait for the Task to be executed without blocking the calling thread, we will return to the processing of the result only when it already is, until the calling thread is left to itself.

In the second variant, we block the calling thread until the result of the method is calculated. This is bad not only because we occupied the thread, such a valuable resource of the program, as simple idleness, but also because if in the method code that we call there is await, and the synchronization context implies a return to the calling thread after await, then we will get deadlock: the calling thread waits for the result of the asynchronous method to be calculated, the asynchronous method tries in vain to continue its execution in the calling thread.

Another disadvantage of this approach is sophisticated error handling. The fact is that errors in asynchronous code when using async/await are very easy to handle - they behave as if the code were synchronous. While, if we apply exorcism synchronous waiting to the Task’s, the original exception turns into an AggregateException, so To handle an exception, you will have to examine the InnerException type and write the if chain yourself within a single catch block or use the catch when construct instead of the more familiar catch block chain in C #.

The third and last examples are also marked bad for the same reason and contain all the same problems.

The WhenAny and WhenAll methods are extremely convenient while waiting for a group of Task'es; they wrap a group of Task'es into one, which will work either when the Task is first activated from the group, or when everything is finished.

Stop Flows


For various reasons, it may be necessary to stop the flow after it starts. For this there are a number of ways.The Thread class has two methods with the appropriate names, Abort and Interrupt . The first is not recommended for use, because after calling it at any random moment, during the processing of any instruction, a ThreadAbortedException exception will be thrown. You do not expect that such an exception will take off with an increment of any integer variable, right? And when using this method, this is a very real situation. If it is necessary to prohibit the CLR from generating such an exception in a specific section of code, you can wrap it in Thread.BeginCriticalRegion , Thread.EndCriticalRegion calls. Any code written in the finally block is wrapped up with such calls. For this reason, in the depths of the framework code, you can find blocks with an empty try, but not a finally finally try. Microsoft does not recommend using this method so much that it is not included in the .net core.

The Interrupt method works more predictably. It can interrupt the thread with the exception of ThreadInterruptedException only when the thread is idle. It goes into such a state by waiting for WaitHandle, lock, or after calling Thread.Sleep.

Both of the above options are bad for their unpredictability. The solution is to use the CancellationToken structure and the CancellationTokenSource class. The point is this: an instance of the CancellationTokenSource class is created, and only those who own it can stop the operation by calling the Cancel method. Only the CancellationToken is passed to the operation itself. The owners of the CancellationToken cannot cancel the operation themselves, but can only check if the operation was canceled. There is a Boolean property IsCancellationRequested and a method ThrowIfCancelRequested . The latter will throw a TaskCancelledException exception if the Cancel method has been called on the CancellationToken instance of the CancellationTokenSource. And this is the method I recommend to use. This is better than the previous options for obtaining full control over when the exception operation can be interrupted.

The cruelest option to stop the thread is to call the Win32 API function TerminateThread. The behavior of the CLR after calling this function can be unpredictable. On MSDN, the following is written about this function: “TerminateThread is the most extreme cases. “

Transform legacy-API to Task Based with FromAsync method


If you are lucky enough to work on a project that was started after the tasks were entered and stopped the silent horror of most developers, then you will not have to deal with a large number of old APIs, both third-party and extorted by your team in the past. Fortunately, the .NET Framework development team took care of us, although perhaps the goal was to take care of ourselves. Be that as it may, in .NET there are a number of tools for painlessly converting code written in the old approaches of asynchronous programming into a new one. One of them is the FromAsync method of the TaskFactory. In the example code below, I wrap the old asynchronous methods of the WebRequest class in Task using this method.

  object state = null;
 WebRequest wr = WebRequest.CreateHttp ("http://github.com");
 await Task.Factory.FromAsync (
  wr.BeginGetResponse,
  we.EndGetResponse
 );
  

This is just an example and you hardly have to do this with the built-in types, but any old project is simply teeming with BeginDoSomething methods that return IAsyncResult and EndDoSthingthing methods to its host.

Transform legacy-API to Task Based using the TaskCompletionSource class


Another important tool to consider is the TaskCompletionSource class. In terms of functions, purpose and principle of operation, it can somehow recall the RegisterWaitForSingleObject method of the ThreadPool class about which I wrote above.Using this class, you can easily and conveniently wrap old asynchronous APIs in a Task.

You will say that I have already spoken about the TaskAsact method of the TaskFactory class intended for this purpose. Here we will have to recall the entire history of the development of asynchronous models in the .net that microsoft offered for the last 15 years: prior to the Task-Based Asynchronous Pattern (TAP), Asynchronous Programming Pattern (APP) existed, which was about the Begin DoSomething return methods IAsyncResult and the End DoSomething methods that accept it and for the legacy of these years the FromAsync method is perfect, but over time, it is replaced by the Event Based Asynchronous Pattern ( EAP) ), which assumed that an event would be triggered upon completion of an asynchronous operation.

TaskCompletionSource is great for wrapping in Task's and legacy-APIs built around an event model. The essence of his work is as follows: an object of this class has a public property of type Task whose state can be managed through the methods SetResult, SetException, etc. of the TaskCompletionSource Class. In the places where the await operator was applied to this Task, it will be executed or crashed with an exception depending on the method applied to the TaskCompletionSource. If everything is still not clear, then let's look at this sample code, where some old EAP-time API is wrapped in a Task using TaskCompletionSource: when the Task event fires, the Task will be transferred to the Completed state, and the await operator will resume its execution getting the object result .

  public static Task & lt; Result & gt;  DoAsync (this SomeApiInstance someApiObj) {

  var completionSource = new TaskCompletionSource & lt; Result & gt; ();
  someApiObj.Done + =
  result = & gt;  completionSource.SetResult (result);
  someApiObj.Do ();

  result completionSource.Task;
 }
  

TaskCompletionSource Tips & amp; Tricks


Wrapping old APIs is not all that can be done with TaskCompletionSource. The use of this class opens up an interesting possibility of designing various APIs, on Task'ah, that threads do not occupy. And the flow, as we remember, is an expensive resource and their number is limited (mainly by the amount of RAM). This limitation is easily achieved by developing, for example, a loaded web application with complex business logic. Consider the possibilities that I am talking about implementing a trick like Long-Polling.

Briefly, the essence of the trick is this: you need to receive information from the API about some events occurring on its side, while the API for some reason cannot report the event, and can only return a state. An example of such is all APIs built on top of HTTP until the time of WebSocket or, if for some reason it is impossible to use this technology. The client may ask the HTTP server. HTTP server itself can not provoke communication with the client. A simple solution is to poll the server on a timer, but this creates an additional load on the server and an additional average delay TimerInterval/2. To circumvent this, a trick called Long Polling was invented, which implies a delay in response from the server until Timeout or an event will occur. If an event has occurred, then it is processed; if not, the request is sent again.

  while (! eventOccures & amp; & amp;! timeoutExceeded) {

  CheckTimout ();
  CheckEvent ();
  Thread.Sleep (1);
 }
  

But such a decision will show itself terribly, as soon as the number of customers waiting for an event grows, because Each such client is waiting for the event takes a whole stream. Yes, and we get an additional 1ms delay on triggering an event, most often it is not significant, but why make the software worse than it can be? If we remove Thread.Sleep (1), then we’ll load one processor core 100% idle rotating in a useless cycle.With TaskCompletionSource, you can easily remake this code and solve all the problems identified above:

  class LongPollingApi {

  private Dictionary & lt; int, TaskCompletionSource & lt; Msg & gt; & gt;  tasks;

  public async Task & lt; Msg & gt;  AcceptMessageAsync (int userId, int duration) {

  var cs = new TaskCompletionSource & lt; Msg & gt; ();
  tasks [userId] = cs;
  await Task.WhenAny (Task.Delay (duration), cs.Task);
  return cs.Task.IsCompleted?  cs.Task.Result: null;
  }

  public void SendMessage (int userId, Msg m) {

  if (tasks.TryGetValue (userId, out var completionSource))
  completionSource.SetResult (m);
  }
 }
  

This code is not production-ready, but merely demonstration. For use in real cases, you still need to, as a minimum, handle the situation when the message came at a time when no one expects it: in this case, the AsseptMessageAsync method should return the already completed Task. If this case is the most frequent, then you can think about using ValueTask.

When we receive a request for a message, we create and put it in the TaskCompletionSource dictionary, and then we wait for what happens first: the specified time interval expires or a message is received.

ValueTask: why and how


The async/await statements, like the yield return statement, generate a state machine from a method, and this is the creation of a new object, which is almost always not important, but in rare cases it can create a problem. This case can be a method called really often, talking about tens and hundreds of thousands of calls per second. If such a method is written in such a way that in most cases it returns a result bypassing all await methods, then .NET provides a tool to optimize it - the ValueTask structure. To make it clear, consider an example of its use: there is a cache in which we go very often. There are some values ​​in it and then we simply return them, if not, then we go to some slow IO behind them. Last I want to do asynchronously, which means the whole method is obtained asynchronous. So the obvious way to write a method is the following:

  public async Task & lt; string & gt;  GetById (int id) {

  if (cache.TryGetValue (id, out string val))
  return val;
  return await RequestById (id);
 }
  

Because of the desire to optimize a little, and a slight phobia about what Roslyn generates when compiling this code, you can rewrite this example as follows:

  public Task & lt; string & gt;  GetById (int id) {

  if (cache.TryGetValue (id, out string val))
  return Task.FromResult (val);
  return RequestById (id);
 }
  

Indeed, the optimal solution in this case will be to optimize the hot-path, namely, retrieving the value from the dictionary without any unnecessary allocations and load on the GC, while in those rare cases when we still need to go to IO, everything will remain plus/minus old:

  public ValueTask & lt; string & gt;  GetById (int id) {

  if (cache.TryGetValue (id, out string val))
  return new ValueTask & lt; string & gt; (val);
  return new ValueTask & lt; string & gt; (RequestById (id));
 }
  

Let's take a closer look at this fragment of the code: if there are values ​​in the cache, we create a structure, otherwise the real task will be wrapped in a meaningful one. The caller doesn’t care which way the code was executed: ValueTask, in terms of C # syntax, will behave just like a regular Task in this case.

TaskScheduler’s: Manage Task Startup Strategies


The next API that I would like to consider is the TaskScheduler class and its derivatives. I have already mentioned above that in the TPL there is an opportunity to manage the strategies for distributing Task'ov to threads. Such strategies are defined in the heirs of the TaskScheduler class.Virtually any strategy you may need will be found in the ParallelExtensionsExtras library, developed by microsoft, but not part of .NET, but supplied as a Nuget package. Let's take a look at some of them:

  • CurrentThreadTaskScheduler - performs a task on the current thread
  • LimitedConcurrencyLevelTaskScheduler - limits the number of simultaneously executed Tasks to the N parameter that it takes in the constructor
  • OrderedTaskScheduler - defined as LimitedConcurrencyLevelTaskScheduler (1), because tasks will be executed sequentially.
  • WorkStealingTaskScheduler - implements a work-stealing task distribution approach. Essentially is a separate ThreadPool. Solves the problem that in .NET ThreadPool is a static class, one for all applications, which means overloading it or misusing it in one part of the program can lead to side effects in another. Moreover, to understand the cause of such defects is extremely difficult. So there may be a need to use separate WorkStealingTaskScheduler’s in those parts of the program where the use of ThreadPool can be aggressive and unpredictable.
  • QueuedTaskScheduler - allows you to perform tasks according to priority queue rules
  • ThreadPerTaskScheduler - creates a separate thread for each Task that is running on it. It can be useful for tasks that run unpredictably long.

There is a nice, detailed article about the TaskScheduler’ah in the microsoft blog.
For easy debugging of everything connected with the Task's in Visual Studio there is a Tasks window. In this window, you can see the current status of the task and go to the currently running line of code.



PLinq and Parallel class


In addition to the Tasks and everything said in .NET, there are two other interesting tools: PLinq (Linq2Parallel) and the Parallel class. The first promises parallel execution of all Linq operations on multiple threads. The number of threads can be configured with the extension method WithDegreeOfParallelism. Unfortunately, most of the time PLinq in the default mode doesn’t have enough information about the insides of your data source to provide significant speed gains, on the other hand, the price of the attempt is very low: you just need to call the AsParallel method before the Linq method chain and perform performance tests. Moreover, it is possible to transfer to PLinq additional information about the nature of your data source using the Partitions mechanism. You can read more here and here .

The static Parallel class provides methods for parallel iterating the Foreach collection, executing the For loop, and executing several delegates in the Invoke parallel. The execution of the current thread will stop until the end of the calculations. The number of streams can be configured by passing ParallelOptions with the last argument. Using the options, you can also specify a TaskScheduler and CancellationToken.

Findings


When I started writing this article based on my report and information that I collected during the work after it, I didn’t expect so much. Now, when a text editor in which I type this article reproachfully tells me that I went to the 15th page, I will sum up the intermediate results. Other tricks, APIs, visual tools and pitfalls will be covered in the next article.

Conclusions:

  • You need to know the tools for working with threads, asynchrony and concurrency to use the resources of modern PCs.
  • There are many different tools in .NET for this purpose
  • Not all of them appeared right away, because you can often find legacy, however there are ways to convert old APIs with little effort.
  • Working with threads in .NET is represented by the Thread and ThreadPool classes
  • Thread.Abort, Thread.Interrupt, Win32 API TerminateThread functions are dangerous and are not recommended for use. Instead, it is better to use the CancellationToken’s mechanism
  • Flow is a valuable resource, their number is limited. It is necessary to avoid situations where threads are busy waiting for events. For this, it is convenient to use the TaskCompletionSource class
  • The most powerful and advanced .NET tools for working with concurrency and asynchrony are Tasks.
  • The c # async/await operators implement the concept of non-blocking wait
  • You can control the distribution of task's across threads using derivative classes to TaskScheduler
  • ValueTask structure can be useful in optimizing hot-paths and memory-traffic
  • Tasks and Threads in Visual Studio provide a lot of useful information for debugging multi-threaded or asynchronous code.
  • PLinq is a cool tool, but it may not have enough information about your data source, however this can be fixed with the help of partitioning
  • To be continued ...

Source text: .NET: Tools for working with multithreading and asynchrony. Part 1