Question Logging to database on a different thread

[BlackRiver1987]

Hello everyone,

I created a simple class library project that should enable me to do logging in my other projects. Now one of the things I wanted to implement was logging the data on a another thread. So I created a function like this:

        public async Task LogMessage(string _Message)
        {
            if (UserID == null)
            {
                if (RetryLogsList.Count < Configuration.LogHistoryMax)
                {
                    LogToRetryList(_Message);
                }
                else if (RetryLogsList.Count >= Configuration.LogHistoryMax)
                {


                    foreach (var one_item in RetryLogsList)
                    {
                        await Task.Run(() => LogToDatabase(one_item.Message, one_item.Created));
                    }


                    RetryLogsList.Clear();


                }




            }
            else
            {


                if (RetryLogsList.Count > 0)
                {
                    foreach (var one_item in RetryLogsList)
                    {
                        await Task.Run(() => LogToDatabase(one_item.Message, one_item.Created));
                    }


                    RetryLogsList.Clear();


                }






                await Task.Run(() => LogToDatabase(_Message));


            }
        }

This is the function that is called by anyone consuming the logging service like so:

await Log.LogMessage("Some log message");

Now, I don't really understand what happens under the hood. LogToDatabase is a synchronous metod that uses SqlConnection and SqlCommand objects to write data to database, and is being called by asynchronous function with Task.Run.

Is this creating a new thread every time Log.LogMessage() function is called? If so, how are threads managed? More importantly, is this a good option at all?

Thank you for your input.

 

[JohnH]

Task.Run Method (Action) (System.Threading.Tasks)

Queues the specified work to run on the thread pool

The Managed Thread Pool

 

[jmcilhinney]

There's not much point calling Task.Run in the loop if you're going to await it there. It would only be useful if you kicked off all the tasks first and then awaited them all afterwards. As it is, you're going to kick of a task and wait for it to complete before kicking off the next one, so what's the point?

That said, if you have multiple records to save to a database, wouldn't it be a good idea to save them all in one go, rather than individually?

 

[BlackRiver1987]

Hello jmcilhinney,

Thanks for taking the time to help me out. I see your point about the for loop, and that's something I can easily fix. Thing I am more concerned about is implementing this in lets say Web Api environment.

For example, you can have hundreds of users hitting a UserLogin endpoint where I might log some steps. For simplicity sake, let's say I create an instance of LogClass on a controller level.

Something like this:

public class MyLoginController: ApiController
{
   private LogClass myLog = new LogClass(ConfigurationObject);
}

Down the line, I might call the log function, to simply log some steps:

await Log.LogMessage("Some log message");

Now, every time a user request comes in, I will have a "new instance" of MyLoginController class, thus creating a new log object. How will this impact performance? Because, as it stands: "Task.Run queues the specified work to run on the thread pool". With a large number of users, wont that polute the thread pool?

 

[jmcilhinney]

With a large number of users, wont that polute the thread pool?

The thread pool is much like the tellers in a bank or the checkout operators in a supermarket. The customers join the queue and, as a teller or checkout operator becomes available, the next person leaves the queue and is served. An issue is only going to arise if the number of people joining the queue is greater than the number leaving over a significant period. In your case, how many tasks do you think you would need to queue over what period for the thread pool to overflow?

 

[BlackRiver1987]

Well, the Web API is currently being used by around 800 users. The logging functionality will mostly be used to log exceptions in production mode. In development, we might log some additional steps. So, only in a situation where I would have all 800 users hitting the same endpoint, at the same time, producing the same exception I might overflow the thread pool. At that is if I am understanding the concept correctly.

 

[jmcilhinney]

The thread pool would be able to queue many thousands of tasks. The whole point is that only a relatively small number of tasks will be executed simultaneously but many more can be queued for subsequent execution.