Question What is the most efficient way of executing many (1000+) tasks concurrently?

[systemlordanubis]

Hi All,

I have an application which goes out and polls 1000+ devices on a recurring basis (around every 2 seconds - faster if possible). I'm seeking ways to improve the performance of this processing by using the most appropriate threading process.

Essentially the parent process obtains a list of units to poll (IP network based units) and fires off the queries in rapid succession. Each query then receives the response and logs it to the database before completing. The parent doesn't need to wait for the child tasks to complete, these can all happen in the background, but any feedback to the parent is welcome.

My question is, what's the fastest way to launch these processes in the most appropriate way? Is using Task.Run() in a for loop the most reasonable way or is there another way this could be handed resulting in more concurrency and rapid processing of the device polls?

Example:

while (_RunParentLoop){
SampleTask[] _Tasks = { 1000+ task items };
foreach (SampleTask _t in _Tasks)
{
Task.Run(() => DoWork(_t.Name));
}
}

Thanks
Anubis.