Question .net core 2.1 console app with http client factory question

raysefo

Well-known member
Joined
Feb 22, 2019
Messages
361
Programming Experience
10+
Hi guys,
I would like to implement a scheduled task which checks an API for every hour and insert data into a database if data is not in the database already. I read some articles and came out with this. I wonder if you can guide me to insert data into a SQL table if it is not already there. (By the way, most probably I will not use the Polly retry mechanism in my code.)


Program.cs:
using System;
using System.IO;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Polly;
using Polly.Extensions.Http;
using Polly.Registry;

namespace TrendyolGamePurchase
{
    class Program
    {
        static async Task Main(string[] args)
        {
            //Read App Settings
            var build = new ConfigurationBuilder();
            BuildConfig(build);

            var config = build.Build();
          
            Console.Write(config["ConnectionStrings:Development"]);
          
            //Polly Retry
            var builder = new HostBuilder()
                .ConfigureServices((hostContext, services) =>
                {
                    IPolicyRegistry<string> registry = services.AddPolicyRegistry();

                    //First Policy
                    IAsyncPolicy<HttpResponseMessage> httpWaitAndRetryPolicy =
                        Policy.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
                            .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(3, retryAttempt)));

                    registry.Add("SimpleWaitAndRetryPolicy", httpWaitAndRetryPolicy);

                    //Second Policy
                    IAsyncPolicy<HttpResponseMessage> noOpPolicy = Policy.NoOpAsync()
                        .AsAsyncPolicy<HttpResponseMessage>();

                    registry.Add("NoOpPolicy", noOpPolicy);

                    //Third Policy
                    var timeOutPolicy = Policy.TimeoutAsync(TimeSpan.FromSeconds(10));
                        
                    registry.Add("timeOutPolicy", timeOutPolicy);

                    services.AddHttpClient("TestClient", client =>
                    {
                        client.BaseAddress = new Uri("http://test//api/v2/web/game/purchase");
                        client.DefaultRequestHeaders.Add("Accept", "application/json");
                    }).AddPolicyHandlerFromRegistry((policyRegistry, httpRequestMessage) =>
                    {
                        if (httpRequestMessage.Method == HttpMethod.Post)
                        {
                            Console.WriteLine(DateTime.Now);
                            return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("SimpleWaitAndRetryPolicy");
                        }
                        return policyRegistry.Get<IAsyncPolicy<HttpResponseMessage>>("NoOpPolicy");
                    });

                    services.AddSingleton<IHostedService, BusinessService>();
                });

            await builder.RunConsoleAsync();
        }
        static void BuildConfig(IConfigurationBuilder builder)
        {
            builder.SetBasePath(Directory.GetCurrentDirectory())
                .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);


        }
    }
}

BusinessService.cs:
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Hosting;

namespace TrendyolGamePurchase
{
    public class BusinessService : IHostedService
    {
        private IHttpClientFactory _httpClientFactory;
        public BusinessService(IHttpClientFactory httpClientFactory)
        {
            _httpClientFactory = httpClientFactory;
        }

        public async Task StartAsync(CancellationToken cancellationToken)
        {
            await MakeTestRequestsToRemoteService();
        }

        public async Task MakeTestRequestsToRemoteService()
        {
            HttpClient httpClient = _httpClientFactory.CreateClient("TestClient");

            var authenticationBytes = Encoding.ASCII.GetBytes("Test:12345");

            httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
                Convert.ToBase64String(authenticationBytes));
            httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

            var content = new FormUrlEncodedContent(new[]
            {
                new KeyValuePair<string, string>("productCode", "1"),
                new KeyValuePair<string, string>("quantity","1"),
                new KeyValuePair<string, string>("shopNo","Palas"),
                new KeyValuePair<string, string>("safeNo","Palas"),
                new KeyValuePair<string, string>("cashierNo","Palas")

            });
            var response = await httpClient.PostAsync("http://test//api/v2/web/game/purchase", content);
            

        }

        public Task StopAsync(CancellationToken cancellationToken)
        {
            return Task.CompletedTask;
        }
    }
}
 
It would be the same way you insert using a web app. Didn't you already have a web app which could talk to a database?
 
As a quick aside, I got the impression that you are building this for a client. Please note that .NET Core 2.1 is hitting end of life on Aug 21, 2021. That in less than 6 months. So unless you are trying to milk more money out of your client by charging them more hours to to upgrade them, it may make more sense to pick a different version.

 
id and orderDate will determine a duplicate
The id should be set to AI and marked as a primary key for the record in which it represents in that table, and it should only be treated as such, and nothing else.

The orderDate should be used to determine if an order in the database is older than the order you get from your API provider or that of a date you want to update a new record.

I'm not sure I like your logic. If you don't want duplicate dates, why not make the orderDate field unique?

Who will be using this system? I ask because what if two people place an order at the exact same time, and how should the records be handled or is there a reason this won't be an issue?
 
He can't set id to auto increment because it comes from the API that he is calling.
 
You've read more of his topics than I have. So you'd know. I assumed he was querying an API and saving the data to his own/external database?

Maybe I'm wrong, or maybe he can explain if what I proposed is at all possible?
If it's not, then create your own API, one you can control, instead of plugging someone else's.

If memory serves me correct, that's what we have been telling him to do for a long time now. If the API you are using, provides limited functionality, then suck it up or else find something else to use.
 
well as @Skydiver mentioned I can't set id autoincrement because I am getting it from the API. Not sure but most probably id will be unique. But not to take chances, I will also use orderDate in the keys. There is this customerId so it won't be a problem if two different user makes orders at the same time.
Now I think I have to create a method in the BusinessService.cs to implement this insert operation.
 
OMG! So you've been working on this project for several months and haven't been writing stuff into a database? If the company/client/product is making a profit based on the number of transactions and you aren't keeping your own records for the transactions the go through the API, does that mean that you were originally just taking the API owner's word for it as what the proceeds due were?

Anyway, better start learning some ADO.NET. I (and @Sheepings) would recommend you steer away from Entity Framework, specially since you say that you are more comfortable working with direct SQL. Unfortunately, a lot of the current literature seems to default to Entity Framework.

 
That is why I said regarding my previous post :
I assumed he was querying an API and saving the data to his own/external database?
I was assuming, but I was also alluding to this. If the OP is not using his own/or his client's database to store the returned data from this questionable API, then how do you know how many orders have been placed through the App you are developing for your client?

You better hope your client doesn't sue you later on for your own incompetence regarding the creation of this app. From what I was gathering from this thread and past threads, is that the OP is also using the API as the source for data transit while not keeping any external records for his client on how many orders have been issued through this system he is building up.

Hopefully, I'm wrong for his sake.

Regarding the whole EF usage, I think you should read over this : iamtimcorey | Should I Use Entity Framework

@raysefo From what you said on p2/#21 - That data received from the API should be getting stored into an external server for your clients record keeping. My my...
 
if you guys have any suggestions to make it more maintainable, testable, robust, and dynamic?

C#:
class Program
    {
        static async Task Main(string[] args)
        {
                          
            var builder = new HostBuilder()
                .ConfigureServices((hostContext, services) =>
                {
                    //Setting up API Client
                    services.AddHttpClient("OrdersClient", client =>
                    {
                        client.BaseAddress =
                            new Uri("https://test/282675/orders?status=Created");

                        client.DefaultRequestHeaders.Add("Accept", "application/json");
                    });

                    services.AddSingleton<IHostedService, BusinessService>();

                    //Setting up app settings configuration
                    var config = LoadConfiguration();   
                    services.AddSingleton(config);
                });

            await builder.RunConsoleAsync();
      
        public static IConfiguration LoadConfiguration()
        {
            var builder = new ConfigurationBuilder()
                .SetBasePath(Directory.GetCurrentDirectory())
                .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);

            return builder.Build();
        }
    }

public class BusinessService : IHostedService
    {
        private IHttpClientFactory _httpClientFactory;
        private readonly IHostApplicationLifetime _applicationLifetime;
        private IConfiguration _configuration;
        public BusinessService(IHttpClientFactory httpClientFactory, IHostApplicationLifetime applicationLifetime, IConfiguration configuration)
        {
            _httpClientFactory = httpClientFactory;
            _applicationLifetime = applicationLifetime;
            _configuration = configuration;
        }

        public async Task StartAsync(CancellationToken cancellationToken)
        {
            await MakeRequestsToRemoteService();
            //Stop Application
            _applicationLifetime.StopApplication();
        }

        public async Task MakeRequestsToRemoteService()
        {
            HttpClient httpClient = _httpClientFactory.CreateClient("OrdersClient");

            var authenticationBytes = Encoding.ASCII.GetBytes("test:1234");

            httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
                Convert.ToBase64String(authenticationBytes));
            httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

            var response = await httpClient.GetAsync("https://test/282675/orders?startDate=1612126800000");
            Console.WriteLine(DateTime.Now);

            if (response.IsSuccessStatusCode)
            {
                Root orders = await response.Content.ReadAsAsync<Root>();

                if (orders.Content.Count > 0)
                {
                    foreach (var content in orders.Content)
                    {
                        Console.Write(content.CustomerId + " " + content.CustomerFirstName + " " + content.CustomerLastName + " " + content.CustomerEmail +" " + content.TotalPrice + " "+ content.InvoiceAddress.Phone + " " + content.Lines[0].ProductCode + " " +content.Lines[0].ProductName );
                        InsertData(content);
                    }
                }

            }
            
            
        }

        public void InsertData(Content content)
        {
          
            
            string connString = _configuration["ConnectionStrings:Development"];
            using (SqlConnection sqlConnection = new SqlConnection(connString))
            {
                sqlConnection.Open();

                using (SqlCommand command = new SqlCommand())
                {
                    command.Connection = sqlConnection;
                    string sql = @"insert into Orders (id, customerId, firstName, lastName, phone, productCode, productName, price, orderDate, status) values (@id, @customerId, @firstName, @lastName, @phone, @productCode, @productName, @price, @orderDate, @status)";
                    command.CommandText = sql;
                    command.Parameters.Clear();
                    
                    try
                    {
                        command.Parameters.Add("id", SqlDbType.BigInt).Value = content.Id;
                        command.Parameters.Add("customerId", SqlDbType.Int).Value = content.CustomerId;
                        command.Parameters.Add("firstName", SqlDbType.VarChar).Value = content.CustomerFirstName;
                        command.Parameters.Add("lastName", SqlDbType.VarChar).Value = content.CustomerLastName;
                        command.Parameters.Add("phone", SqlDbType.VarChar).Value = content.InvoiceAddress.Phone;
                        command.Parameters.Add("productCode", SqlDbType.Int).Value = content.Lines[0].ProductCode;
                        command.Parameters.Add("productName", SqlDbType.VarChar).Value = content.Lines[0].ProductName;
                        command.Parameters.Add("price", SqlDbType.Float).Value = content.TotalPrice;
                        command.Parameters.Add("orderDate", SqlDbType.BigInt).Value = content.OrderDate;
                        command.Parameters.Add("status", SqlDbType.TinyInt).Value = 1; //Retrieved

                        command.ExecuteNonQuery();
                    }
                    catch (SqlException exception)
                    {
                        if (exception.Number == 2627) // Cannot insert duplicate key row in object error
                        {
                            Console.WriteLine("Duplicates...");
                        }
                        else
                            throw; // Throw exception if this exception is unexpected
                    }
                }
            }
        }

        public Task StopAsync(CancellationToken cancellationToken)
        {
            return Task.CompletedTask;
        }
    }
 
Why are you stopping the application on line 53?

It's unclear to me how lines 3-34 can possibly compile.

There's no need for line 73. The foreach will not iterate if there are zero items.

Line 120: Magic numbers should be avoided.

There is no need for liken 101. It's a brand new SqlCommand instance. Why would you need to clear out the Parameters list when it's already empty.
 
Back
Top Bottom