Question How to call a Rest API in a schedule

raysefo

Well-known member
Joined
Feb 22, 2019
Messages
336
Programming Experience
10+
I have an ASP.NET Core 2.1 web application. The admin gets online game codes from the page below by selecting the related game (Razer Gold Pin), quantity, and email to send those codes.


A Rest API call is being made for the purchase. There is a limit for purchasing, a maximum of 200 purchases can be made at one time.


At first, there wasn't much to buy so there was no problem. But the demand has increased, when there are 10 thousand, 20 thousand purchases, it is necessary to purchase from this screen for hours. I wonder if can we make a large number of purchases without waiting in front of the screen with scheduling this somehow?

YqL9F.png


Javascript in the cshtml:
JavaScript:
$("#btn_add").html(
                            '<span class="spinner-border spinner-border-sm" role="status" id="spinner" aria-hidden="true"></span> Loading...'
                        );
                        var selText = $("#validationCustom04").val();
                        var gameName = $("#validationCustom04 option:selected").text();
                        var quantity = $("#quantity").val();
                        var email = $("#email").val();
                        var price = $("#total").val();
                        var company = $("#validationCustom05").val();

                        if ($("#total").val() !== '') {

                            price = $("#total").val();

                        }

                        var serviceUrl = '/GameBanks/A101PinAsync?quantity=' + quantity + '&game=' + selText + '&email=' + email + '&prc=' + price + '&gameName=' + gameName + '&company=' + company;
                        $.ajax({
                            type: "GET",
                            url: serviceUrl,
                            dataType: 'json',
                            success: function (data) {
                                //alert(JSON.stringify(data));
                                ShowResult(data);
                                $("#spinner").remove();
                                $("#btn_add").html('Add');

                            }, error: function (xhr, status, error) {
                                $("#spinner").remove();
                                $("#btn_add").html('Add');

                                var errorMessage = xhr.responseText;

                                alert('Error - ' + errorMessage);
                            }

                        });

controller method:
C#:
[HttpGet]
public async Task<IActionResult> A101PinAsync(int quantity, string game, string email, int prc, string gameName, string company)
{
    var price = 0;

    try
    {
        string result = null;
                
        var dList = new DemoList();

        if (prc > 0)
        {
            price = prc;
        }

        var games = new Game { Product = game, Quantity = quantity };

        var content = await games.CallGameAsync("Test", "12345", game, quantity, company);

        var deserializedResult = JObject.Parse(content);

        var root = JsonConvert.DeserializeObject<Root>(content);

        if ((string)deserializedResult["coupons"]?[0]?["Serial"] == null)
        {
            result = result + gameName + ":" + (string)deserializedResult["Message"] + "\n";
        }
        else
        {
            foreach (var coupon in root.coupons)
            {
                result = result + gameName + "  Serial:" + coupon.Serial + "  Pin:" + coupon.Pin + "\n";
            }
        }
              
        dList.gameList = result;
                
        // NLOG
        NLogPin(logger, User.Identity.Name, DateTime.Now, result, email);
                
        return new JsonResult(dList);
    }
    catch (Exception e)
    {
        // NLOG
        NLog(logger2, "OyunPalas " + e.Message, DateTime.UtcNow,0);
                
        return StatusCode(500,e.Message);
    }
}

Web API calling method:
C#:
public class Game
{
    public string Username { get; set; }
    public string Password { get; set; }
    public string Product { get; set; }
    public int Quantity { get; set; }
    public string ShopNo { get; set; }

    private static readonly Logger logger2 = LogManager.GetLogger("exceptionFile");

    public async Task<string> CallGameAsync(string username, string password, string product, int quantity, string company)
    {
        try
        {
            HttpWebRequest request = null;
            
            request = (HttpWebRequest)WebRequest.Create("http://111.111.111.111:1907//api/v2/web/purchase");

            var svcCredentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(username + ":" + password));
            
            request.Headers.Add("Authorization", "Basic " + svcCredentials);
            request.Method = "POST";
            request.KeepAlive = false;
            request.ContentType = "application/x-www-form-urlencoded";

            var content = new FormUrlEncodedContent(new[]
            {
                new KeyValuePair<string, string>("productCode", product),
                new KeyValuePair<string, string>("quantity", quantity.ToString()),
                new KeyValuePair<string, string>("shopNo", company),
                new KeyValuePair<string, string>("safeNo", company),
                new KeyValuePair<string, string>("cashierNo", company)
            });

            var urlEncodedString = await content.ReadAsStringAsync();

            using (var streamWriter = new StreamWriter(await request.GetRequestStreamAsync()))
            {
                await streamWriter.WriteAsync(urlEncodedString);
            }

            var httpResponse = (HttpWebResponse) (await request.GetResponseAsync());

            var response = new HttpResponseMessage
            {
                StatusCode = httpResponse.StatusCode,
                Content = new StreamContent(httpResponse.GetResponseStream()),
            };

            //Read response
            var htmlResponse = await response.Content.ReadAsStringAsync();

            return htmlResponse;
        }
        catch (Exception e)
        {
            //NLOG
            NLog(logger2, "Test" + e.Message);
            throw;
        }
    }

    public void NLog(Logger logger, string user)
    {
        var sb = new StringBuilder();
        sb.AppendLine("Test: " + user);
        logger.Info(sb.ToString());
    }
}

I have to pass the product (game type) to the job/scheduler. I read the https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/more-about-jobs.html#jobdatamap documentation but did not fully understand how to use it in my case. And I also need if I can stop/cancel the job from this page before its end time. I really appreciate any help you can provide. I have no experience with this scheduling.

Thank you.
 

cjard

Well-known member
Joined
Jan 25, 2012
Messages
433
Programming Experience
10+
I don't really understand what you're trying to do;

* do you own/maintain/have the source code of the webpage you're trying to call programmatically?
* Do you own/maintain/have the source code of the rest service it calls?
* Are you trying to automate the use of the webpage (by this I mean are you trying to make an app that automatically types into fields and clicks buttons, to replicate what a human does)?
* Or are you looking toss the page aside and call the API directly (by this I mean write an app that sends carefully crafted requests to the API that are equivalent to a human clicking on buttons and typig in fields)?
 
Last edited:

cjard

Well-known member
Joined
Jan 25, 2012
Messages
433
Programming Experience
10+
"I have the web page" is quite vague and so I'm not sure what it actually means in a real life sense, but I'm also trying to find a solution to your problem. Could you please quote my post and answer the questions directly? I asked 3 questions, but I've expanded this to more like 4, with a hopefully more accurate wording
 

raysefo

Well-known member
Joined
Feb 22, 2019
Messages
336
Programming Experience
10+
I don't really understand what you're trying to do;

* do you own/maintain/have the source code of the webpage you're trying to call programmatically? YES
* Do you own/maintain/have the source code of the rest service it calls? YES
* Are you trying to automate the use of the webpage (by this I mean are you trying to make an app that automatically types into fields and clicks buttons, to replicate what a human does)? NOPE
* Or are you looking toss the page aside and call the API directly (by this I mean write an app that sends carefully crafted requests to the API that are equivalent to a human clicking on buttons and typig in fields)? NOPE
I am trying to find a solution to my problem. The problem is I can't get game codes as much as I want from the web API because of the limitation. How can I improve this process instead of spending hours on screen? Was it descriptive enough for you?
 

cjard

Well-known member
Joined
Jan 25, 2012
Messages
433
Programming Experience
10+
Not really. You say you own the source code to the API that has the "200 at a time" limitation. Edit the source code to remove the limitation, publish a new version of the API and order 20,000 game codes in one hit
 

cjard

Well-known member
Joined
Jan 25, 2012
Messages
433
Programming Experience
10+
Write a program that calls the third party API directly, and has a timer. It can be done in one line of code in a winforms app

Write a program that calls your API that calls their API, and has a timer.
It can be done in one line of code in a winforms app

Sign up for a third party scheduler service that will call your API or their API on a timer. It can be done in a few clicks in Windows Azure scheduled jobs
 

raysefo

Well-known member
Joined
Feb 22, 2019
Messages
336
Programming Experience
10+
Write a program that calls your API that calls their API, and has a timer.
It can be done in one line of code in a winforms app
I am doing this with the asp.net core application which has a screenshot in my first post.

Sign up for a third party scheduler service that will call your API or their API on a timer.
I was planning to use quartz but as I mentioned in my first post I couldn't understand how to use it.

It can be done in a few clicks in Windows Azure scheduled jobs
Don't wanna use Azure.
 

Skydiver

Staff member
Joined
Apr 6, 2019
Messages
6,514
Location
Chesapeake, VA
Programming Experience
10+
Then don't use Quartz. Just use Hangfire instead to let you run long running or periodic processes. Your life would be so much easier, but the system admins whose administer your IIS box will hate your guts.

I know because I regret the day I gave the link to blog post about Hangfire to some devs in another team who innocently asked "how do I keep my request handler that runs a lot of long running jobs from being timed out by IIS?" I should have known they were planning on doing this on the IIS boxes that my team manages for the sake for separation of duties. Now they keep the CPU on those servers busy even when they don't have a long running job because Hangfire requires the app pool never die. Even worse they have a memory leak somewhere, and so their app pool that never dies is slowly eating up all the memory on a server that is shared with a couple hundred other apps.
 

raysefo

Well-known member
Joined
Feb 22, 2019
Messages
336
Programming Experience
10+
I thought of a solution like this, but I'm not sure if it's viable.

- The user will enter which game, how many they want to buy, and their e-mail address on the screen.
- This data will be saved in a table (BulkRequest) containing this information and the status field in the SQL database. (game: ABC, quantity:20000, status:0)

- One IHostedService will get the saved data in a schedule which status=0 from the BulkRequest table and the save requests with status information in a new table (GameRequests) in multiples of 100. (game: ABC, quantity:100, status:0)

- Another IHostedService will get the records in 10 minutes schedule which status=0 from the GameRequests table and make the 3rd party API call. Write a response to another table (GameResponses) and set the status=1 if there is no error.

- When all requests are completed (status = 1), the status in the BulkRequest table will also be updated to 1.

- A third IHostedService will check the status of BulkRequest, if status=1 then maybe a fourth service will prepare a file and send the responses to the e-mail.
 

cjard

Well-known member
Joined
Jan 25, 2012
Messages
433
Programming Experience
10+
More involved than I'd have gotten, I think.. table for requests; request ID, game, quantity, table for codes: requestID, code. One app with a timer that finds all requests where the count of codes is less than the request, then chew through them requesting 100 codes at a time and loading them into the DB, wait a minute on a simple await delay, then go again

But we aren't here to design it for you, so you feel free to engineer it however you like!
 

raysefo

Well-known member
Joined
Feb 22, 2019
Messages
336
Programming Experience
10+
I don't want you to design it 4 me. Just brainstorming and If you have experience with scope, I would like to benefit.
 

cjard

Well-known member
Joined
Jan 25, 2012
Messages
433
Programming Experience
10+
Start small, get something working and out the door. Kaizen says "go for the simple solution, not the perfect one"..

Eventually, when it's drowning in its own limitations, the prtotype will have taught you a fairly exhaustive list of problems it needs to solve, so it can be done "properly" with quite an accurate scope..
 

raysefo

Well-known member
Joined
Feb 22, 2019
Messages
336
Programming Experience
10+
I think I am gonna try using RabbitMQ. Made some research, asked questions, and tried to decide which solution to choose. There is this web application for the user in order to enter a total number of codes and the game type. There will be IHostedService1 (producer) to get this total number of codes and send it to RabbitMQ in pieces that the 3rd party service can digest. There will be IHostedService2 (consumer) who gets those pieces after some time (not immediately) and call the 3rd party web service with Parallel.ForEachAsync may be reducing the total time elapsed by setting the max number of concurrent tasks we will enable in one call.

What do you say? Any suggestions?
 

Skydiver

Staff member
Joined
Apr 6, 2019
Messages
6,514
Location
Chesapeake, VA
Programming Experience
10+
If you are going to break up the big order into chunks for RabbitMQ, why not just break it up into chunks for yourself, and then do the same strategy that you would have implemented had you called RabbitMQ?

I can imagine that for RabbitMQ, you would have broken the big order into chunks, and then returned to the caller letting them know that you will notify them when the order is fulfilled. Why can't you just do the same and break up the big order into chunks and queue them up into your database, and then let the caller know that you will notify them when the order is fulfilled?
 

raysefo

Well-known member
Joined
Feb 22, 2019
Messages
336
Programming Experience
10+
You mean without using RabbitMQ right? I think it can be removed from the solution. How can I implement Parallel.ForEachAsync
for 100 by 100 codes in the IHostedService?
 

cjard

Well-known member
Joined
Jan 25, 2012
Messages
433
Programming Experience
10+
Egads; you've spent nearly 2 months designing and researching how to implement Enterprise FizzBuzz.

If you'd slung together and crappy winforms app that has a 1 second timer, that ordered a single code every time it fired, you'd have ordered 3,954,700 codes by now

If you're doing it as an academic exercise, by all means crack on, but if the business is riding on this just get something out the door..
 
Top Bottom