Question sending an HTTP request/POST to a website

nofu1

Member
Joined
Oct 30, 2020
Messages
23
Programming Experience
Beginner
I am still a beginner in c#. I have a console application. In javascript, we can do the following document.getElementById("myText").value = "Mary; which is used to set an HTML element. Is this possible in a console application that sending an HTTP requests to a website or should the application be using a post method. Please see code so far below. There is a textbox with the tag name testmedownelement at the website(www.~~~~~~~~~~~~~~~~~~~~~~.com).

This is the javascript code: document.getElementsByName('testmedownelement').value = "XXXXXX", I am looking for a C# equivalent since I can't use java script in my console application

C#:
            WebRequest request = WebRequest.Create("www.~~~~~~~~~~~~~~~~~~~~~~.com");
        

           request.Method = "GET";
            request.ContentType = "application/html";
            
          

            // Get the response.

            HttpWebResponse response = (HttpWebResponse)request.GetResponse();

            //Display Status

            Console.WriteLine(response.StatusDescription);

            // Get the stream containing content returned by the server.

            Stream dataStream = response.GetResponseStream();

            // Open the stream using a StreamReader for easy access.

            StreamReader reader = new StreamReader(dataStream);

            // Read the content.

            string responseFromServer = reader.ReadToEnd();

            // Display the content.

            Console.WriteLine(responseFromServer);

            // Cleanup the streams and the response.

            reader.Close();

            dataStream.Close();

            response.Close();

            Console.ReadKey();
 

jmcilhinney

C# Forum Moderator
Staff member
Joined
Apr 23, 2011
Messages
3,555
Location
Sydney, Australia
Programming Experience
10+
In JavaScript you are manipulating the DOM of an HTML page loaded in your browser. That has no effect at all on the server. It is only when you post the form in that page to the server that the form data, including the value you set, gets processed. If you're using a console application then there is no HTML page so there's no DOM to manipulate. It is up to you to construct and send a post to the server that contains the same data as would be sent by submitting the web page form. There is no document so there is no element. You just have to send the appropriate data in the same format as that form on that HTML page would.
 

nofu1

Member
Joined
Oct 30, 2020
Messages
23
Programming Experience
Beginner
Thanks Jmcilhinney, I suspected so, are there any good tutorials on how to construct and send a post to the server, For instance, How does one know the format, how many parameters to pass and the order of parameters.


For example, I tried the following modification using a post method to the code but i got the error message, the underlying connection was closed: An unexpected error occurred on a send

C#:
WebRequest request = WebRequest.Create("www.~~~~~~~~~~~~~~~~~~~~~~.com");
request.Method = "POST";
request.ContentType = "application/html";

string thinginfo = "XXXXXX";
request.Headers.Add("testmedownelement", thinginfo);
         
// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

//Display Status
Console.WriteLine(response.StatusDescription);

// Get the stream containing content returned by the server.
Stream dataStream = response.GetResponseStream();

// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);

// Read the content.
string responseFromServer = reader.ReadToEnd();

// Display the content.
Console.WriteLine(responseFromServer);

// Cleanup the streams and the response.
reader.Close();
dataStream.Close();
response.Close();
Console.ReadKey();
 
Last edited:

Sheepings

Retired Programmer
Staff member
Joined
Sep 5, 2018
Messages
1,971
Location
UK
Programming Experience
10+
Sounds like you might be dealing with a TLS issue. If so, this would also suggest that you are using an older version of .net than 4.6. On your WebRequest.Create(", specify the http/https protocol. You also need to know the exact address of your forms post uri. Every form has an action uri that a form is submitted too :
<form role="form" id="myform" action="/somefile.ext" method="post">
And even still, this is not the appropriate way to do it. If its your server, then create an API for your server.

See the web request example and documentation here : How to: Send data by using the WebRequest class
How does one know the format, how many parameters to pass
It's not much different than knowing how the action uri knows what values the form will send. You should too. If its not your server, I would suggest something like Selenium.

You might also find the answers over here useful : How to submit http form using C#
 

nofu1

Member
Joined
Oct 30, 2020
Messages
23
Programming Experience
Beginner
Sounds like you might be dealing with a TLS issue. If so, this would also suggest that you are using an older version of .net than 4.6. On your WebRequest.Create(", specify the http/https protocol. You also need to know the exact address of your forms post uri. Every form has an action uri that a form is submitted too :
<form role="form" id="myform" action="/somefile.ext" method="post">
And even still, this is not the appropriate way to do it. If its your server, then create an API for your server.

See the web request example and documentation here : How to: Send data by using the WebRequest class

It's not much different than knowing how the action uri knows what values the form will send. You should too. If its not your server, I would suggest something like Selenium.

You might also find the answers over here useful : How to submit http form using C#
Hi I was able to modify the code but I am now getting 401 authorization error. Please see modified code below

C#:
   private static CredentialCache GetCredential()
        {
            ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls | (SecurityProtocolType)3072;
            string url = @"https://~~~~~~~~~~~~~~~~~~~~~~.com/~~~~~";
            CredentialCache credentialCache = new CredentialCache();
            credentialCache.Add(new System.Uri(url), "Basic", new NetworkCredential(~~~~, ~~~~~~~~~~~~));
            return credentialCache;
        }

public static void testcall (string url)
        {
     
            WebRequest request = WebRequest.Create(url);
            request.PreAuthenticate = true;
            request.Credentials = GetCredential();
       
            ((System.Net.HttpWebRequest)request).Referer = url;
            ((System.Net.HttpWebRequest)request).Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
          ((System.Net.HttpWebRequest)request).UserAgent = "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0";

            // Set the Method property of the request to POST.
            request.Method = "POST";
            // Create POST data and convert it to a byte array.
            string postData = "param1=XXXXXX";
            ASCIIEncoding encoding = new ASCIIEncoding();

            byte[] byteArray = encoding.GetBytes(postData);
            // Set the ContentType property of the WebRequest.
            request.ContentType = "application/x-www-form-urlencoded";
            // Set the ContentLength property of the WebRequest.
            request.ContentLength = byteArray.Length;
            // Get the request stream.
            Stream dataStream = request.GetRequestStream();
            // Write the data to the request stream.
            dataStream.Write(byteArray, 0, byteArray.Length);
            // Close the Stream object.
            dataStream.Close();
   

         
            // Get the response.
            WebResponse response = request.GetResponse();
            // Display the status.
            Console.WriteLine(((HttpWebResponse)response).StatusDescription);
            // Get the stream containing content returned by the server.
            dataStream = response.GetResponseStream();
            // Open the stream using a StreamReader for easy access.
            StreamReader reader = new StreamReader(dataStream);
            // Read the content.
            string responseFromServer = reader.ReadToEnd();
            // Display the content.
            Console.WriteLine(responseFromServer);
            // Clean up the streams.
            reader.Close();
            dataStream.Close();
            response.Close();
        }
 

Sheepings

Retired Programmer
Staff member
Joined
Sep 5, 2018
Messages
1,971
Location
UK
Programming Experience
10+
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls | (SecurityProtocolType)3072;
You don't need to do this if you are running the latest version of dotnet frameworks. The latest versions take care of this for you.

What version are you running?

You need to send a WWW-Authenticate header that contains information on how to authorise your action. 401 means that you are not using the correct credentials for whatever resource you are trying to access.

At this stage, I would suggest debugging your code, and make sure everything is operating as expected. Sorry I don't have much more time to help you further, as I am busy at present. Perhaps one of the other guys can step in and help. You can read a little more about your error here : 401 Unauthorized — httpstatuses.com
 

nofu1

Member
Joined
Oct 30, 2020
Messages
23
Programming Experience
Beginner
You don't need to do this if you are running the latest version of dotnet frameworks. The latest versions take care of this for you.

What version are you running?

You need to send a WWW-Authenticate header that contains information on how to authorise your action. 401 means that you are not using the correct credentials for whatever resource you are trying to access.

At this stage, I would suggest debugging your code, and make sure everything is operating as expected. Sorry I don't have much more time to help you further, as I am busy at present. Perhaps one of the other guys can step in and help. You can read a little more about your error here : 401 Unauthorized — httpstatuses.com
I am running asp.net 3.5. All help is appreciated
 

Sheepings

Retired Programmer
Staff member
Joined
Sep 5, 2018
Messages
1,971
Location
UK
Programming Experience
10+
But why are you running under 3.5? o_O Complete your windows updates. Install the latest dotnet frameworks...

In your project settings. Change your framework :
In your project, change the target framework to 4.8 and tell me if it has any affect on the 401 error. It it doesn't, you may need to consider how the server authenticates requests.
 

Sheepings

Retired Programmer
Staff member
Joined
Sep 5, 2018
Messages
1,971
Location
UK
Programming Experience
10+
I am running asp.net 3.5. All help is appreciated
I thought you said you was running this as a console app?

If you are running this as .net mvc3 razor app. Make this change in your web.config.
<system.web>
<compilation debug="true" targetFramework="4.8" />
<httpRuntime targetFramework="4.8" />
 

Skydiver

Staff member
Joined
Apr 6, 2019
Messages
2,586
Location
Chesapeake, VA
Programming Experience
10+
If you are running that old version, then yes, masking in that value for TLS 1.2 support would be your only choice.

Unless you are running on Windows Server 2019, beware that .NET Framework 3.5 hit end of life back in July 2011.

 

Sheepings

Retired Programmer
Staff member
Joined
Sep 5, 2018
Messages
1,971
Location
UK
Programming Experience
10+
Regarding what @Skydiver said, see p/1#6 regarding requirements. The TLS requirement mentioned by Skydiver above is not required in later versions of Dotnet frameworks.
 

nofu1

Member
Joined
Oct 30, 2020
Messages
23
Programming Experience
Beginner
But why are you running under 3.5? o_O Complete your windows updates. Install the latest dotnet frameworks...

In your project settings. Change your framework :
In your project, change the target framework to 4.8 and tell me if it has any affect on the 401 error. It it doesn't, you may need to consider how the server authenticates request
It is a console application and the max .net target framework I see, is 3.5. Sorry I might have misunderstood this point and suggestion,
masking in that value for TLS 1.2 support would be your only choice.

Can you please help clarify the solution
 

Sheepings

Retired Programmer
Staff member
Joined
Sep 5, 2018
Messages
1,971
Location
UK
Programming Experience
10+
Sorry I might have misunderstood this point and suggestion,
Ok This :
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls | (SecurityProtocolType)3072;
Well...
You don't need to do this if you are running the latest version of dotnet frameworks. The latest versions take care of this for you.
If you run the latest dot net frameworks, you don't need to put that line in there.

Then there is this :
I am running asp.net 3.5. All help is appreciated
And :
It is a console application and the max .net target framework I see, is 3.5.
Which is it?
 

nofu1

Member
Joined
Oct 30, 2020
Messages
23
Programming Experience
Beginner
Ok This :

Well...

If you run the latest dot net frameworks, you don't need to put that line in there.

Then there is this :

And :

Which is it?
It is a console applicatin. In visual studio, if i right-click on the .csproj file, and go to the application tab, it shows

target framework: .net framework 3.5
and output type: Console Application
 

Sheepings

Retired Programmer
Staff member
Joined
Sep 5, 2018
Messages
1,971
Location
UK
Programming Experience
10+
I have a console app and I can use 4.8.

There is an option to install other versions.

Move up to 4.8, and remove the line I told you to remove. Then tell us if you get a new error or not?
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls | (SecurityProtocolType)3072; is not required in 4.8.
And your problem is indicative of a problem with either TLS or your credentials.

You will need someone else to help from here. I have server deployments to attend to.
 

Attachments

  • Screenshot_38.jpg
    Screenshot_38.jpg
    202.6 KB · Views: 4

nofu1

Member
Joined
Oct 30, 2020
Messages
23
Programming Experience
Beginner
I installed .net 4.8 framework but it didn't integrated automatically with my visual studio. In otherwords, it is not included in the drop-down list like yours
visual_studio_screenshot.png
 

Skydiver

Staff member
Joined
Apr 6, 2019
Messages
2,586
Location
Chesapeake, VA
Programming Experience
10+
You need to install .NET Framework SDK, not the .NET Framework runtime, for it to show up in the VS list.

Also that light gray gradient on the tabs suggests that you may have an older version of Visual Studio. I don't think I've seen that in quite a while since the newer Visual Studio uses flat colouring. I recommend upgrading to the latest version.
 

JuggaloBrotha

Staff member
Joined
Apr 23, 2011
Messages
177
Location
Lansing, MI; USA
Programming Experience
10+
Skydiver and Sheepings, he's using VS 2008 which can only target .Net 2.0, 3.0, and 3.5
What he needs to do first is download the VS2019 Community Edition (it's free) then he can open his project in it and change it to target .Net 4.8
 

Skydiver

Staff member
Joined
Apr 6, 2019
Messages
2,586
Location
Chesapeake, VA
Programming Experience
10+
Using VS2019 by itself won't fix things. You still need to build your code targeting .NET Framework 4.8. What VS2019 does is allow you to pick that target.

As an aside, you could also build your code with just the .NET Framework 4.8 SDK on the command line, but that will require you to use VS Code, or some other text editor. Since you are still on the learning curve, keep things simple and just install VS2019. You learn the language first, then later learn about how things are actually done by the IDE for you.

(Yes, we C# folks are less strict than the C and C++ folks that insist that you first learn how the compiler and linker works before for learn how to write code.)
 
Top Bottom