HtmlAgilityPack Issue


Well-known member
Feb 2, 2013
Programming Experience
I am scraping data from a series of web pages, each page has 50 links for other pages , i load these pages one at a time and collect data from each page, each page has up to 60 fields of data that needs to be collected...this is working fine. My issue is that random fields will not be collected on random pages. I load a HtmlListCollection of all the fields and then do a foreach loop on the list to collect the data for each field....the issue is not all the fields are being could be 1 in 1000 pages that doesn't collect correctly or it could be a series of pages in a row...I never know which fields aren't getting collected until after the program is finished usually takes 3 hrs to complete.

could this be a performance issue on my pc or maybe a connection problem to the internet?

any ideas as to what you may think is the problem would be greatly appreciated!

Thank You

Latest posts

Top Bottom