Sora Innosia WebScrapper

innosia

New member
Joined
Dec 2, 2013
Messages
3
Programming Experience
5-10
Imagine you usually write this to scrape a website
C#:
[COLOR=#4CB6D0][FONT=Courier New]WebClient[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] wc = [/FONT][/COLOR][COLOR=blue][FONT=Courier New]new[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] [/FONT][/COLOR][COLOR=#4CB6D0][FONT=Courier New]WebClient[/FONT][/COLOR][COLOR=#333333][FONT=Courier New]();[/FONT][/COLOR]
[COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] googlecom = wc.DownloadString([/FONT][/COLOR][COLOR=#800000][FONT=Courier New]"http://www.google.com"[/FONT][/COLOR][COLOR=#333333][FONT=Courier New]);[/FONT][/COLOR]
[COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New][] links = GetLinks(googlecom);            [/FONT][/COLOR]
[COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] LINK1 = GetLink(links[5]);[/FONT][/COLOR]
[COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] LINK2 = links[6];[/FONT][/COLOR]
[COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] newData = wc.DownloadString(LINK1);[/FONT][/COLOR]

[COLOR=blue][FONT=Courier New]public[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] [/FONT][/COLOR][COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New][] GetLinks([/FONT][/COLOR][COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] content)[/FONT][/COLOR]
[COLOR=#333333][FONT=Courier New]{[/FONT][/COLOR]
[COLOR=green][FONT=Courier New]// ... Parsing of links[/FONT][/COLOR]
[COLOR=#333333][FONT=Courier New]}[/FONT][/COLOR]

[COLOR=blue][FONT=Courier New]public[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] [/FONT][/COLOR][COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] GetLink([/FONT][/COLOR][COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] content)[/FONT][/COLOR]
[COLOR=#333333][FONT=Courier New]{[/FONT][/COLOR]
[COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] search = [/FONT][/COLOR][COLOR=#800000][FONT=Courier New]"onclick=gbar.logger.il(1,{t:5}); class=gbzt id=gb_5 href=\""[/FONT][/COLOR][COLOR=#333333][FONT=Courier New];[/FONT][/COLOR]
[COLOR=blue][FONT=Courier New]int[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] startIndex = content.IndexOf(search);[/FONT][/COLOR]
[COLOR=blue][FONT=Courier New]int[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] endIndex = content.IndexOf([/FONT][/COLOR][COLOR=#800000][FONT=Courier New]"\""[/FONT][/COLOR][COLOR=#333333][FONT=Courier New], startIndex + search.Length);[/FONT][/COLOR]
[COLOR=blue][FONT=Courier New]return[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] content.Substring(startIndex + search.Length, endIndex - (startIndex + search.Length));[/FONT][/COLOR]
[COLOR=#333333][FONT=Courier New]}[/FONT][/COLOR]

But using WebScrapper found in Sora Innosia - Projects List
You can write as simple as
C#:
[COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] syn = [/FONT][/COLOR][COLOR=#800000][FONT=Courier New]"SetResult('LINK1,LINK2', TagMatch(Filter(TagMatch(Download('http://www.google.com'), '<a', ''), '5,6'), 'onclick=gbar.logger.il(1,{t:5}); class=gbzt id=gb_5 href=\"', '\"'));Download(GetResult('LINK1'))"[/FONT][/COLOR][COLOR=#333333][FONT=Courier New];[/FONT][/COLOR]
[COLOR=#333333][FONT=Courier New]WebScrapper.[/FONT][/COLOR][COLOR=#4CB6D0][FONT=Courier New]Scrapper[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] scr = [/FONT][/COLOR][COLOR=blue][FONT=Courier New]new[/FONT][/COLOR][COLOR=#333333][FONT=Courier New] WebScrapper.Scrapper();[/FONT][/COLOR]
[COLOR=blue][FONT=Courier New]string[/FONT][/COLOR][COLOR=#333333][FONT=Courier New][] result = scr.Multiple(syn);[/FONT][/COLOR]

WebScrapper is a C like Language interpreter that has special predefined function that specially use for uploading and downloading of files in the internet
Overview : Sora Innosia - WebScrapper
Download : Sora Innosia - WebScrapper - Download
Sample : Sora Innosia - WebScrapper - Sample 4

And much more. If you interest of the project, give you some opinions
 

Latest posts

Top Bottom