Processing Large Xml Files

[inkedGFX]

I am processing large xml files and need an efficient way to load the xml into memory and save the updates....what I am doing is getting data from web pages each page has 30 or more fields , once I have all the fields the program loads the xml with XDocument load....it then saves all the fields ..then goes to the next web page and gets the fields and then loads the xml again and saves it.

is there a way to load the xml once and then continue to save the new fields to it?

below is the code to load and save the xml

 XDocument spiderXml = XDocument.Load(xmlPath);
                            XElement Listing =  new XElement("Listing",          
                                                              new XElement("Position", pos),
                                                              new XElement("ID", splitID[1]),
                                                              new XElement("Title", carTitleFinal),
                                                              new XElement("ListingImages",
                                                                  new XElement("Img0", imageListOne[0]),
                                                                  new XElement("Img1", imageListOne[1]),
                                                                  new XElement("Img2", imageListOne[2])),
                                                              new XElement("Dealer",
                                                                  new XElement("DealerName", dName),
                                                                  new XElement("DealerAddress", dealerAddress),
                                                                  new XElement("DealerPhoneNumber", dPhoneNumber),
                                                                  new XElement("DealerImageLink", dealerImageFinal),
                                                                  new XElement("DealerLink", DealerLink)),
                                                              new XElement("Description", descFinal),
                                                              new XElement("StockNumber", stockNumber),
                                                              new XElement("Price", price),
                                                                new XElement("VehicleLocation",
                                                                  new XElement("Country", country),
                                                                  new XElement("CityState", cityState),
                                                                  new XElement("Zipcode", zip)),
                                                                new XElement("VehicleBasics",
                                                                  new XElement("TrimLevel", trim),
                                                                  new XElement("Odometer", odometer),
                                                                  new XElement("Year", year),
                                                                  new XElement("Vin", vin),
                                                                  new XElement("Style", style),
                                                                  new XElement("Convertible", convertible)),
                                                                new XElement("ExteriorDetails",
                                                                  new XElement("ExteriorColor", exteriorColor),
                                                                  new XElement("ExteriorRestorationHistory", exteriorRestorationHistory),
                                                                  new XElement("ExteriorCondition", exteriorCondition)),
                                                                new XElement("InteriorDetails",
                                                                  new XElement("InteriorColor", interiorColor),
                                                                  new XElement("SeatMaterial", interiorSeatMaterial),
                                                                  new XElement("InteriorCondition", interiorCondition)),
                                                                new XElement("EngineDetails",
                                                                  new XElement("EngineConfig", engineConfig),
                                                                  new XElement("EngineSize", engineSize),
                                                                  new XElement("EngineModifications", engineMods),
                                                                  new XElement("EngineHistory", engineHistory),
                                                                  new XElement("EngineCondition", engineCondition)),
                                                                new XElement("TransmissionDetails",
                                                                  new XElement("TransmissionType", transType),
                                                                  new XElement("DriveTrain", driveTrain)));
                                                              
                            
                            spiderXml.Root.Add(Listing);
                            spiderXml.Save(xmlPath);
                        }

Thank You for any help
-InkedGFX

 

[wim sturkenboom]

If all fields are defined, you can consider the use of xml serialisation and deserialisation.

You define classes like Dealer, VehicleLocation, EngineDetails etc; for anything that is like an array (e.g. the images) I would use a List<> (List<string> for the images). You combine those into the class Listing.

Assuming you can have multiple listings in one xml, create a class for that contains a member List<Listing>.