LINQ Query in Parser
- Thread starter patrick
- Start date
Skydiver
Staff member
Export the list into a CSV. Then have Sylvan read that new CSV.
Skydiver
Staff member
Skydiver
Staff member
I don't have the documentation for that library on front of me, but I thought I saw somewhere that the Sylvan takes a stream to read from. So in theory, you could create a stream that contains the data formatted as a CSV, and pass the stream to Sylvan. So technically, you didn't create another CSV file, but you have a virtual CSV file in memory. As I recall, though, @Anwind and you were trying to prevent loading all the CSV data into memory, so now you are at an impasse.
The amount of coding you would need to do to only have a small window of the data in memory at a time is going to be a pretty big undertaking.
The problem compounds so much more because your intermediate list will also be another copy of the CSV data that will be in memory. You'll need to also figure out how not to load all of that into memory, but somehow be able to perform your LINQ queries against that list. If you use any kind of sorting in your query, you won't be able to use the LINQ
The amount of coding you would need to do to only have a small window of the data in memory at a time is going to be a pretty big undertaking.
The problem compounds so much more because your intermediate list will also be another copy of the CSV data that will be in memory. You'll need to also figure out how not to load all of that into memory, but somehow be able to perform your LINQ queries against that list. If you use any kind of sorting in your query, you won't be able to use the LINQ
OrderBy() because it keeps references to all the objects that need to be sorted in memory. You'll need to go back to your data structures and algorithms textbook to implement external data sorting.
Skydiver
Staff member
I know nothing about
I would point out though that your standing requirements has been that in the end you want a
Sylvan.Data and what facilities it provides. You are on your own. Perhaps you should read the documentation.I would point out though that your standing requirements has been that in the end you want a
CsvDataReader. With your code in post #9, you end up with a DbDataReader. In the past @Anwind at one point was willing to accept any kind of DbDataReader, and then later when she found the code offered to her that wrapped a list or enumerable to present a DbDataReader she said she didn't want to deal with lambdas, but found it too tedious to convert the code to not use lamdas, so she said she was willing to work with List directly. When offered up code which dealt with List, she then reverted back to wanting a CsvDataReader. Hopefully, you'll stand firm with your requirement that you want a CsvDataReader.
Last edited:
