Sort, Orderby

[patrick]

Not

 

[Skydiver]

Check the documentation and source code. Sylvan.Data.Csv does not, but Sylvan.Data may. I'm not familiar with the latter.

 

[Skydiver]

I also see that you seemed to have softened your hard requirement that the the return type had to be a CsvDataReader. Yesterday evening, you started bending towards allowing DbDataReader. Now it looks like you are willing to go even one more step up the inheritance tree and go up to IDataReader.

If you are willing to go all the way up to IEnumerable you could potentially use Cast<T>() and OrderBy<T>(). But if you are willing to use Cast<T>(), you might as well have just used the automatic class mapping that the Sylvan CsvDataReader provides.

 

[Skydiver]

No. I was implying that you could have something like the following pseudo-code:

IEnumerable<Person> persons = CsvDataReader("persons.csv).Cast<Person>().OrderBy(p => p.Age);

or alternatively:

IEnumerable<Person> persons = CsvDataReader("persons.csv).GetRecords<Person>().OrderBy(p => p.Age);

 

[Skydiver]

What does your compiler and tests tell you?

 

[Skydiver]

I mean does the code compile successfully? If it does compile successfully, does it behave as expected when you test it?

 

[cjard]

I wish the parser was faster

Probably user error, given that Mark's tested it to be the fastest already - GitHub - MarkPflug/Benchmarks: Benchmarks for various .NET libraries.

 

[cjard]

I really would prefer you to just tell me what needs to be done, using language that would e.g. come from your boss' non technical perspective..

Expecting an answer like "use an old pentium 4 with 8 megabytes of ram to stream a 17 column, half million row CSV weighing 958 megabytes, as JSON to a remote API, and not include any records that have a status of 4"

 

[Skydiver]

If the compiler compiles the code, then the syntax is correct. (The logic behind the code may not be, but as far as the compiler is concerned, the syntax is correct.)

 

[Skydiver]

Yes. Actually step back and explain the full problem that you are trying to solve rather than the artificial constraints that you have to needing to read from a CSV file, and not wanting it use SQLite, and needing a CsvDataReader interface to talk to.

This is an XY Problem and you don't want to tell us what the problem X that you are trying to solve. You keep insisting on trying to solve problem Y.

The XY Problem

Asking about your attempted solution rather than your actual problem

xyproblem.info

 

[Skydiver]

You are still only addressing problem Y. What is is problem X?

 

[Skydiver]

To tell us about problem X, you need to use descriptive words because that what problem X is going to be. You are too focused on your problem Y of trying to use a CSV reader, do some sorting, and then still get back a CSV reader.

We are basically just going around in circles right now about your problem Y.

 

[Skydiver]

And just to get you off your current obsession about the syntax or grammar being wrong with your post #22, here is proof working code:

Can we please stop talking about your Problem Y. There is no problem. The code in screenshot could have been simply:

using System;
using Sylvan.Data;
using Sylvan.Data.Csv;
using System.Collections.Generic;
using System.Linq;

using var csvReader = CsvDataReader.Create("data.csv");
IEnumerable<Planet> planets = csvReader.GetRecords<Planet>();
IEnumerable<Planet> sortedPlanets = planets.OrderBy(r => r.Size);

foreach(var planet in sortedPlanets)
    Console.WriteLine($"Name: {planet.Name}, Size: {planet.Size}");

class Planet
{
    public string Name { get; set; }
    public int Size { get; set; }
}

There was no need to go back to a DbDataReader or IDataReader.

Please tell us about your Problem X. What problem are you actually trying to solve?

 

[patrick]

thank you because of you. I solved

 

[Skydiver]

No, you have not resolved the problem. Very shortly you are going to reply or open a new thread that you are using too much memory because as I recall you have 43 columns and enough rows to fill up 10 GB. All of that will have to be loaded into memory to use the LINQ OrderBy().

As I told @Anwind, doing a code migration has nothing to do with changing from using SQLite to switch to using the ACE, JET, or Sylvan CSV reader. Since at one time you see considering using ACE or JET, that means that your target platform is Windows. Since SQLite works on Windows, you are actually losing functionality by reading straight from the CSV instead importing the data into SQLite. Furthermore, she/you are now imposing a "usage" tax on your users. Instead of paying the one time cost of importing the data into SQLite, now your users have to pay the cost of always reading from the CSV all into memory and sorting every time.