Question Linq: Group items that have equal elements > 200

[lyex81]

I'm using the below code to determine the number of equal elements in two lists (x of 256):

    List<string> secretData1 = new List<string>() { "orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","orange","apple","apple","apple","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple" };
    List<string> secretData2 = new List<string>() { "orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","orange","orange","apple","orange","orange","apple","orange","orange","orange","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","orange","orange","orange","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange" };
    
    int equalElements = secretData1.Zip(secretData2, (i, j) => i == j).Count(eq => eq);     
    Console.WriteLine(equalElements);

Equal Elements (Output): 114

Now, I want to convert this code on a larger scale for a single list.

public class SecretAgents
{
    public string Name {get; set;}
    public List<string> SecretData {get; set;}
    public int GroupID {get; set;}
}

    List<SecretAgents> secretAgents = new List<SecretAgents>();
    
    secretAgents.Add(new SecretAgents { Name= "SecretAgent 1", GroupID= 0, SecretData= new List<string> { "orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","orange","apple","apple","apple","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple" } });
    secretAgents.Add(new SecretAgents { Name= "SecretAgent 2", GroupID= 0, SecretData= new List<string> { "orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","orange","orange","apple","orange","orange","apple","orange","orange","orange","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","orange","orange","orange","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange" } });
    secretAgents.Add(new SecretAgents { Name= "SecretAgent 3", GroupID= 0, SecretData= new List<string> { "orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","apple","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","orange","apple","apple","apple","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple" } });
    secretAgents.Add(new SecretAgents { Name= "SecretAgent 4", GroupID= 0, SecretData= new List<string> { "orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","orange","orange","apple","orange","orange","apple","orange","orange","orange","apple","apple","apple","orange","orange","orange","orange","apple","apple","apple","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","orange","orange","orange","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","apple","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange","apple","orange","orange","orange","orange","apple","apple","orange","orange","orange","orange","orange","orange","orange","orange","orange"} });
    secretAgents.Add(new SecretAgents { Name= "SecretAgent 5", GroupID= 0, SecretData= new List<string> { "apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","apple","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","apple","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","apple","orange","orange","apple","apple","apple","apple","apple","apple"} });

I want to group SecretAgents that have equal elements > 200.

Expected Output:

• Group 1: SecretAgent 1 & SecretAgent 3 (Reason: Equal elements: 248)
• Group 2: SecretAgent 2 & SecretAgent 4 (Reason: Equal elements: 256)
• Group 3: SecretAgent 5

Any help is appreciated. Thanks.

 

[Skydiver]

The main issue is that you are trying to do two separate things and calling it one thing. The two problems that you need to solve are:
1) Compare the SecretData with every other SecretAgents within the your list of SecretAgents.
2) Group by the count of matches resulting from #1 above.

Right now its not clear exactly clear what rules you followed to get that specific expected result. Why is secret agent 5 alone? It seems l like he has at least some matches with some of the other agents.

Instead of dealing with 256 items for each secret data, let's look at just 5 items for each secret data:

1: OOOOa
2: OOOaa
3: OOaaa
4: Oaaaa
5: aaaaa
6: zzzzz

Is this a valid result:
Group 1: contains 1, 2, 3, 4, 5 because of 4 matches
1 and 2
2 and 3
3 and 4
4 and 5

Group 2: contains 1, 2, 3, 4, 5 because of 3 matches:
1 and 3
2 and 4
3 and 5

Group 3: contains 1, 2, 4, 5 because of 2 matches:
1 and 4
2 and 5

Group 5: contains 1, 5 because of 1 match:
1 and 5

Group 6: contains 1, 2, 3, 4, 5, 6 because of 0 matches:
1 and 6
2 and 6
3 and 6
4 and 6
5 and 6

 

[JohnH]

An agent may occur in multiple groups if it shares the same amount of equal elements.

First generate the combinations (1-2, 1-3, 1-4 etc), this Linq expression generates each agent pair to test:

var combinations = secretAgents.SelectMany((value, index) => secretAgents.Skip(index + 1), (first, second) => new[] { first, second });

The equalElements comparison is here an inline function, that is used in following code:

int equalElements(List<string> A, List<string> B) => A.Zip(B, (i, j) => i == j).Count(eq => eq);

Then another Linq expression to compare, filter and group:

var groups = combinations.Select(a => new
{
    Count = equalElements(a[0].SecretData, a[1].SecretData),
    Agents = a
})
    .Where(a => a.Count > 200)
    .GroupBy(a => a.Count)
    .OrderByDescending(g => g.Key)
    .Select(g => new
    {
        Count = g.Key,
        Agents = g.SelectMany(g2 => g2.Agents).Distinct().ToArray()
    });

The first select creates an anonymous type with the count of equal elements and the two agents compared.
Where filters for count>200, then sorts and groups with the agents of each pair in group flattened and distinct. Without the last Select (line 9-13) to flatten and distinct the groups would contain each matching pair if that is of interest.

Non-grouped agents can be found like this:

var nongrouped = secretAgents.Except(groups.SelectMany(g => g.Agents).Distinct());

Debug output:

foreach (var group in groups)
{             
    Debug.Write($"{group.Count}: ");         
    var names = group.Agents.Select(a => a.Name);
    Debug.WriteLine(string.Join(", ", names));
}
Debug.Write($"non-grouped: ");
Debug.WriteLine(string.Join(", ", nongrouped.Select(a => a.Name)));

256: SecretAgent 2, SecretAgent 4
248: SecretAgent 1, SecretAgent 3
non-grouped: SecretAgent 5

 

[JohnH]

An agent may occur in multiple groups if it shares the same amount of equal elements.

If you for example copy "SecretAgent 1" and call it "SecretAgent 1b" you will see this output:

256: SecretAgent 1, SecretAgent 1b, SecretAgent 2, SecretAgent 4
248: SecretAgent 1, SecretAgent 3, SecretAgent 1b
non-grouped: SecretAgent 5

Now 1 and 1b also has 256 equal elements (but they only have 114 same as 2 and 4), and they are also in the 248 group as before compared to 3.

 

[Skydiver]

Or more interesting, instead of adding Agent 1b, change the criteria from 200 down to 100.

256: SecretAgent 2, SecretAgent 4
248: SecretAgent 1, SecretAgent 3
156: SecretAgent 1, SecretAgent 5, SecretAgent 3
114: SecretAgent 1, SecretAgent 2, SecretAgent 4
112: SecretAgent 2, SecretAgent 3, SecretAgent 4
non-grouped:

 

[lyex81]

Thank you so much for your help John and Skydiver! I really appreciate both of you taking your valuable time to help me