Sending list via TCP

[MattNorman]

I currently have my app set up in a way which the server imports data from CSV files and writes it to the database. The clients then read from the database rather than request data from the application server. This works fine however is quite inefficient as the data is being written and read every 3 seconds.

I would like to switch over to a setup such as:

1. Server imports data from CSV and holds in memory every 3 seconds
2. Clients connect to server whenever they need to register for data retrieval
3. Server stores a list of clients to send data to
4. Server opens connection to all clients and send generic list data

This is quite new to me and after a bit of research I am a little lost.

Is anyone able to point me in the right direction or to any useful articles?

Regards
Matt

 

[Skydiver]

Sounds like a great fit for using SignalR.

I know that in your subject line you specifically asked for TCP, but HTTP and WebSockets are built on top of TCP. It's perfect for realtime push model that you seem to be leaning towards.

If you really want to go down to the TCP level, but still avoid directly dealing with TCP, you may want to look into using NetMQ and in particular it's pub-sub support. The great part of this library is that it deals with most of the problems of having to do the fault tolerance and handling timing of getting things setup. The downside of NetMQ is that it is frame based, not stream based, so you'll need to figure out how to send your list data within those frames. It's not an insurmountable issue since it supports multiple frames per message, and it's really easy to use.

If you really want to use raw TCP, then there is great support in the .NET with the TcpClient and TcpListener classes. It's also going to be an easier learning curve to get an something up and running.

 

[Skydiver]

These maybe interesting reads:

SignalR + RX (Streaming Data Demo App 1 of 2)

Small demo app showing how to create a streaming Publisher / Resilient clients using SignalR + RX

www.codeproject.com

NetMQ+ RX (Streaming Data Demo App 2 of 2)

Small demo app showing how to create a streaming Publisher / Resilient clients using NetMQ + RX

www.codeproject.com

RX maybe overkill for your needs, but these two projects interesting followed my same recommendations about looking at either SignalR or NetMQ for publishing data.

 

[MattNorman]

These maybe interesting reads:

SignalR + RX (Streaming Data Demo App 1 of 2)

Small demo app showing how to create a streaming Publisher / Resilient clients using SignalR + RX

www.codeproject.com

NetMQ+ RX (Streaming Data Demo App 2 of 2)

Small demo app showing how to create a streaming Publisher / Resilient clients using NetMQ + RX

www.codeproject.com

RX maybe overkill for your needs, but these two projects interesting followed my same recommendations about looking at either SignalR or NetMQ for publishing data.

Thanks I will take a look at these

 

[MattNorman]

I decided to go with the NetworkCommsDotNet library for this as it looked very simple to implement.

I have everything working fine when running both the client and server on the same machine. As soon as I run them separately I get issues around the connection being closed or already closed.

This is quite an old library so not sure if anyone knows much about it anymore.

I basically have it set up as follows:

• Server starts by mapping a packet header to a method that will run whenever packets are received
• Server then starts listening on all local IP's port 9999
• Client starts by mapping packet header to a method that will run whenever packets are received
• Client sends a registration message to the server with it's own IP/Port info
• Server adds the client to a list
• Server periodically sends data to all clients in the list

It's the last step that fails when run on a separate server. I can see that the client is correctly listening on the required port.

It's a little confusing as the exception is around the connection being closed however there are no steps in the implementation where a connection is manually opened/closed.

There is a NetworkComms.Shutdown method however I am only using this when the app is closed.

Exception:

Exception encountered: NetworkCommsDotNet.CommunicationException: Attempting to send packet on connection which has been closed or is currently closing.
   at NetworkCommsDotNet.Connections.Connection.SendPacket[packetPayloadObjectType](IPacket packet, Int64& packetSequenceNumber)
   at NetworkCommsDotNet.Connections.Connection.SendPacket[packetPayloadObjectType](IPacket packet)
   at NetworkCommsDotNet.Connections.Connection.SendObject[sendObjectType](String sendingPacketType, sendObjectType objectToSend, SendReceiveOptions options)
   at NetworkCommsDotNet.Connections.Connection.SendObject[sendObjectType](String sendingPacketType, sendObjectType objectToSend)
   at NetworkCommsDotNet.NetworkComms.SendObject[sendObjectType](String packetTypeStr, String destinationIPAddress, Int32 destinationPort, sendObjectType sendObject)
   at ApexCCM.Services.DataCollectorCMSRealTimeService2.<>c__DisplayClass41_2.<SendAgentDataToClients>b__0()

If anyone is familiar with this library and thinks they might be able to help, I can post the relevant code.

Regards
Matt

 

[Skydiver]

You should post a question in their support forums:

Forums

networkcomms.net

 

[MattNorman]

Managed to get this resolved now. For some reason the firewall in between the server and client was allowing traffic in one direction but not the other.