2

Hi i have create a program in C# to receive a string of ASCII the string is an XML markup.

The computer i receive the data from i have no control over and does not accept a response it sends out data on the COM port about every 10 mins

The Console app i have collects and stores this data but it does not always work i would say about 50% of the time data is missing like a packet or byte was lost and the XML string wont read into XmlDocument

I have been trying for about a week now to make this more stable but this is my first time in C# and would like some help is there anyway to improve this.

CODE

  class SerialPortProgram : IDisposable
  {
    // Create the serial port with basic settings
    private SerialPort port = new SerialPort("COM1",
       115200, Parity.None, 8, StopBits.One);

    string sBuffer = null;
    string filePath1 = @"C:\Data\data1.xml";
    string filePath2 = @"C:\Data\data2.xml";

    [STAThread]
    static void Main(string[] args)
    {
        // Instatiate this class
        new SerialPortProgram();

    }

    private SerialPortProgram() 
    {
        Console.WriteLine("Started Data Monitoring:");

        //Attach a method to be called when there
        //is data waiting in the port's buffer
        port.ReadBufferSize = 20971520;
        port.ReceivedBytesThreshold = 1;
        port.DataReceived += new SerialDataReceivedEventHandler(Port_DataReceived);

        //Begin communications
        port.Open();

        //Enter an application loop to keep this thread alive
        Application.Run();

    }

    private void Port_DataReceived(object sender, SerialDataReceivedEventArgs e)
    {
        //Show all the incoming data in the port's buffer
        SerialPort sp = (SerialPort)sender;
        sBuffer += sp.ReadExisting();

        if (sBuffer.Length > 26000) // check the file size
        {
            if (sBuffer.Substring(sBuffer.Length - 6) == "</xml>") // check for end of file
            {
                Console.WriteLine("Found: Processing...");
                //Thread.Sleep(1000);
                ProcessXML();
                sBuffer = null;
                Console.WriteLine("Done!");
                DateTime now = DateTime.Now;
                Console.WriteLine(now);
                Console.WriteLine("Monitoring...");
            }
            else
            {
                Console.WriteLine("Still Receiving Data: " + sBuffer.Length);
            }
        }
        else
        {
            Console.WriteLine("Receiving Data: " + sBuffer.Length);
        }
    }
 private void ProcessXML()
    {
        XmlDocument xmlDoc = new XmlDocument();
        try
        {
            xmlDoc.LoadXml("<wrapper>" + sBuffer + "</wrapper>");
            int index = 0;
            XmlNodeList xnl = xmlDoc.SelectNodes("wrapper/xml");
            foreach (XmlNode node in xnl)
            {
                // Console.WriteLine(index.ToString());
                if (index == 0)// xml file 1
                {
                    using (XmlReader r = new XmlNodeReader(node))
                    {
                        DataSet ds = new DataSet();
                        ds.ReadXml(r);
                        ds.WriteXml(filePath1);
                        Console.WriteLine("NEW Data1");
                        ds.Dispose();
                        var db = new Database();
                        db.SaveMetersToDatabase(ds);
                    }
                }
                else if (index == 1)// xml file 2
                {
                    using (XmlReader r1 = new XmlNodeReader(node))
                    {
                        DataSet dst = new DataSet();
                        dst.ReadXml(r1);
                        dst.WriteXml(filePath2);
                        Console.WriteLine("NEW Data2");
                        dst.Dispose();
                    }
                }

                index++;
            }
        }
        catch
        {
            Console.WriteLine("Error: in data");
            try
            {
                string now = DateTime.Now.ToString("yyyyMMddHHmmss");
                System.IO.File.WriteAllText(@"C:\Data\log" + now + ".xml", "<wrapper>" + sBuffer + "</wrapper>");
            }
            catch
            {
                Console.WriteLine("Failed to write to log");
            }
        }
    }

    protected virtual void Dispose(bool disposing)
    {
        if (disposing && port != null)
        {
            port.Dispose();
            port = null;
        }
    }

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

  }
}

UPDATED CODE:

  private void Port_DataReceived(object sender, SerialDataReceivedEventArgs e)
    {
        //Show all the incoming data in the port's buffer
        SerialPort sp = (SerialPort)sender;
        sBuffer += sp.ReadExisting();

        new Thread(() =>
        {
            Thread.CurrentThread.IsBackground = true;
            if (sBuffer.Length > 25000) // check the file size
            {
                if (sBuffer.Substring(sBuffer.Length - 6) == "</xml>") // check for end of file
                {
                    Console.WriteLine("Found: Processing...");
                    Task.Run(() =>
                    {
                        ProcessXML();
                        sBuffer = null;
                        Console.WriteLine("Done!");
                        DateTime now = DateTime.Now;
                        Console.WriteLine(now);
                        Console.WriteLine("Monitoring...");
                    });
                }
                else
                {
                    Console.WriteLine("Still Receiving Data: " + sBuffer.Length);
                }
            }
            else
            {
                Console.WriteLine("Receiving Data: " + sBuffer.Length);
            }
        }).Start();
    }

UPDATE

still having this issue could it be possible that the sending computer sometimes does not send all the data or there is packet loss i have tried everything i have tried this new code below using Serial Port BaseStream BeginRead

  private SerialPortProgram() 
    {
        Console.WriteLine("Started Data Monitoring:");

        //Attach a method to be called when there
        try
        {
            Port.BaudRate = 115200;
            Port.DataBits = 8;
            Port.Parity = Parity.None;
            Port.StopBits = StopBits.One;
            Port.Handshake = Handshake.None;
            Port.DtrEnable = true;
            Port.NewLine = Environment.NewLine;
            Port.ReceivedBytesThreshold = 2048;
            Port.Open();

            byte[] buffer = new byte[35000];
            Action StartRead = null;
            StartRead = () => {
                Port.BaseStream.BeginRead(buffer, 0, buffer.Length, async (IAsyncResult ar) =>
                {
                    try
                    {
                        int actualLength = Port.BaseStream.EndRead(ar);
                        byte[] received = new byte[actualLength];
                        Buffer.BlockCopy(buffer, 0, received, 0, actualLength);
                        await Task.Run(() =>
                        {
                            sBuffer += Encoding.ASCII.GetString(received);
                            CheckBuffer();
                        });
                    }
                    catch (Exception exc)
                    {
                        Console.WriteLine(exc);
                    }
                    StartRead();
                }, null);
            };
            StartRead();

        }
        catch (Exception ex)
        {
            Console.WriteLine("Error accessing port." + ex);
            Port.Dispose();
            Application.Exit();
        }

        //Enter an application loop to keep this thread alive
        Application.Run();
    }

    private void CheckBuffer()
    {
        if (sBuffer != null && sBuffer.Length > 26000) // check the file size
        {
            if (sBuffer.Substring(sBuffer.Length - 6) == "</xml>") // check for end of file
            {
                new Thread(async () =>
               {
                   Console.WriteLine("Found: Processing...");

                   await Task.Run(() => ProcessXML());

                   sBuffer = null;
                   Console.WriteLine("Done!");
                   DateTime now = DateTime.Now;
                   Console.WriteLine(now);
                   Console.WriteLine("Monitoring...");
               }).Start();
            }
            else
            {
                Console.WriteLine("Still Receiving Data: " + sBuffer.Length);
            }
        }
        else if (sBuffer != null && sBuffer.Length > 0)
        {
            Console.WriteLine("Receiving Data: " + sBuffer.Length);
        }

    }
12
  • what's the sleep for? that's just asking to lose data Jul 31, 2017 at 4:02
  • yeah ok i didnt think it would affect it it was there as it was saving the file a reopening it but no longer needed have removed
    – Anthony
    Jul 31, 2017 at 4:07
  • 2
    With communications, you want to minimise the amount of time in your callback else you will lose data. I suggest you use a circular buffer or similar technique and move your processing of said buffer to another thread
    – user585968
    Jul 31, 2017 at 4:15
  • Please take a look at my anwer here stackoverflow.com/questions/33310396/… and tell if the problem persists while using the control mentioned in the answer.
    – Peuczynski
    Jul 31, 2017 at 5:37
  • Oh, and you should use much smaller buffer, receive all the data, put it on queue or sth and then analyze the data (when the data is present on your side). Consider one thread for receiving the data only so you won't experience any lags (remember - single responsibility) and another for parsing it. I've always separated it into two classes - Receiver, and Processor. I can show you sample implementation if you'd like
    – Peuczynski
    Jul 31, 2017 at 5:46

2 Answers 2

1

This is my implementation. You should get the general idea

First - get the data from the connection

internal class SerialListener : Listener
{
    private SerialPort sp;
    private ConnectionInfo _connection;
    private Timer _listenerTimer;
    private bool should_exit = false;
    private bool busy = false;
    ConcurrentQueue<byte> fifo_peekonly = null;
    BlockingCollection<byte> fifo_queue = null;

    public SerialListener(ConnectionInfo connection)
        : base(connection)
    {
        _connection = connection;
        InitSerialConnection();
    }

    private void InitSerialConnection()
    {
        sp = new SerialPort(_connection.ifname_ip);
        sp.BaudRate = _connection.baudrate_port;
        sp.Parity = _connection.parity;
        sp.DataBits = _connection.charactersize;
        sp.StopBits = _connection.stopbits;
        sp.Handshake = _connection.flowcontrol;
        sp.DtrEnable = true;
        sp.ReadTimeout = 100;
        sp.Open();
        fifo_peekonly = new ConcurrentQueue<byte>();
        fifo_queue = new BlockingCollection<byte>(fifo_peekonly);
        sp.DataReceived += (sender, e) => 
        { 
            byte[] buffer = new byte[sp.BytesToRead];
            if (!sp.IsOpen)
            {
                throw new System.InvalidOperationException("Serial port is closed.");
            }
            sp.Read(buffer,0,sp.BytesToRead);
            foreach (var b in buffer)
                fifo_queue.Add(b);
        };
    }

    public override byte GetByteFromDevice()
    {
        byte b;
        b = fifo_queue.Take();
        return b;
    }

    public override byte PeekByteFromDevice ()
    {
        byte b;
        bool peeked = false;
        do {
            peeked = fifo_peekonly.TryPeek(out b);
            if (!peeked)
                Thread.Sleep(100);
        } while (!peeked);
        return b;
    }

    public override void Close()
    {
        base.Close();
        sp.DiscardInBuffer();
        sp.DiscardOutBuffer();
        Thread.Sleep(3000);
        sp.Close();
    }
}

Then implement another class that will call the GetByteFromDevice method. Unfortunatelly the code is too specific to bo posted here as it would be only confusing for you.

So - separate the logic, and push all the bytes in real time, and in a separate thread (some loop, timer etc.) get the bytes read previously and then analyze them.

Also the method you are using for xml detection is not optimal in my opinion. I have implemented detecting the start and end sequence in the stream. Let me know if you'd like to look at the code

6
  • thanks for you help but i was not successful using some of this code in my current program
    – Anthony
    Aug 3, 2017 at 3:28
  • I have accepted this answer I have now got this code working after some changes and it have fix my problem. If you have code for xml detection that would good to see
    – Anthony
    Aug 20, 2017 at 5:05
  • 1
    I don't have such code but I can tell you how I'd do it. You have all the characters on the queue you can peek the queue to monitor the xml start and then trace the opening vs closing tags. Look for similar questions like this one stackoverflow.com/questions/891223/better-way-to-detect-xml.
    – Peuczynski
    Aug 20, 2017 at 11:17
  • Thanks for your help I had tried a lot of different code for serial port read over the past month this was the only code I have got to work fast and flawless every time
    – Anthony
    Aug 20, 2017 at 11:35
  • IMO the only thing that could be not optimal in your solution is that you had is that you had big buffer instead of discarding what you didn't need
    – Peuczynski
    Aug 20, 2017 at 11:35
1

Maybe it's not exactly the desired answer,

but I am sharing the solution to the data evasion that I encountered. Actually, I was not escaping data. In the SerialPort default settings, ReadBufferSize is set to 4096. When data packet was over 4096 Bytes it was ignoring other data and was not displayed in DataReceived.

Setting the ReadBufferSize to a value that suited me fixed the issue.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.