2

I'm writing a project using an STM32F407_VG board that uses an RS232 connection to send batch of data of different sizes (~400 bytes) on a serial port, and those data must be written on file. On desktop side I'm using a python 3 script with pyserial 3.3.

I've tried reading a single byte at time with ser.read() but I think it's too slow because I'm losing some of the data. So I'm trying to send the size of the batch as an integer before the batch itself, in order to reduce the overhead, and write data to file during time interval within a batch and the following one.

PROBLEM IS: ser.read(n) behave in a very strange way, and 99% of the times it blocks when it's time to read the batch and do not return. It also happens that sometimes it can read the first batch and writes it to file successfully, but it blocks at the second loop iteration. It's strange because I can use ser.read(4) to get the batch size with zero problem, and I use ser.readline() at the beginning of the script when listening to a starting signal, but I cannot read the data.

I'm sure that data are there and are well formed because I checked with a logic analyzer, and I've already tried with enabling and disabling flow control or set different baud rates on both the board and the script. I think it could be a config problem of python, but actually I've run out of ideas.


PYTHON SCRIPT CODE -SNIPPET

ser = serial.Serial(str(sys.argv[1]),           \
                int(sys.argv[2]),               \
                stopbits=serial.STOPBITS_ONE,   \
                parity=serial.PARITY_NONE,      \
                bytesize=serial.EIGHTBITS,      \
                timeout=None                    \
                )
outputFile = open(sys.argv[3],"wb")

# wait for begin string
beginSignal = "ready"
word = ""
while word != beginSignal:
   word = ser.readline().decode()
   word = sample.split("\n")[0]
print("Started receiving...")

while True:
   # read size of next batch
   nextBatchSize = ser.read(4)
   nextBatchSize = int.from_bytes(nextBatchSize,byteorder='little', signed=True)
   # reads the batch: 
   # THIS IS THE ONE THAT CREATES PROBLEMS
   batch = ser.read(nextBatchSize)
   # write data to file
   outputFile.write(batch)

BOARD CODE - SNIPPET

// this function sends the size of the batch and the batch itself
void sendToSerial(unsigned char* mp3data, int size){
   // send actual size of the batch
   write(STDOUT_FILENO,&size,sizeof(int));
   // send the batch of data
   write(STDOUT_FILENO,mp3data,size);
}

Any idea? Thanks!

1 Answer 1

0

You might have already tried this, but print out nextBatchSize to confirm that it is what you expect, just in case the byte order is reversed. If this is wrong your Python code could be trying to read too many bytes, and would therefore block.

Also you can check ser.in_waiting to see how many bytes are available to be read from the input buffer before your read attempt:

print(ser.in_waiting)
batch = ser.read(nextBatchSize)

You should also check the return value of write() in your C code, which is the number of bytes actually written, or -1 if there is an error. write() is not guaranteed to write all of the bytes in the given buffer, so there might remain some that have not been written. Or there might have been an error.

Data loss suggests a flow control issue. You need to ensure that it is enabled at both ends. You've said that you tried it, but in your posted code it is not enabled. How are you opening and configuring the serial port at the board end?

10
  • Yes I've checked nextBatchSize and it's correct and write() should be correct because I can see that data are correctly output with logic analyzer. I'm not writing bare metal because I'm using an embedded OS written by my professor (Miosix OS) and I'm enabling flow control setting a flag at True in a config file so I'm not so sure about what I'm doing, but searching around I think I'm enabling RTSCTS. Enabling it on the script does not provide any improvement Apr 29, 2017 at 12:09
  • @GiovanniBeri: good, what about the return value from write()?
    – mhawke
    Apr 29, 2017 at 12:10
  • @GiovanniBeri: I have updated my answer to suggest using ser.in_waiting to check how many bytes are available before reading.
    – mhawke
    Apr 29, 2017 at 12:17
  • ok, using inWaiting() (it's not in_waiting anymore since pyserial3), I've seen that, when read(nextBatchSize) is blocked, the serial buffer is not empty but it does not contain the amount of bytes needed to unlock the read(nextBatchSize). I made a cycle which prints for 1000 times the value of inWaiting() and I see that it start from 0 and increases, but it stops at a certain seemingly random value (e.g. nextBatchSize == 400, inWaiting() == 200). Do you think it could be caused by flow control issues? Apr 29, 2017 at 13:25
  • @GiovanniBeri: Did you check the return value of write() on the board? Does it match the length of the data that you gave it?
    – mhawke
    Apr 29, 2017 at 14:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.