0

I have an issue. I'm using matplotlib on a project to do some real-time programming, I used this library because it was suggested by my mentor although I think now it was not a good idea since I learned that matplotlib is not real-time oriented. As I was working at the lab with the lab computer, real-time made the computer slow, but not that slow as now that I'm remote working on my laptop that is 5 years old, but I don't think that's the problem. In the project, I use Python 2.7, postgresql, pandas, wxpython, etc, so you get the idea. I have a notebook with multiple tabs and in each tab I implemented one figure of mpl for each instrument I start adding. For performance things I make it stop animation and star as you go between the tabs and only if the have been started with the desirable variable.

The thing is that as I started working on my laptop, as soon as I start the real-time plotting, the applications becomes very laggy, it's mostly unusable because it jumps too much when I'm trying to interact with it, or when is running too long. Then I think it's something more of the code and how it's plot. I'm seeking a way of making it run faster. My code for plotting is the following.

class live_panel_2(wx.Panel):  # Real Time plotter for 2 Variable
    def __init__(self, parent):
        super(live_panel_2, self).__init__(parent)
        #sns.set()  # Set Plot Style

        self.parent = parent
        self.x_axis = []  # First x-axis
        self.x2_axis = []  # Second x-axis
        self.y_axis = []  # First y-axis
        self.y2_axis = []  # Seconf y-axis
        self.line_width = 1  # Line width for plots
        self.flag = False
        self.switch = True  # Switch Flag
        self.animation = False  # Animation Flag
        self.figure = Figure(figsize=(10, 3))  # Figure Size
        self.canvas = FigureCanvas(self, -1, self.figure)  # Canvas of Figure
        self.axis = self.figure.add_subplot(1, 1, 1)
        self.axis2 = self.axis.twinx()  # Adding Second Axis
        self.toolbar = NavigationToolbar(self.canvas)  # Adding Navigation Tool
        self.toolbar.Realize()
        self.ani = None

        self.figure.subplots_adjust(left=0.09, right=0.92, top=0.92, bottom=0.2, wspace=1)  # Figure Space Adjust
        self.axis.format_xdata = mdates.DateFormatter('%Y-%m-%d')  # Format Dates
        self.axis2.format_xdata = mdates.DateFormatter('%Y-%m-%d')  # Format Dates
        # self.axis.tick_params(axis='x', direction='inout', length=5, labelrotation=0)

        # Sizer Adds
        sizer = wx.BoxSizer(wx.VERTICAL)
        sizer.Add(self.canvas, 0, wx.EXPAND, 5)
        sizer.Add(self.toolbar, 0, wx.LEFT | wx.EXPAND, 5)
        # sizer.Add(self.button, 0, wx.ALL, 5)

        self.SetSizerAndFit(sizer)
        self.Bind(wx.EVT_CLOSE, self.stop)

        self.canvas.draw()

    def build(self, model, model_2, y, y2, delta, localization):  # Method that builds the plot based on info received:
        """"
        Model: Mapped Model to be Used.
        Y: Mapped parameter to be plotted.
        Delta: Plot Range
        Localization: Legend localization
        """
        self.axis.set_title(model.Datetime.name + ' vs ' + y.name + ' vs ' + y2.name)
        self.axis.set_xlabel(model.Datetime.name)
        self.axis.set_ylabel(y.name)


        # self.axis.annotate(str(value), xy=(time, value), xytext=(10, -2), textcoords='offset pixels',
        #                    bbox=dict(boxstyle=custom_box_style, alpha=0.2))

        # Set Axis 2 Format

        self.axis2.set_xlabel(model.Datetime.name)
        self.axis2.set_ylabel(y2.name)

        # self.axis2.annotate(str(value2), xy=(time, value2), xytext=(10, -2), textcoords='offset pixels',
        #                    bbox=dict(boxstyle=custom_box_style, alpha=0.2))


        # Turn of scientific notation
        self.axis.yaxis.set_major_formatter(mticker.ScalarFormatter())
        self.axis.yaxis.get_major_formatter().set_scientific(False)
        self.axis.yaxis.get_major_formatter().set_useOffset(False)
        self.axis2.yaxis.set_major_formatter(mticker.ScalarFormatter())
        self.axis2.yaxis.get_major_formatter().set_scientific(False)
        self.axis2.yaxis.get_major_formatter().set_useOffset(False)

        # Set legend and turn axes 2 grid off

        self.axis2.grid(False)

        if self.animation:  # If Animation is true
            self.ani._stop()  # Stop animation
            self.x_axis = []  # Make Arrays empty
            self.x2_axis = []
            self.y_axis = []
            self.y2_axis = []
        self.delta = delta  # Set Delta
        self.ani = animation.FuncAnimation(self.figure, self.animate,
                                           fargs=(
                                               self.x_axis, self.x2_axis, self.y_axis, self.y2_axis, model, model_2, y,
                                               y2,
                                               localization), interval=500)  # Call Animate Method.
        self.animation = True  # Set Animation to True
        self.ani._start()  # Start Animation

    def close(self):
        self.ani._stop()

    def stop(self, event):  # Stop method when changing between real-time notebook
        if self.ani != None:  # If animation have been created.
            #  If animation is running and is not current page.
            if self.flag == False and self.animation == True and self.parent != self.parent.parent.GetCurrentPage():
                self.ani.event_source.stop()  # Stop animation
                self.flag = True
                self.animation = False  # Set Animation flag to False
            # If animation is not running and is current page.
            elif self.flag == True and self.animation == False and self.parent == self.parent.parent.GetCurrentPage():
                self.ani.event_source.start()
                self.flag = False
                self.animation = True
        event.Skip()

    def mode_stop(self, event):  # Stop method when changing between views.
        if self.ani != None:
            if self.switch != True and self.parent == self.parent.parent.GetCurrentPage():
                self.ani.event_source.start()
                self.switch = True
            else:
                self.ani.event_source.stop()
                self.switch = False
        event.Skip()

    # Animation
    def animate(self, i, x_axis, x2_axis, y_axis, y2_axis, model, model2, y, y2, localization):
        # Data query
        self.now = datetime.now()  # Get Present Time
        self.now_delta = self.now - timedelta(minutes=self.delta)  # Get past time with delta.
        if not x_axis:  # If x_axis array have not been created.
            with session_scope() as s:  # Open SQL Session
                value = s.query(y).order_by(model.Datetime.desc()).filter(model.Datetime > self.now_delta).filter(
                    model.Datetime < self.now).all()  # Query Values for y array in axis 1.
                value2 = s.query(y2).order_by(model2.Datetime.desc()).filter(model2.Datetime > self.now_delta).filter(
                    model2.Datetime < self.now).all()  # Query Values for y array in axis 2.
                time = s.query(model.Datetime).order_by(model.Datetime.desc()).filter(
                    model.Datetime > self.now_delta).filter(
                    model.Datetime < self.now).all()  # Query dates for x array in axis 1.
                time2 = s.query(model2.Datetime).order_by(model2.Datetime.desc()).filter(
                    model2.Datetime > self.now_delta).filter(
                    model2.Datetime < self.now).all()  # Query date for x array in axis 2.

            # Reverse List.
            for i in reversed(value):
                y_axis.append(i[0])
            for i in reversed(value2):
                y2_axis.append(i[0])
            for i in reversed(time):
                x_axis.append(i[0])
            for i in reversed(time2):
                x2_axis.append(i[0])

        if x_axis:  # If X Axis Array Exist
            with session_scope() as s:  # Open SQL Session
                value = s.query(y).filter(model.Datetime > x_axis[
                    -1]).all()  # Query Valyes for Y array in axis 1, after the date of the last point in the array.
                value2 = s.query(y2).filter(model2.Datetime > x2_axis[
                    -1]).all()  # Query Valyes for Y array in axis 2, after the date of the last point in the array.
                time = s.query(model.Datetime).filter(model.Datetime > x_axis[
                    -1]).all()  # Query Valyes for X array in axis 1, after the date of the last point in the array.
                time2 = s.query(model2.Datetime).filter(model2.Datetime > x2_axis[
                    -1]).all()  # Query Valyes for X array in axis 2, after the date of the last point in the array.

                # Appent in Array
                for i in value:
                    y_axis.append(i[0])
                for i in value2:
                    y2_axis.append(i[0])
                for i in time:
                    x_axis.append(i[0])
                for i in time2:
                    x2_axis.append(i[0])

        # Alinate Array
        x_axis, y_axis = array_alinate(x_axis, y_axis)
        x2_axis, y2_axis = array_alinate(x2_axis, y2_axis)

        # Create Dataframe from the arrays
        data_1 = pd.DataFrame(y_axis, x_axis, [y.name], dtype=float)
        data_2 = pd.DataFrame(y2_axis, x2_axis, [y2.name], dtype=float)

        # Set Index Name to Models Datetime Name
        data_1.index.name = model.Datetime.name
        data_2.index.name = model2.Datetime.name

        # Resapmle to second if the models are from wibs-neo
        if model == (wibs_neo_monitoring or wibs_neo_particle):
            data_1 = data_1.resample('S').mean()
            data_2 = data_2.resample('S').mean()

        # Secure data points dont past delta, asuming each datapoint lapse one second.
        data_1 = data_1.iloc[-self.delta * 60:]
        data_2 = data_2.iloc[-self.delta * 60:]

        # Clear Axis
        #self.axis.clear()
        #self.axis2.clear()

        self.axis.set_xlim(min(data_1.index), max(data_1.index))
        self.axis2.set_xlim(min(data_2.index), max(data_2.index))
        line_2 = self.axis2.plot(data_2, linewidth=self.line_width, color='G', label=y2.name)
        line_1 = self.axis.plot(data_1, linewidth=self.line_width, color='B', label=y.name)

        lines = line_1 + line_2
        labels = [l.get_label() for l in lines]
        self.axis2.legend(lines, labels, loc=localization)

        # Format plot
        self.figure.canvas.mpl_connect('close_event', self.close)
        # tm.sleep(1)
14
  • Matplotlib is not made for real time plotting. see stackoverflow.com/a/57461819/7919597
    – Joe
    Mar 9, 2020 at 19:38
  • Good example with multiple scrolling plots github.com/pyqtgraph/pyqtgraph/blob/develop/examples/…
    – Joe
    Mar 9, 2020 at 19:39
  • Hi @Joe, i was looking around to re-write in my code in PyQTGrapth and right now i'm using python 2.7 to do this. I got this error Exception: PyQtGraph requires one of PyQt4, PyQt5 or PySide; none of these packages could be imported. Mar 9, 2020 at 20:49
  • and if i'm good, PyQTGraph is not compatible with wxpython Mar 9, 2020 at 21:29
  • They are two different frameworks, Qt is a very well know and mighty toolkit. Not sure what your project is aiming at. If you are trying something professional, you might be better of changing to Qt. github.com/ZELLMECHANIK-DRESDEN/ShapeOut/issues/150
    – Joe
    Mar 10, 2020 at 6:12

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.