TCLab Historian

Basic logging

The tclab.Historian class provides data logging. Given an instance of a TCLab object, an historian is created with the commands

import tclab
lab = tclab.TCLab()
h = tclab.Historian(lab.sources)

The historian initializes a data log. The sources for the data log are specified in the argument to tclab.Historian. A default set of sources for an instance lab is given by lab.sources. The specification for sources is described in a later section.

The data log is updated by issuing a command

h.update(t)

Where t is the current clock time. If t is omitted the historian will calculate its own time.

import tclab

TCLab = tclab.setup(connected=False, speedup=10)

with TCLab() as lab:
    h = tclab.Historian(lab.sources)
    for t in tclab.clock(20):
        lab.Q1(100 if t <= 10 else 0)
        print("Time:", t, 'seconds')
        h.update(t)

TCLab version 0.4.7dev
Simulated TCLab
Time: 0 seconds
Time: 1.1 seconds
Time: 2.0 seconds
Time: 3.1 seconds
Time: 4.1 seconds
Time: 5.0 seconds
Time: 6.1 seconds
Time: 7.0 seconds
Time: 8.2 seconds
Time: 9.0 seconds
Time: 10.0 seconds
Time: 11.0 seconds
Time: 12.0 seconds
Time: 13.1 seconds
Time: 14.0 seconds
Time: 15.1 seconds
Time: 16.1 seconds
Time: 17.2 seconds
Time: 18.0 seconds
Time: 19.2 seconds
Time: 20.0 seconds
TCLab Model disconnected successfully.

Accessing the Data Log from the Historian

Historian maintains a data log that is updated on each encounter of the .update() function. The list of variables logged by an Historian is given by

h.columns

['Time', 'T1', 'T2', 'Q1', 'Q2']

Individual time series are available as elements of Historian.fields. For the default set of sources, the time series can be obtained as

t, T1, T2, Q1, Q2 = h.fields

For example, here’s how to plot the history of temperature T1 versus time from the example above.

%matplotlib notebook
import matplotlib.pyplot as plt

t, T1, T2, Q1, Q2 = h.fields
plt.plot(t, T1)
plt.xlabel('Time / seconds')
plt.ylabel('Temperature / °C')
plt.grid()

Accessing fields by name

To enable easy access of a particular field by name, Historian allows individual field access via the logdict dictionary. It also has a t property for quick access of the time field, so the above code could also have been written this way:

%matplotlib notebook
import matplotlib.pyplot as plt

column = 'T1'
plt.plot(h.t, h.logdict['T1'])
plt.xlabel('Time / seconds')
plt.ylabel('Temperature / °C')
plt.grid()

Here is a simple code using these access methods to plot all the fields an Historian is logging:

def plotlog(historian):
    t = historian.t
    fig, axes = plt.subplots(nrows=len(historian.fields) - 1,
                        ncols=1, sharex='col')
    for axis, column in zip(axes, historian.columns[1:]):
        axis.step(t, historian.logdict[column], where='post')
        axis.grid()
        axis.set_ylabel(column)
    plt.xlabel('Time / Seconds')

plotlog(h)

Tabular access

The entire data history is available from the historian as the attribute .log. Here we show the first three rows from the log. The columns are returned in the same order as .columns.

h.columns

['Time', 'T1', 'T2', 'Q1', 'Q2']

h.log[:3]

[(0, 20.949499999999997, 20.6272, 100, 0),
 (1.1, 20.949499999999997, 20.949499999999997, 100, 0),
 (2.0, 20.949499999999997, 20.949499999999997, 100, 0)]

Saving to a file

The log can be saved to a CSV file using .to_csv(). This is useful for spreadsheet access.

h.to_csv('saved_data.csv')

Accessing log data using Pandas

Pandas is a widely use Python library for manipulation and analysis of data sets. Here we show how to access the tclab.Historian log using Pandas.

The log can be converted to a Pandas dataframe.

import pandas as pd

df = pd.DataFrame.from_records(h.log, columns=h.columns, index='Time')
df.head()

	T1	T2	Q1	Q2
Time
0.0	20.9495	20.6272	100	0
1.1	20.9495	20.9495	100	0
2.0	20.9495	20.9495	100	0
3.1	20.9495	20.9495	100	0
4.1	20.9495	20.9495	100	0

The following cells provide examples of plots that can be constructed once the data log has been converted to a pandas dataframe.

df.plot()

<matplotlib.axes._subplots.AxesSubplot at 0x10a5e2cf8>

df[['T1','T2']].plot(grid=True)

<matplotlib.axes._subplots.AxesSubplot at 0x10aa7e8d0>

Specifying Sources for tclab.Historian

To create an instance of tclab.Historian, a set of sources needs to be specified. For many cases the default sources created for an instance of TCLab is sufficient. However, it is possible to specify additional sources which can be useful when implementing more complex algorithms for process control.

sources is specified as a list of tuples. Each tuple as two elements. The first element is a label for the source. The second element is a function that returns a value.

The following cell shows how to create a source with the label Power with a value equal to the estimated heater power measured in watts. This is created on the assumption that 100% of a maximum power of 200 corresponds to 4.2 watts.

import tclab

TCLab = tclab.setup(connected=False, speedup=10)

with TCLab() as lab:
    sources = [
        ('T1', lambda: lab.T1),
        ('Power', lambda: lab.P1*lab.U1*4.2/(200*100))
    ]
    h = tclab.Historian(sources)
    for t in tclab.clock(20):
        lab.Q1(100 if t <= 10 else 0)
        print("Time:", t, 'seconds')
        h.update(t)

TCLab version 0.4.7dev
Simulated TCLab
Time: 0 seconds
Time: 1.0 seconds
Time: 2.0 seconds
Time: 3.0 seconds
Time: 4.1 seconds
Time: 5.1 seconds
Time: 6.1 seconds
Time: 7.0 seconds
Time: 8.0 seconds
Time: 9.0 seconds
Time: 10.1 seconds
Time: 11.2 seconds
Time: 12.1 seconds
Time: 13.2 seconds
Time: 14.2 seconds
Time: 15.0 seconds
Time: 16.1 seconds
Time: 17.0 seconds
Time: 18.0 seconds
Time: 19.0 seconds
Time: 20.1 seconds
TCLab Model disconnected successfully.

import pandas as pd

df = pd.DataFrame.from_records(h.log, columns=h.columns, index='Time')
df.head()

	T1	Power
Time
0.0	20.9495	4.2
1.0	20.9495	4.2
2.0	20.9495	4.2
3.0	20.9495	4.2
4.1	20.9495	4.2

Functions with multiple returns

In some cases it is easier to calculate a number of different variables to be logged in one function, especially if intermediate results are used in following calculations. This can be accommodated by Historian by passing None as the function for subsequent values if a previous value returned a list of values.

import tclab

TCLab = tclab.setup(connected=False, speedup=10)

def log_values():
    T1 = lab.T1
    T1Kelvin = T1 + 273.15
    power = lab.P1*lab.U1*4.2/(200*100)
    return T1, T1Kelvin, power

with TCLab() as lab:
    h = tclab.Historian([('T1', log_values),
                         ('T1Kelvin', None),
                         ('Power', None)])
    for t in tclab.clock(20):
        lab.Q1(100 if t <= 10 else 0)
        print("Time:", t, 'seconds')
        h.update(t)

TCLab version 0.4.7dev
Simulated TCLab
Time: 0 seconds
Time: 1.0 seconds
Time: 2.0 seconds
Time: 3.0 seconds
Time: 4.0 seconds
Time: 5.0 seconds
Time: 6.0 seconds
Time: 7.2 seconds
Time: 8.0 seconds
Time: 9.2 seconds
Time: 10.1 seconds
Time: 11.0 seconds
Time: 12.2 seconds
Time: 13.0 seconds
Time: 14.0 seconds
Time: 15.0 seconds
Time: 16.1 seconds
Time: 17.0 seconds
Time: 18.1 seconds
Time: 19.1 seconds
Time: 20.0 seconds
TCLab Model disconnected successfully.

import pandas as pd

df = pd.DataFrame.from_records(h.log, columns=h.columns, index='Time')
df.head()

	T1	T1Kelvin	Power
Time
0.0	20.9495	294.0995	4.2
1.0	20.9495	294.0995	4.2
2.0	20.9495	294.0995	4.2
3.0	20.9495	294.0995	4.2
4.0	20.9495	294.0995	4.2

h.log

[(0, 20.949499999999997, 294.0995, 4.2),
 (1.0, 20.949499999999997, 294.0995, 4.2),
 (2.0, 20.949499999999997, 294.0995, 4.2),
 (3.0, 20.949499999999997, 294.0995, 4.2),
 (4.0, 20.949499999999997, 294.0995, 4.2),
 (5.0, 21.2718, 294.42179999999996, 4.2),
 (6.0, 21.2718, 294.42179999999996, 4.2),
 (7.2, 21.594099999999997, 294.7441, 4.2),
 (8.0, 21.594099999999997, 294.7441, 4.2),
 (9.2, 21.594099999999997, 294.7441, 4.2),
 (10.1, 21.9164, 295.0664, 0.0),
 (11.0, 21.9164, 295.0664, 0.0),
 (12.2, 22.238699999999998, 295.3887, 0.0),
 (13.0, 22.238699999999998, 295.3887, 0.0),
 (14.0, 22.561, 295.71099999999996, 0.0),
 (15.0, 22.561, 295.71099999999996, 0.0),
 (16.1, 22.8833, 296.0333, 0.0),
 (17.0, 22.8833, 296.0333, 0.0),
 (18.1, 22.8833, 296.0333, 0.0),
 (19.1, 23.205599999999997, 296.3556, 0.0),
 (20.0, 23.205599999999997, 296.3556, 0.0)]

Sessions

It is possible to run multiple experiments using the same Historian and page back to them after running them:

import tclab

TCLab = tclab.setup(connected=False, speedup=10)

with TCLab() as lab:
    h = tclab.Historian(lab.sources)
    for t in tclab.clock(20):
        lab.Q1(100 if t <= 10 else 0)
        h.update(t)
    h.new_session()
    for t in tclab.clock(20):
        lab.Q1(0 if t <= 10 else 100)
        h.update(t)

TCLab version 0.4.7dev
Simulated TCLab
TCLab Model disconnected successfully.

To see the stored sessions, use get_sessions:

h.get_sessions()

[(1, '2018-03-18 08:10:02', 21), (2, '2018-03-18 08:10:03', 21)]

The historian log shows the data for the last session, where the heater started open

h.log[:2]

[(0, 23.205599999999997, 20.949499999999997, 0, 0),
 (1.1, 23.205599999999997, 20.949499999999997, 0, 0)]

To roll back to a different session, use load_session():

h.load_session(1)

h.log[:2]

[(0, 20.949499999999997, 20.949499999999997, 100, 0),
 (1.0, 20.949499999999997, 20.949499999999997, 100, 0)]

Persistence

Historian stores results in a SQLite database. By default, it uses the special file :memory: which means the results are lost when you shut down or restart the kernel. To retain values across runs, you can specify a filename (by convention SQLite databases have the .db extension).

import tclab

TCLab = tclab.setup(connected=False, speedup=10)

with TCLab() as lab:
    h = tclab.Historian(lab.sources, dbfile='test.db')
    for t in tclab.clock(20):
        lab.Q1(100 if t <= 10 else 0)
        h.update(t)

TCLab version 0.4.7dev
Simulated TCLab
TCLab Model disconnected successfully.

Every time you run this cell, new sessions will be added to the file. These sessions can be loaded as shown in the Sessions section above. There is currently no support for managing sessions. If you want to remove the old sessions, delete the database file.