Data-acquisition software¶
sdaq¶
The sdaq tool streams data from a SPIDR4 source into file(s). The data may originate from one of 4 possible sources:
a SPIDR4’s 10GbE link, identified by an IP address and port, for example 192.168.100.1:8192,
a SPIDR4 DAQ board stream identified by an index number (starting from 0, <1024) as assigned by wupper, the driver for the SPIDR4 PCIe DAQ board (in this case sdaq needs to run on the machine with the DAQ board),
a Timepix4 slow-control data stream managed by the SPIDR4 Controller, addressed with the IP address of the Controller and port number 0, 1 or 2,
an IP port on the local host’s network interface (in combination with the use of sdaqi, and port number >1023).
One instance of sdaq can stream data from multiple sources of the same type, for example with a stream identifier 192.168.100.1:8192,8193, would stream data from 2 (network interface) sources (port 8192 and 8193) into 2 separate sets of files. If the sources would be PCIe-based, a stream identifier string such as 0,1 would indicate two such sources.
When taking data and writing to file sdaq automatically creates a new file each time a certain configurable file size is reached.
File names are composed of the provided base name (which may include a path), followed by date and time, followed by the IP port or PCIe device number and a counter that increments for each subsequent file created during the acquisition (optionally date+time can be left out), each separated from the other by a dash (‘-‘), so a file name looks like this: <basename>-<portnumber>-<year><month><day>-<hour><min><sec>-<counter>.dat
(Note that the file extension by default is ‘.dat’ but can be set to something else by providing it with the base name, for example as test.bin). An example of a file name is: test-8192-201029-124555-1.dat
In addition it is possible to optionally assign a run number to a data acquisition run, meaning that this run number is added to the names of the files created, in front of the counter, separated by a dash. An example of a file name with run number 123 is: test-8192-201029-124555-123-1.dat.
Optionally date and time may be left out from the file names, and consecutive file names would look like this: test-8192-1.dat, test-8192-2.dat, etc. or, with run number, like this: test-8192-123-1.dat, test-8192-123-2.dat, etc.
The duration of an sdaq data acquisition is configured either in seconds, and/or by a (minimum) amount of data to acquire. Every second during the acquisition a status line is displayed (per active stream) showing current data rates, data totals and the (host PC) memory buffer status (see below for examples of sdaq output).
If no filename is provided sdaq consumes the data, while displaying the status every second and optionally (using option -x) processing/inspecting the data received.
This is the ‘usage’ of sdaq, listing the available options:
Usage: sdaq [-h|V] [-D] [-b <size>] [-B <mb>] [-f <size>] [-i <dma>] [-I]
[-r <runnr>] [-t <secs>] [-T] [-c <caddr>] [-C]
<address> [<filename-base>]
-h : Show this help text.
-V : Show version.
-D : Debug mode on, i.e. output some additional info;
continue when memory buffer overflows.
-b <size> : Receive buffer memory size (from cmem_rcc or heap) to use,
in MB (default 1024, max 32768).
-B <mb> : Amount of data to acquire in MB
(by default DAQ is buffer-full or time limited).
DAQ duration is unlimited unless explicitly set by option -t
or until a buffer(-almost)-full condition arises.
-c <addr> : (IP) address of the SPIDR4 Controller,
or select by number from this predefined list:
0 = 192.168.1.10
1 = 192.168.100.10
2 = 192.16.192.232
-C : Read out the TPX4 configuration (option -c required)
and store at the start of each file.
-f <size> : Maximum file size [MB] (default 4096, max 16384).
-i <dma> : DAQ card DMA controller to use (default: 0).
-I : Use DAQ card (PCIe) interrupts to receive data (default: polling)
-r <runnr> : Run number to use in file names (default: none).
-t <secs> : Number of seconds to run the data acquisition (default: 0=unlimited)
(also applies in combination with option -B).
Input of data is halted and when writing to file,
all data received is saved, before the application exits.
Select 0 for unlimited duration of acquisition.
-T : Do NOT add datetime as part of file names
(could overwrite existing file(s) in the next run).
-w <size> : Maximum file write chunk size [MB] (default 16, max 512).
-x : Enable online pixel data processing, currently by collecting
a histogram of pixel hit positions and of ToA.
<address> : Datastream address(es): stream numbers separated by ',' (no spaces)
for PCIe streams, e.g. "0,1", or IP-number plus port(s),
e.g. "192.168.100.1:8192,8193", or (a) port number(s) only (>1023),
located on the host's standard network interface,
i.e. to receive data forwarded by the 'sdaqi' tool.
NB: if the selected port number is 0, 1 or 2
the 'SC' (SlowControl) data path is used, i.e. data is acquired
via the SPIDR4 Controller command/control connection:
port 0: from PACKETREADBOTTOM register (ca.50KB/s),
port 1: from PACKETREADTOP register (ca.50KB/s),
port 2: from SPIDR4 Controller 'SC stream' (ca.500KB/s).
(if <address> is equal to the Controller address (option -c)
the 'SC' data path is used as well (default port 2)).
<filename-base>:
Data files receive this name with the port or stream number followed
by datetime+runnumber+counter appended (datetime and runnumber optional),
separated by '-' characters.
Here is an example of running sdaq to receive data (for a duration of 100 seconds), not writing data to file:
$ sdaq -t 100 192.168.100.1
Consume SPIDR4 data
SpidrIpReceiver8192 @ 192.168.100.1:8192
Started SPIDR4 receiver 0 (buffersize=1024MiB)
Secs | Port | Recvd[MB/s] | File[MB/s] | Total[(M)B] | Rec[(M)B] | Buf[%] | Wraps
-------|------|-------------|------------|-------------|-----------|--------|-------
1 8192 616.0 0.0 616.0 0 1 0
2 8192 615.9 0.0 1231.9 0 0 1
3 8192 615.7 0.0 1847.6 0 0 1
4 8192 615.6 0.0 2463.2 0 1 2
5 8192 616.0 0.0 3079.2 0 0 2
6 8192 615.9 0.0 3695.2 0 0 3
7 8192 615.6 0.0 4310.8 0 1 4
8 8192 615.9 0.0 4926.7 0 0 4
9 8192 616.0 0.0 5542.7 0 0 5
10 8192 615.6 0.0 6158.3 0 1 5
....etc....
Here is an example of running sdaq to receive data on 2 streams, writing data to file, for at least 2 secs (no data is being produced though, in this example):
$ sdaq -t 2 192.168.0.4:8192,8193 test
Write SPIDR4 data to file
SpidrIpReceiver8192 @ 192.168.0.4:8192
Started SPIDR4 receiver 0 (buffersize=1024MiB)
SpidrIpReceiver8193 @ 192.168.0.4:8193
Started SPIDR4 receiver 1 (buffersize=1024MiB)
Started FileWriter 0 (filesize=4096MiB)
Started FileWriter 1 (filesize=4096MiB)
Secs | Port | Recvd[MB/s] | File[MB/s] | Tot[(M)B] | Rec[(M)B] | Buf[%]| Wrap
------|------|-------------|------------|-----------|-----------|-------|------
1 8192 0.0 0.0 0 0 0 0
1 8193 0.0 0.0 0 0 0 0
2 8192 0.0 0.0 0 0 0 0
2 8193 0.0 0.0 0 0 0 0
3 8192 0.0 0.0 0 0 0 0
3 8193 0.0 0.0 0 0 0 0
4 8192 0.0 0.0 0 0 0 0
4 8193 0.0 0.0 0 0 0 0
**STOP**
-> Totals #0: Recvd 0 MB, File 0 MB (last 0 MB), 1 files
-> File #0: ./test-8192-201130-193456-1.dat
-> Totals #1: Recvd 0 MB, File 0 MB (last 0 MB), 1 files
-> File #1: ./test-8193-201130-193456-1.dat
Exiting..
Afterwards 2 files have been created (each containing just a single 8-byte frame header, because in this example no data was received):
$ ls -l *.dat
-rw-r--r-- 1 user group 8 Nov 30 19:35 test-8192-201130-193456-1.dat
-rw-r--r-- 1 user group 8 Nov 30 19:35 test-8193-201130-193456-1.dat
Here is an example of running sdaq to receive at least 64MB of data (and then exit), not writing data to file:
$ sdaq -B 64 192.168.100.1
sdaqi¶
The sdaqi tool streams data from a SPIDR4 source to a network endpoint, i.e. a selected host IP address plus port number, using the UDP protocol. It is basically identical to sdaq (allowing the same choice of sources), but without the options to stream the data to file(s), but instead to relay the raw SPIDR4 data across the network, each defined stream to a single IP host/port pair.
Here is an example of running sdaqi to receive data on 2 PCIe-based streams, relaying the data streams to 2 different hosts:
$ sdaqi -t 2 0,1 agogna:8000 gimone:8001
where PCIe stream 0 is forwarded to host agogna port 8000, and PCIe stream 1
is forwarded to host gimone port 8001.
On the receiving hosts, e.g host gimone, one could receive the data and
store it to file using sdaq, in this example on host gimone as follows:
$ sdaq 8001 test
Typically one would start sdaq first, in case sdaqi immediately starts producing data at start-up.
sdaqs¶
The sdaqs is a high-performance server DAQ application which can be controlled over gRPC. It provides the same functions as sdaqs and sdaqi but then programmatically.
Please see grpc reference for more information on the calls, specifically on the gRPC service description in daq.proto.
sinspect¶
The sinspect tool is used to ‘analyse’ and inspect data in files created with sdaq. The file format is described in a next section. The basic unit of storage in the file is called a ‘frame’ which may contain data but also meta-information, such as Timepix4 device configuration settings or SPIDR4 board information.
With sinspect one can:
check the integrity of the frame format in a file, displaying some general information about the file and a list of the number of frames of each type encountered, optionally displaying the content of frames with JSON content, for example (the 2nd example runs inspect on a file that was cut in two parts to demonstrate some error reporting):
$ sinspect -j test-0-1.dat
File opened: test-0-1.dat
File header: SPIDR4
File size : 1056965224
S4HDR_BOARDINFO:
{
"controller": {
"address": "192.16.192.232",
"name": "Spidr4 Control Service",
"version": [
0,
1,
0
]
}
}
End-of-file reached (index=1056965224)
Frames = 64, Total bytes = 1056964608
Frame counters:
S4HDR_TPX4DATA_COMBINED 63
S4HDR_BOARDINFO 1
$ sinspect split-front.dat
File opened: split-front.dat
File header: SPIDR4
File size : 10000000
### Truncated last frame #2
End-of-file reached (index=10000000)
Frames = 2, Total bytes = 16777216
Frame counters:
S4HDR_TPX4DATA_COMBINED 1
S4HDR_BOARDINFO 1
### There were errors in this file
$ sinspect split-back.dat
### Doesn't look like a SPIDR4 data file:
expected header 0x0000345244495053, got 0xffffffffffffffff
display a listing of each frame type with pixel data and with number of pixels per frame in order of occurrence in the file, for example:
$ sinspect -p tst-0-1.dat
File opened: tst-0-1.dat
File header: SPIDR4
File size : 640991392
PIX_IN_FRAME = 2097152
PIX_IN_FRAME = 2097152
PIX_IN_FRAME = 1334675
PIX_IN_FRAME = 314439
PIX_IN_FRAME = 48117
PIX_IN_FRAME = 20142
PIX_IN_FRAME = 19023
PIX_IN_FRAME = 1557648
PIX_IN_FRAME = 941079
PIX_IN_FRAME = 243942
PIX_IN_FRAME = 27975
PIX_IN_FRAME = 1556529
...etc...
Total pixels = 80123757
Frame counters:
S4HDR_TPX4DATA_COMBINED 166
display information about individual pixels one-by-one, including EoC value, pixel address (below in the example output called ‘pixnr’ and also converted to x/y coordinates), ToA and the other time-related values (in hexadecimal), and in case of 8-bit or 16-bit frame data displays segment number, pixel index and pixel values. Here is an example, starting from the 174575th pixelpacket in the selected file (with option -L one could choose to display only every n-th pixelpacket, with n<=8 currently):
$ sinspect -C -P -s 174574 Sr90_0p1-8192-1.dat | more
=> Display pixels (using stride=1)
File opened: Sr90_0p1-8192-1.dat
File header: SPIDR4
File size : 2082384 bytes
Skip to pixel 174575 (bytes 154f70)
174575: HEARTBEAT T 32770
174575: HEARTBEAT T 32770 (diff=1025)
174576: HEARTBEAT T 33795 (diff=1025)
174577: HEARTBEAT T 34820 (diff=1025)
174578: HEARTBEAT T 35845 (diff=1025)
174579: SHUTTER_RISE T 36515
174580: top=1 eoc=5b pixnr=1fc (x=264 y=259) toa=c9fe uftoa-sta/sto=1,f ftoa-f/r= 2, d tot= 0 pileup=0
174581: top=1 eoc=5b pixnr=1ff (x=264 y=256) toa=c9ff uftoa-sta/sto=7,f ftoa-f/r= b, 4 tot= 1 pileup=0
174582: HEARTBEAT T 36870 (diff=1025)
174583: HEARTBEAT T 37895 (diff=1025)
174584: top=1 eoc=cd pixnr= ef (x= 36 y=392) toa=da1d uftoa-sta/sto=1,f ftoa-f/r= 6,15 tot= 14 pileup=0
174585: HEARTBEAT T 38920 (diff=1025)
174586: HEARTBEAT T 39945 (diff=1025)
174587: top=1 eoc=53 pixnr= cb (x=281 y=408) toa=d622 uftoa-sta/sto=e,f ftoa-f/r= d, f tot= 8 pileup=0
174588: top=1 eoc=53 pixnr= d7 (x=280 y=404) toa=d622 uftoa-sta/sto=c,1 ftoa-f/r= c, 6 tot= 0 pileup=0
174589: top=1 eoc=53 pixnr= d2 (x=281 y=405) toa=d623 uftoa-sta/sto=0,e ftoa-f/r= 4, 4 tot= 10 pileup=0
174590: top=1 eoc=52 pixnr= cf (x=282 y=408) toa=d622 uftoa-sta/sto=0,f ftoa-f/r= 6, f tot= 13 pileup=0
174591: top=1 eoc=53 pixnr= d0 (x=281 y=407) toa=d623 uftoa-sta/sto=0,1 ftoa-f/r=10, 4 tot= 19 pileup=0
174592: top=1 eoc=52 pixnr= d4 (x=282 y=407) toa=d623 uftoa-sta/sto=e,f ftoa-f/r=10, 3 tot= 2b pileup=0
174593: top=1 eoc=53 pixnr= d1 (x=281 y=406) toa=d623 uftoa-sta/sto=0,f ftoa-f/r= 6, 5 tot= 15 pileup=0
174594: top=1 eoc=53 pixnr= d6 (x=280 y=405) toa=d623 uftoa-sta/sto=0,7 ftoa-f/r= b, 5 tot= 3e pileup=0
174595: top=1 eoc=54 pixnr= d2 (x=279 y=405) toa=d623 uftoa-sta/sto=0,f ftoa-f/r= 8, 4 tot= 51 pileup=0
174596: HEARTBEAT T 40970 (diff=1025)
174597: top=1 eoc=3b pixnr=14d (x=328 y=346) toa=f0d6 uftoa-sta/sto=3,f ftoa-f/r=12,11 tot= a pileup=0
...etc...etc...
or display the same pixel packets in ‘raw’ mode:
$ sinspect -C -R -s 174574 Sr90_0p1-8192-1.dat | more
=> Display pixels (using stride=1)
File opened: Sr90_0p1-8192-1.dat
File header: SPIDR4
File size : 2082384 bytes
Skip to pixel 174575 (bytes 154f70)
174575: f000000000008002
174576: f000000000008403
174577: f000000000008804
174578: f000000000008c05
174579: f080000000008ea3
174580: adff327fbc5a2000
174581: adfff27ffdc8b002
174582: f000000000009006
174583: f000000000009407
174584: e6bbf6877c6a6028
174585: f000000000009808
174586: f000000000009c09
174587: a9b2f588bf9ed010
174588: a9b5f588870cc000
174589: a9b4b588f8084020
174590: a933f588bc1e6026
174591: a9b43588c4090032
174592: a9353588ff870056
174593: a9b47588fc0a602a
174594: a9b5b588dc0ab07c
174595: aa34b588fc0880a2
174596: f00000000000a00a
174597: 9dd37c35bce32014
...etc...etc...
accumulate a histogram of ToA values, where the output could be copied into a program to convert it into a plot; here is an example generating 25 bins:
$ sinspect -p tst-8192-1.dat -H 25
File opened: tst-8192-1.dat
File header: SPIDR4
File size : 1449943592 bytes
ToA Bin Count
------- ---------
0 7254251
2622 7248408
5244 7248391
7866 7253139
10488 7247974
13110 7253767
15732 7250829
18354 7250887
20976 7253813
23598 7248043
26220 7253152
28842 7248224
31464 7248399
34086 7254922
36708 7250726
39330 7254493
41952 7250710
44574 7250844
47196 7253611
49818 7248099
52440 7252551
55062 7249906
57684 7250765
60306 7254791
62928 7211889
65550
Total pixelpackets = 181242584
Chunk counters:
S4HDR_TPX4DATA_COMBINED 364
This is the ‘usage’ of sinspect, listing the available options:
Usage: sinspect [-h|V] [-c] [-C] [-e] [-f] [-j] [-H <bins>] [-L <links>] [-p|P|Q|R] [-s <cnt>] <filename>
-h : Show this help text.
-V : Show version.
-c : In case of a chunk containing a Timepix4 configuration,
display the contents, i.e. each register setting.
-C : Display a pixel packet counter as the first number (with options -P/Q).
-e : Display a list of EoC counters (with option -p).
-f : Display individual frame headers and sizes (without options -p/P/Q).
-j : Show JSON string in case of JSON-formatted frames (without options -p/P/Q).
-H <bins> : Display a histogram with <bins> bins (lines of text) of:
ToA (with option -p) or data frame sizes (without -p).
-L <links> : Number of (Timepix4) links (with option -Q) *or*
the stride when displaying pixels (with option -P;
use option -s to start from a pixel other than first).
-p : Scan the pixels, collecting statistics
on ToA and EoC and pixel x/y 'hits'
(a greyscale image representation of the x/y counters is generated).
-P : Scan the pixels, display info on each pixel.
-R : Scan the pixels, display raw 64-bit value (hex) of each pixel.
-Q : Scan the pixels, checking for ...
(NB currently: check for test-generator format)
-s <cnt> : Skip <cnt> pixels before starting the pixel scan (with option -p/P/Q).
<filename>: Name of data file.