%0 Journal Article %1 WINDISCH2023108719 %A Windisch, Dominic %A Kelling, Jeffrey %A Juckeland, Guido %A Bieberle, André %D 2023 %J Computer Physics Communications %K xack %P 108719 %R https://doi.org/10.1016/j.cpc.2023.108719 %T Real-time data processing for ultrafast X-ray computed tomography using modular CUDA based pipelines %U https://www.sciencedirect.com/science/article/pii/S0010465523000644 %V 287 %X In this article, a new version of the Real-time Image Stream Algorithms (RISA) data processing suite is introduced. It now features online detector data acquisition, high-throughput data dumping and enhanced real-time data processing capabilities. The achieved low-latency real-time data processing extends the application of ultrafast electron beam X-ray computed tomography (UFXCT) scanners to real-time scanner control and process control. We implemented high performance data packet reception based on data plane development kit (DPDK) and high-throughput data storing using both hierarchical data format version 5 (HDF5) as well as the adaptable input/output system version 2 (ADIOS2). Furthermore, we extended RISA's underlying pipelining framework to support the fork-join paradigm. This allows for more complex workflows as it is necessary, e.g. for online data processing. Also, the pipeline configuration is moved from compile-time to runtime, i.e. processing stages and their interconnections can now be configured using a configuration file. In several benchmarks, RISA is profiled regarding data acquisition performance, data storage throughput and overall processing latency. We found that using direct IO mode significantly improves data writing performance on the local data storage. We could further prove that RISA is now capable of concurrently receiving, processing and storing data from up to 768 detector channels (3072 MB/s) at 8000 fps on a single-GPU computer in real-time. Program summary Program Title: GLADOS/RISA CPC Library link to program files: https://doi.org/10.17632/65sx747rvm.2 Developer's repository link: https://codebase.helmholtz.cloud/risa Licensing provisions: Apache-2.0 Programming language: C++ Journal reference of previous version: Comput. Phys. Commun. 219 (2017) 353-360 1 Does the new version supersede the previous version?: Yes. Reasons for the new version: Extended capabilities for real-time operation with latest UFXCT hardware. Summary of revisions: (i) Add forking and joining of processing pipeline branches (ii) Add runtime (re-)configuration of pipeline stages and connections (iii) Add UDP receiver stage to acquire detector data in real-time (iv) Add high-throughput data dumping Nature of problem: Ultrafast electron beam X-ray computed tomography scanners stream multiple Gigabytes of raw data per second via Ethernet to a control computer. Receiving the data with low latency, real-time image-based control would become possible. For this, data need to be captured from the network, stored on disk, reconstructed and post-processed concurrently. The current total data rate of up to 3072 MB/s requires high-throughput solutions for each of these tasks. Solution method: Using a pipeline scheme, RISA processes incoming raw data in distinct stages (sources, processors, sinks). These are implemented in GPU kernels and are executed concurrently to exploit data parallelism as well as task parallelism. To capture detector data, we implemented a UDP packet capturing stage based on DPDK 2 which acts as a source stage. By allowing the pipeline to fork up into multiple branches, we concurrently acquire, store and process the data. For storing these data, we use the HDF5 format 3. We achieve the required data rates by writing in direct IO mode onto an SSD array in RAID 0 configuration. Additional comments including restrictions and unusual features: RISA provides a set of general-purpose processing stages which are suitable for generic image stream processing. References 1Frust T., et al., Comput. Phys. Commun. 219 (2017) 353-3602Data Plane Development Kit, https://www.dpdk.org/3Hierarchical Data Format Version 5, https://www.hdfgroup.org/solutions/hdf5/

@article{WINDISCH2023108719, abstract = {In this article, a new version of the Real-time Image Stream Algorithms (RISA) data processing suite is introduced. It now features online detector data acquisition, high-throughput data dumping and enhanced real-time data processing capabilities. The achieved low-latency real-time data processing extends the application of ultrafast electron beam X-ray computed tomography (UFXCT) scanners to real-time scanner control and process control. We implemented high performance data packet reception based on data plane development kit (DPDK) and high-throughput data storing using both hierarchical data format version 5 (HDF5) as well as the adaptable input/output system version 2 (ADIOS2). Furthermore, we extended RISA's underlying pipelining framework to support the fork-join paradigm. This allows for more complex workflows as it is necessary, e.g. for online data processing. Also, the pipeline configuration is moved from compile-time to runtime, i.e. processing stages and their interconnections can now be configured using a configuration file. In several benchmarks, RISA is profiled regarding data acquisition performance, data storage throughput and overall processing latency. We found that using direct IO mode significantly improves data writing performance on the local data storage. We could further prove that RISA is now capable of concurrently receiving, processing and storing data from up to 768 detector channels (3072 MB/s) at 8000 fps on a single-GPU computer in real-time. Program summary Program Title: GLADOS/RISA CPC Library link to program files: https://doi.org/10.17632/65sx747rvm.2 Developer's repository link: https://codebase.helmholtz.cloud/risa Licensing provisions: Apache-2.0 Programming language: C++ Journal reference of previous version: Comput. Phys. Commun. 219 (2017) 353-360 [1] Does the new version supersede the previous version?: Yes. Reasons for the new version: Extended capabilities for real-time operation with latest UFXCT hardware. Summary of revisions: (i) Add forking and joining of processing pipeline branches (ii) Add runtime (re-)configuration of pipeline stages and connections (iii) Add UDP receiver stage to acquire detector data in real-time (iv) Add high-throughput data dumping Nature of problem: Ultrafast electron beam X-ray computed tomography scanners stream multiple Gigabytes of raw data per second via Ethernet to a control computer. Receiving the data with low latency, real-time image-based control would become possible. For this, data need to be captured from the network, stored on disk, reconstructed and post-processed concurrently. The current total data rate of up to 3072 MB/s requires high-throughput solutions for each of these tasks. Solution method: Using a pipeline scheme, RISA processes incoming raw data in distinct stages (sources, processors, sinks). These are implemented in GPU kernels and are executed concurrently to exploit data parallelism as well as task parallelism. To capture detector data, we implemented a UDP packet capturing stage based on DPDK [2] which acts as a source stage. By allowing the pipeline to fork up into multiple branches, we concurrently acquire, store and process the data. For storing these data, we use the HDF5 format [3]. We achieve the required data rates by writing in direct IO mode onto an SSD array in RAID 0 configuration. Additional comments including restrictions and unusual features: RISA provides a set of general-purpose processing stages which are suitable for generic image stream processing. References [1]Frust T., et al., Comput. Phys. Commun. 219 (2017) 353-360[2]Data Plane Development Kit, https://www.dpdk.org/[3]Hierarchical Data Format Version 5, https://www.hdfgroup.org/solutions/hdf5/}, added-at = {2025-01-29T14:43:33.000+0100}, author = {Windisch, Dominic and Kelling, Jeffrey and Juckeland, Guido and Bieberle, André}, biburl = {https://puma.scadsai.uni-leipzig.de/bibtex/2f09296b3296a886c101569e936f4651c/cosp536g}, doi = {https://doi.org/10.1016/j.cpc.2023.108719}, interhash = {298f90e64aa5501f08001c4647b12bcf}, intrahash = {f09296b3296a886c101569e936f4651c}, issn = {0010-4655}, journal = {Computer Physics Communications}, keywords = {xack}, pages = 108719, timestamp = {2025-01-29T14:43:33.000+0100}, title = {Real-time data processing for ultrafast X-ray computed tomography using modular CUDA based pipelines}, url = {https://www.sciencedirect.com/science/article/pii/S0010465523000644}, volume = 287, year = 2023 }