update ReadMe

README.md

@@ -1,72 +0,0 @@
-# MA Autoware Trace Analysis
-
-Automatically extract data dependencies and end-to-end (E2E) latencies from ROS2 trace data and source code.
-
-## Prerequisites
-* Python 3.10 or newer (this is crucial!)
-* [JupyterLab](https://jupyter.org/install#jupyterlab) or [Jupyter Notebook](https://jupyter.org/install#jupyter-notebook)
-* [ROS2 Tracing](https://github.com/ros2/ros2_tracing)
-* [Tracetools Analysis](https://gitlab.com/ros-tracing/tracetools_analysis)
-* `python3-babeltrace` and `python3-lttng`, e.g. via `sudo apt install`, **for Python 3.10** (this requires either Ubuntu 22.04 or custom installation)
-
-## Requirements for Optional Features
-* `ros2_internal_dependency_analyzer` for finding intra-node dependencies via static analysis, which can then be
-  utilized by this tool
-* `ros2 multitopic bw` for recording message sizes which can then be processed by this tool
-
-
-## Installation
-
-```shell
-# Make sure you are running Python 3.10 and the corresponding pip:
-python3.10 -V
-pip3.10 -V
-
-pip3.10 install -r requirements.txt
-```
-
-Make sure that ROS2 Tracing and Tracetools Analysis were compiled with `colcon build [...] --symlink-install [...]`. Without `--symlink-install`, the build folder structure will be incorrect and the libraries required by this tool cannot be located.
-
-## Usage
-
-The `trace_analysis.ipynb` notebook is the entry point for users.
-Configure the notebook according to the comments in the user settings cell.
-Settings can either be changed in the notebook or via environment variables:
-```python
-# In the notebook (User Settings cell near the top):
-TR_PATH = "path/to/trace/dir"
-E2E_ENABLED = True
-...
-```
-
-```shell
-# In the shell of your choice (choose Bash!):
-# Each setting is named the same as in the notebook but prefixed by "ANA_NB_".
-# For strings, quotes have to be included IN THE VARIABLE VALUE, i.e. double quotation "'...'" 
-# has to be used in most cases
-ANA_NB_TR_PATH="'path/to/trace/dir'"
-ANA_NB_E2E_ENABLED="True"
-```
-
-You can run the notebook via the "Run All" command in Jupyter or you can execute it headless
-from the command line:
-```shell
-jupyter nbconvert --to notebook --execute trace-analysis.ipynb
-```
-
-The notebook also supports invocation via [Papermill](https://papermill.readthedocs.io/en/latest/).
-
-The output files are found in the configured output dir (default: `out/`). 
-Inputs are processed and cached in `cache/`.
-
-## Outputs
-
-The `plot_e2es_violin_XYZ.csv` files correspond to the latencies for each recorded dataflow on path item XYZ.
-The labels for these path items are found in `plot_e2es_violin_labels.csv`.
-The latency types (DDS, Idle, Computation) for these path items are found in `plot_e2es_violin_types.csv`.
-`calc_times.csv` provides calculation times for all callbacks on the path (not only until publication, which is the case for the computation times above).
-
-
-## Known Issues
-
-* The histograms output in the notebook for Idle, Computation and DDS times, do not aggregate these values over the end-to-end path. Thus, the latencies of all path elements are added into the same histogram.

ReadMe.md

@@ -0,0 +1,81 @@
+# ROS 2 Trace Analysis for Semantic Scheduling (Thesis)
+
+This repo provides the data analysis framework used for my thesis  
+**"Semantic Scheduling for Multi-Modal Sensor Data on the Example of ROS 2"** (TU Berlin & DLR, 2025).
+
+It is based on [TUM-AVS/ros2_latency_analysis](https://github.com/TUM-AVS/ros2_latency_analysis) (`dataflow-analysis` branch) by Betz et al.  
+**Please cite their original work when using this code (see below).**
+
+## Key Features
+
+- Automated extraction of end-to-end and component latencies from ROS 2 traces
+- Extended batch processing, including meta-anaylsis
+- Scripts and fixes tailored for semantic scheduling experiments
+
+## Main Additions & Changes
+
+- **New scripts:**  
+  `add_csv_header.sh`, `batch_analysis_analysis.py`, `csv2table.py`, `csvfix.sh`,  
+  `run_batch_analysis_analysis.sh`, `run_batch_analyze.sh`
+- **Main modifications:**  
+  `batch_analyze.py`, `trace-analysis.ipynb`
+- **Submodules:**  
+  Includes `ros2_tracing` and `tracetools_analysis`
+
+All changes are documented in the git history.
+
+## Setup
+
+- Requires **Python 3.10+**, ROS 2, JupyterLab, and [ros2_tracing](https://github.com/ros2/ros2_tracing)
+- Install dependencies:  
+  `pip3.10 install -r requirements.txt`
+- Run interactive analysis in Jupyter (`trace-analysis.ipynb`) or batch via provided scripts.
+
+See the [upstream README](https://github.com/TUM-AVS/ros2_latency_analysis/tree/dataflow-analysis) for more details on prerequisites and methodology.
+
+## Usage
+
+First, make sure all your traces are in sub-directories conforming to the common experiment type, e.g.:
+```
+traces/
+  ros_multi/
+    ros_multi_1
+    ros_multi_2
+    ...
+    ros_multi_n
+  ros_single/
+    ros_single_1
+    ros_single_2
+    ...
+    ros_single_m
+```
+Then use `run_batch_analyze.sh` to run the analysis (trace-analysis.ipynb) for the traces you gathered. This will create a summarizing `results.csv` in the respecitve directory (here thus `traces/ros_multi/` and 'traces/ros_multi/').  
+The csv will not have headers, to add them, use `add_csv_header.sh`.  
+Now you can run `run_batch_analysis_analysis.sh` to generate the end-to-end analysis box-plots from that results.csv (will also generate a `results_summary.csv`). The latter is ideal to be turned into LaTeX tables, using `csv2table.py`.
+
+## Output
+
+- Latency CSVs (E2E and per-chain), plots (PDF/PNG), and LaTeX-ready tables
+
+## Citation
+
+If you use this code, **please cite**:
+
+Betz, T. et al., “Latency Measurement for Autonomous Driving Software Using Data Flow Extraction,”  
+*IEEE Intelligent Vehicles Symposium (IV), 2023.*  
+[DOI: 10.1109/iv55152.2023.10186686](https://doi.org/10.1109/iv55152.2023.10186686)
+
+```bibtex
+@inproceedings{Betz2023,
+  doi = {10.1109/iv55152.2023.10186686},
+  url = {https://doi.org/10.1109/iv55152.2023.10186686},
+  year = {2023},
+  author = {Tobias Betz and Maximilian Schmeller and Andreas Korb and Johannes Betz},
+  title = {Latency Measurement for Autonomous Driving Software Using Data Flow Extraction},
+  booktitle = {2023 {IEEE} Intelligent Vehicles Symposium ({IV})}
+}
+```
+
+## License
+
+See upstream for original license; thesis-specific additions are for academic/non-commercial use.

traces/experiments2/supplementary.csv

@@ -1,12 +0,0 @@
-chain,input_delay,ideal_execution_time
-/input/baroA/alt --> /output/flight/cmd,100,44
-/input/baroB/alt --> /output/telemetry/radio,100,23
-/input/cameraA/raw --> /output/cameraA/mapped,200,68
-/input/cameraA/raw --> /output/telemetry/radio,200,76
-/input/cameraB/raw --> /output/classifier/classification,150,75
-/input/gpsA/fix --> /output/flight/cmd,100,44
-/input/gpsB/fix --> /output/telemetry/radio,100,23
-/input/imuA/data --> /output/flight/cmd,100,44
-/input/imuB/data --> /output/telemetry/radio,100,23
-/input/lidar/scan --> /output/flight/cmd,100,30
-/input/operator/commands --> /output/flight/cmd,100,30

traces/experiments2/supplementary_timed.csv

@@ -1,12 +0,0 @@
-chain,input_delay,ideal_execution_time
-/input/baroA/alt --> /output/flight/cmd,14,44
-/input/baroB/alt --> /output/telemetry/radio,6,23
-/input/cameraA/raw --> /output/cameraA/mapped,200,68
-/input/cameraA/raw --> /output/telemetry/radio,6,76
-/input/cameraB/raw --> /output/classifier/classification,150,75
-/input/gpsA/fix --> /output/flight/cmd,14,44
-/input/gpsB/fix --> /output/telemetry/radio,6,23
-/input/imuA/data --> /output/flight/cmd,14,44
-/input/imuB/data --> /output/telemetry/radio,6,23
-/input/lidar/scan --> /output/flight/cmd,18,30
-/input/operator/commands --> /output/flight/cmd,18,30