Demonstration experiment of CSI sensing technology and AI utilization for the 6G era

1. About Radio Wave Sensing

In the 6G era, efforts to expand the possibilities of mobile communications are gaining momentum. Three key themes have emerged: "Fusion of Sensing and Communication," "Fusion of AI and Communication," and "Ubiquitous Communication."
Radio wave sensing can be broadly categorized into two types: radar-based methods and CSI-based methods. While we have previously discussed the differences between these two technologies, this article delves deeper into the CSI-based method.

Figure1. Comparison of Radio Wave Sensing Methods

In mobile communications such as LTE and 5G, and in wireless data communications such as wireless LAN, a process called demodulation is carried out to extract data from the signals received by the radio waves. As this wireless data travels through the air, the waveform of the radio waves changes due to various factors. When the quality of the signal degrades due to these waveform changes, data communication may fail, leading to a decrease in communication speed.
To address the degradation of signal quality, reference signals are employed. In recent wireless data communication standards, almost all specifications include these reference signals, allowing the receiving terminal to infer the changes that occurred as the radio waves traveled through the air.
Specifically, in the communication standards we commonly use in daily life, such as LTE, 5G, and wireless LAN (like IEEE 802.11n), a modulation method called OFDM (Orthogonal Frequency Division Multiplexing) is employed, and CSI (Channel State Information) is defined within these standards. CSI contains information about the changes in the signal and is utilized to enhance the quality of techniques such as beamforming and MIMO (Multiple Input Multiple Output).
In CSI sensing, the state of the space (hereinafter referred to as the "channel") through which the radio waves travel is inferred based on the CSI values. For example, when a person enters a previously empty room, the state of the space changes. By observing and interpreting changes in power (signal amplitude) and arrival time, (signal phase) it becomes possible to calculate and infer what changes are happening within the space. This technology is known as "CSI sensing technology."

Figure2. Principle of CSI Sensing - Detection Image

This CSI sensing technology utilizes information about changes in the spatial environment, similar to spatial division MIMO technology. Therefore, the more antennas and the broader the frequency bandwidth, the more information can be obtained.

CSI sensing is a technology that understands changes between wireless devices based on the variations in the amplitude and phase of radio waves. The specific applications depend on the type of information being sought. Examples include detecting that a change or anomaly has occurred ("detection"), determining what is present ("identification"), and counting the number of objects ("counting").
Consider a scenario where a wireless device for data communication is placed in a room. If the room is initially unoccupied and an object or person enters, parts of the radio waves may become obstructed, or additional reflective surfaces may appear. This results in changes in the amplitude and phase of the signals received. If the goal is simply to detect that a change has occurred, the receiver can determine that "some change has happened" based on these variations. However, tasks like “identification” or “counting” require a more detailed analysis of the differences in the amplitude and phase shifts caused by each individual object or person entering the room.

The development and research of CSI sensing technology have been underway since around 2010. However, with recent advancements in AI, the technology has regained significant attention. High-precision tasks such as “identification” and “counting” require analyzing signal changes by breaking them into detailed time-series patterns. This demand aligns closely with the strengths of AI, which has made the integration of the two increasingly attractive.
Furthermore, these patterns are highly dependent on the surrounding environment, as they vary with changes in radio wave reflections. This environmental dependence makes it challenging to implement CSI sensing in versatile systems. As a result, its applications are expected to focus on relatively stable environments, such as indoors, where it can be effectively used for security and monitoring purposes.

2. Global Trends Toward the Practical Use of CSI Sensing

Recently, solutions utilizing CSI information from wireless LANs for human detection have begun to emerge.
Commercial products leveraging the "detection" feature include solutions to prevent children from being left behind in vehicles and solutions for indoor security [1][2].
In academic research, studies are being conducted on "identification" and "counting," as well as more advanced estimations. A research team from Carnegie Mellon University successfully detected human skeletons by combining CSI information from wireless LANs with camera data, as reported in a 2023 paper [3].
Regarding standardization, discussions are underway in IEEE 802.11bf to establish standards based on CSI sensing, which will allow for high-precision location information using Wi-Fi signals while ensuring compatibility with existing Wi-Fi standards. The IEEE 802.11bf standard is expected to reach final agreement by 2025 [4][5].

[1]https://www.inspirecorp.co.jp/news/pressrelease/origin-20240710
[2]https://ai6.jp/
[3]https://arxiv.org/abs/2301.00250
[4]https://fst.sophia.ac.jp/wp/wp-content/uploads/2024/02/k105_12_1466.pdf
[5]https://www.ieee802.org/11/Reports/tgbf_update.htm

3. Develop and Verify a CSI Sensing x AI Model

At SoftBank, we aim to realize real-world applications of CSI sensing using signals from both indoor and outdoor base stations in the future. As a first step, we developed and tested CSI sensing in a conference room within SoftBank's offices.
For this experiment, our goal was to develop an AI model to determine human posture. We used WLAN CSI information for the experiment. The diagram below outlines the test system and shows the actual conference room within our office where the tests were conducted.

Figure3. Overview of the Posture Detection System

Figure4. Conference Room Used for Testing

The following table shows the wireless specifications used in this experiment.

Table. Wireless Specifications

The tool used to obtain the CSI values is a Linux PC with the Linux 802.11n CSI Tool installed.
Reference：https://dhalperi.github.io/linux-80211n-csitool/

＜AI Model Development＞

The AI model was built based on five machine learning models.
The first is Linear Discriminant Analysis (LDA). This model is a statistical method widely used for multi-class classification problems, primarily aiming to find the optimal linear boundary that separates data belonging to different classes. LDA maximizes the differences between classes and minimizes the differences within the same class, thereby enhancing classification accuracy. It is computationally efficient and easy to interpret, making it applicable in various fields.
The second is the Naive Bayes Support Vector Machine (NB-SVM). This model combines the Naive Bayes classifier with the Support Vector Machine (SVM) algorithm. It is suitable for handling high-dimensional data like the present CSI problem, demonstrating particularly excellent performance in text classification. In NB-SVM, the Naive Bayes method is first used to extract probability information for each feature, which is then input into the SVM for final classification. By combining the efficient feature processing of Naive Bayes with the classification power of SVM, we aimed for high-accuracy posture recognition.
The third model is the Kernel Support Vector Machine. This model employs the kernel method with the Support Vector Machine (SVM), mapping data that are difficult to separate linearly into a higher-dimensional space to facilitate separation. The kernel function allows for reduced computational complexity while addressing complex classification problems requiring non-linear separation. In this experiment, we selected the optimal kernel function to construct a model for higher accuracy in posture recognition.
The fourth model is Random Forest. This powerful ensemble learning algorithm combines multiple decision trees for prediction. Each decision tree has the ability to capture non-linear patterns, and Random Forest makes the final prediction based on the majority vote of these trees. Using multiple decision trees prevents overfitting and enhances the generalization performance of the model. In this study, we built a posture recognition model using Random Forest to accurately and robustly classify various posture patterns.
The fifth model is Deep Learning. For this case, we built the model based on Convolutional Neural Networks (CNN), which are commonly used in image recognition.

In this experiment, we optimized the input shape of the CNN model to the characteristics of the CSI data. First, we excluded phase data from the conventional five-dimensional data structure and used only amplitude data, converting it into a four-dimensional structure. This transformation aimed to avoid instability caused by learning different characteristics of amplitude and phase simultaneously, thereby improving the model's convergence.
Next, we integrated the combinations of transmitting and receiving antennas into a three-dimensional data structure, which was then transformed into the final input shape through batch processing and dimension rearrangement. The overall architecture of the CNN model we created is shown in the figure below. This model structure allows us to capture the inherent temporal and spatial features in the CSI data effectively, aiming for high-accuracy posture recognition. Additionally, this method seeks to achieve more robust and diverse feature representations compared to conventional CSI-based posture recognition methods.

Diagram. Overall Architecture of the CNN Model

For the test data, we performed averaging in the time direction to remove noise effects from the acquired CSI data, processed it into a one-dimensional array, and then input it into the AI model. The output was classified into three types: "Standing," "Sitting," and "Sleeping."

In this experiment aimed at human posture recognition, we obtained 2000 samples of data for each of the three states: standing, sitting on a chair, and lying down on the floor. We varied the individuals to obtain the data to provide generalizability to the model. The number of data samples obtained was 6000 (3 postures x 2000 samples), 90% of which were used as training data and 10% as test data.

4. Results and Insights from the AI Model Determination

The best results were obtained with the deep learning CNN model, achieving an accuracy of 86%. The video below shows the scene when we incorporated the CNN model into the actual test system and executed it.

Movie. Demonstration video of posture recognition model using CSI sensing

From these results, we successfully identified human postures using CSI sensing technology in an indoor conference room setting.

In this experiment, a significant portion of our time was spent acquiring training data using actual subjects. If methods for automating the acquisition and annotation of training data could be established, or if a model could be built with a small amount of data through fine-tuning, it would be possible to shorten the development period and broaden the range of applications to various environments.
However, when we deployed the developed model in a different environment, both the deep learning model and other machine learning models showed a significant drop in accuracy. This highlights the importance of incorporating training data from multiple environments to mitigate the impact of environmental differences in CSI sensing technology.

5. Future Prospects

This article introduced SoftBank's research and development efforts towards the practical application of CSI sensing technology. In this experiment, we succeeded in developing an AI model for posture recognition.
However, the experiment also revealed challenges, such as the environmental dependency of CSI sensing.
Currently, in the world of mobile communications, the integration of communication technology and sensing technology towards 6G is being discussed. Leveraging the experience from this experimental demonstration, SoftBank will continue to advance its research and development efforts toward the realization of 6G.

Writer：Mizuki Funakoshi, Tomoya Tanaka, Ayumu Yabuki