JoWUA

Abstract

The development of MIMO-OTFS (Multiple Input Multiple Output–Orthogonal Time Frequency Space) has proven to be a great breakthrough and a highly viable waveform for next-generation and future 5G wireless communication networks. However, it suffers from performance degradation when MMSE and ZF are applied to large-scale MIMO systems and fast-changing environments. In order to overcome these limitations, this paper proposes an MNSO-based ensemble learning approach for detecting signals in MIMO-OTFS systems. This approach is based on using Modified Non-Smooth Optimization (MNSO) for dimensionality reduction in conjunction with a two-level ensemble learning model. At the first level, there are machine learning classifiers, namely Linear Regression, Random Forest, Decision Tree, and Support Vector Machine, to detect feature-level signals, and at the second level, there is a Gradient Boosting Bayesian Neural Network (GB-BNN) that learns non-linearities in the delay-Doppler domain. In order to test the system, use a set of 5000 signals generated in MATLAB under Rayleigh and Rician fading with 16×16, 64×64, and 256×256 MIMO structures and 0 to 10 dB SNR values. The experimental findings show that the proposed ensemble model using MNSO shows considerable success in reducing BER from 0.0600 at low SNR to 0.0030 at high SNR. In the case of 256×256 MIMO configurations, the proposed model can achieve up to 83.33% error reduction from the traditional MMSE techniques. The ablation study shows the effectiveness of the MNSO as well as ensemble techniques in improving the detection accuracy while reducing complexity. Conclusion: In summary, the proposed framework is scalable and robust and can provide high-accuracy detection solutions for B5G MIMO-OTFS communication systems.