Objective Few-mode fibers can support the transmission of multiple low-order modes, making them suitable for high-capacity laser communication systems. Addressing the coupling characteristics of space light to few-mode fibers and the evaluation of communication bit error rates under atmospheric turbulence, a model for coupling efficiency of space light to few-mode fibers under atmospheric turbulence has been established. This model expands the modal field distribution of a four-mode fiber based on Laguerre-Gaussian modes, introducing the propagation distance from the beam waist to the receiving plane as a variable, and constructs the coupling efficiency expression by combining the coherence function. The model considers the negative effects of wavefront curvature and phase factors on the coupling efficiency of the system. Additionally, a bit error rate model of the communication system is constructed in conjunction with the Málaga turbulence channel, achieving a unified modeling from the coupling process to system performance. This model can be used to analyze the variation of coupling efficiency and bit error rate under different system parameters. The results indicate that under weak turbulence conditions, the total coupling efficiency of the four-mode fiber is approximately 7.1% higher compared to that of single-mode fiber, reaching a maximum system coupling efficiency of 88.24% when the coupling parameter (the ratio of the receiving aperture radius to the receiving plane spot radius) is 1.684. It is also found that few-mode fibers can effectively enhance communication performance under turbulence. The established model achieves a unified analysis of space light coupling efficiency and communication system bit error rate, revealing the relationship between key parameters and communication performance, thus providing theoretical support for parameter selection in space laser communication systems.

Methods This article establishes a coupling efficiency model for spatial light coupling to few-mode fibers under atmospheric turbulence conditions, as well as a Bit Error Rate model for communication systems based on Málaga turbulent channels. For the coupling efficiency model, the variations in coupling efficiency of different modes in few-mode fibers are analyzed by comparing the coupling efficiency of single-mode and four-mode fibers under various system parameters. Regarding the BER model, simulations are conducted using single-mode, dual-mode, and four-mode fibers to evaluate the changes in system BER under different turbulence intensities, lens apertures, and link distances.

Results and Discussions Simulations based on established models indicate that under weak turbulence conditions, the coupling efficiency of four-mode fibers can be improved by approximately 7.1% compared to single-mode fibers, with maximum coupling efficiency reaching 88.24% when the coupling parameters are set to 1.684. Additionally, the simulations show that as the distance from the beam waist to the coupling lens increases, there is a significant adverse effect on coupling efficiency. Furthermore, as the turbulence intensity increases, the coupling efficiency of four-mode fibers will also decline. In terms of communication performance, the simulation results of the bit error rate model demonstrate that, compared to single-mode fiber systems, few-mode fiber systems consistently exhibit lower bit error rates under various turbulence intensity conditions, indicating that few-mode fibers possess superior communication performance in turbulent transmission.

Conclusions This paper develops a coupling efficiency model for free-space optical beams coupled into few-mode fibers under atmospheric turbulence, together with a communication bit error rate model based on the Málaga distribution. Results show that the coupling efficiency of four-mode fibers improves by about 7.1% compared with single-mode fibers, reaching 88.24% at a coupling parameter of 1.684. The efficiency decreases with transmission distance and turbulence strength, while higher-order modes exhibit a rise-then-fall trend under turbulence. BER analysis further indicates that system performance is jointly influenced by turbulence, lens aperture, and link distance, and that few-mode fibers provide clear performance advantages. Compared with existing models, this work introduces transmission distance as a key variable and achieves unified modeling of coupling efficiency and BER, offering theoretical guidance for free-space optical communication system design.