Objective Based on the master oscillator power amplifier (MOPA) structure, high-power fiber lasers have important applications in industrial, scientific, and defense fields. However, as the output power increases, transverse mode instability (TMI) in fiber amplifiers degrades the laser beam quality, ultimately limiting laser performance. Integrating a shorter-wavelength auxiliary laser into the seed source has been proposed as a novel approach to controlling the TMI threshold. Yet, experimental results from different studies have reported inconsistent effects on the TMI threshold. Therefore, establishing a TMI threshold model for dual-wavelength fiber amplifiers and clarifying the impact of the auxiliary laser are essential for further power scaling of high-power, high-brightness fiber lasers. This study developed a theoretical model for fiber amplifiers that accounts for dual-wavelength gain competition. Through numerical simulations, we investigate the relationship between the auxiliary laser wavelength and the TMI threshold, as well as the underlying physical mechanisms. Furthermore, we optimize the auxiliary laser parameters to maximize the TMI threshold.

Methods Based on the stimulated thermal Rayleigh scattering (STRS) theory model of TMI, this study developed a TMI threshold analysis model for a 20/400 μm ytterbium-doped fiber amplifier incorporating an auxiliary laser by introducing a dual-wavelength gain competition process. The model was validated by fitting experimental data of the 1080/1030 nm dual-wavelength amplification experiments. Using this model, simulations were conducted to evaluate the TMI threshold with respect to different auxiliary wavelengths. By comparing the evolution of STRS in amplifiers with different auxiliary wavelengths, the physical mechanism through which the auxiliary laser influences the TMI threshold was revealed: through gain competition with the signal laser, the auxiliary laser suppresses the peak intensity of the dynamic refractive index grating (RIG) while simultaneously extending its effective interaction length, thereby affecting the nonlinear gain of higher-order mode (HOM) and ultimately altering the TMI threshold. Furthermore, the regulatory mechanism of the auxiliary laser on the TMI threshold was analyzed. The optimization of auxiliary laser parameters was performed with constraint of maintaining a sufficient signal power ratio in the output.

Results and Discussions The validity of the model is demonstrated by the good agreement between the simulation results and the experimental data from other researchers (Fig.1). Simulations of the TMI threshold variation as a function of the auxiliary wavelength were performed under a 1∶1 signal-to-auxiliary power ratio in the seed (Fig.2). The results indicate that when the auxiliary wavelength is below 1050 nm, the TMI threshold decreases by up to 14%; when it exceeds 1050 nm, the threshold increases by up to 12.3%. Under identical output signal power conditions, the regulatory mechanism of different auxiliary wavelengths was clarified by analyzing the nonlinear gain coefficients of higher-order modes in three amplifier configurations with auxiliary wavelengths of 1035 nm, 1050 nm, and 1055 nm (Fig.5). Optimization of the auxiliary laser parameters was performed under the constraint that the signal power ratio remains 95%, 90% and 85% in the output laser (Fig.6). Under the three constraints, the optimal auxiliary wavelengths are 1052 nm, 1053 nm, and 1054 nm and the corresponding TMI threshold improvement ranges from 11% to 17%, while the optical efficiency of the system decreases from 86% to 77%.

Conclusions Based on the nonlinear mode coupling theory of STRS, this study established an analytical model for TMI effects in high-power ytterbium-doped fiber amplifiers that incorporates the gain competition process between the auxiliary and signal lasers. By quantitatively analyzing the influence of the auxiliary wavelength on the amplifier's TMI threshold and the STRS process, the physical mechanism by which the auxiliary laser regulates the TMI threshold is revealed: through gain competition with the signal laser, the auxiliary laser suppresses the peak intensity of the dynamic RIG while simultaneously extending its effective interaction length, thereby affecting the nonlinear gain of HOM and ultimately controlling the TMI threshold. By optimizing the parameters of the auxiliary laser while maintaining the optical-to-optical efficiency of the amplifier, the TMI threshold of the amplifier can theoretically be increased by 15%. The theoretical model and auxiliary laser parameter optimization method presented in this study are applicable to fiber laser amplification systems with different pumping configurations and operating wavelengths, providing research insights for further development of TMI suppression techniques in high-power fiber lasers.