Modeling of Signal-Noise Interactions in Nonlinear Fiber Transmission with Different Modulation Formats