We report an atomic magnetic sensor that leverages the spatial symmetry-broken (SSB) properties of the four-wave mixing (FWM) process induced by an external weak magnetic field. The dispersive signals around the center of zero magnetic field are obtained from counterpropagating polarization-selective FWM signals in a rubidium vapor cell. The SSB magnetometer has a linear response within −5 μT to 5 μT, and the magnetic sensitivity is about 10 pTHz at 10 Hz, and the bandwidth is in the range of 18.25 kHz. This method would open diverse applications ranging from biomagnetic sensing to Earth’s magnetic field mapping as a robust atomic magnetic sensor.