Nanophotonic modes within rectangular cross sections are typically considered to have transverse rectangular field profiles. In this work, we show that, despite the rectangular cross section of most integrated waveguides and microring resonators, there exists considerable hybridization of transverse rectangular modes and transverse circular modes. These hybridized modes can be advantageous in nonlinear wave mixing processes. We use third-harmonic generation as an example to confirm that such a hybridized mode is advantageous in combining reasonable mode overlap and waveguide coupling to a fundamental mode in a silicon nitride microring. Our work illuminates the potential of using transverse circular modes in nanophotonic applications. (C) 2021 Optical Society of America