Cavity quantum electrodynamics (cQED), the interaction of a two-level system with a high quality factor (Q) cavity, is a foundational building block in different architectures for quantum computation, communication, and metrology. The strong interaction between the atom and the cavity enables single-photon operation, which is required for quantum gates and sources. Cold atoms, quantum dots, and color centers in crystals are among the systems that have shown single-photon operations, but they require significant physical infrastructure. Atomic vapors, on the other hand, require limited experimental infrastructure and are hence much easier to deploy outside a laboratory, but they consist of an ensemble of moving atoms that results in short interaction times involving multiple atoms, which can hamper quantum operations. A solution to this issue can be found in nanophotonic cavities, where the optical mode is confined to a small volume and light-matter interaction is enhanced, so that fast single-atom, single-photon operations are enabled. In this work, we study the interaction of an atomically clad microring resonator (ACMRR) with different-sized ensembles of Rb atoms. We demonstrate strong coupling between an ensemble of $\approx$50 atoms interacting with a high quality factor (Q$=$4.3×105) ACMRR, yielding a many-atom cooperativity C$=$(5.5±0.3). We continue to observe signatures of atom-photon interaction for a few (<3) atoms, for which we observe saturation at the level of a few intracavity photons. Further development of our platform, which includes integrated thermo-optic heaters to enable cavity tuning and stabilization, should enable the observation of interactions between single photons and single atoms.