Quantum two-level systems (TLSs) are present in the materials of qubits and are considered defects because they limit qubit coherence. For superconducting qubits, the quintessential Josephson junction barrier is made of amorphous alumina, which hosts TLSs. However, TLSs are not generally understood, either structurally or in atomic composition. In this study, we greatly extend the quantitative data available on TLSs by reporting on the physical dipole moment in two alumina types: polycrystalline gamma-Al2O3 and amorphous a-AlOx. To obtain the dipole moments p(z), rather than the less-structural coupling parameter g, we tune individual TLSs with a known external electric field in a cavity quantum electrodynamic system. We find a clear difference in the dipole moment distribution from the film types, indicating a difference in TLS structures. A large sample of approximately 400 individual TLSs are analyzed from the polycrystalline film type. Their dipoles along the growth direction p(z) have a mean value of 2.6 +/- 0.3 debye (D) (0.54 +/- 0.06 e angstrom) and standard deviation sigma = 1.6 +/- 0.2 D (0.33 +/- 0.03 e angstrom). The material distribution fits well to a single Gaussian function. Approximately 200 individual TLSs are analyzed from amorphous films. Both the mean p(z) = 4.6 +/- 0.5 D (0.96 +/- 0.1 e angstrom) and sigma = 2.5 +/- 0.3 D (0.52 +/- 0.05 e angstrom) are larger. Amorphous alumina also has very large p(z), greater than 8.6 D (1.8 e angstrom), in contrast to polycrystalline which has none of this moment. These large moments agree only with oxygen-based TLS models. Based on data and the candidate models (delocalized O and hydrogen-based TLSs), we find polycrystalline alumina has smaller ratio of O-based to H-based TLS than amorphous alumina.