Our investigation reveals deviations from the predictions of continuous electrodynamics models, including density-dependent line shifts and broadening effects. In particular, our results demonstrate that by carefully controlling the saturation of single atoms and the interactions among multiple atoms using nanostructures, along with controlling the geometry of the atomic cloud, it becomes possible to manipulate the effective optical nonlinearity of the entire atomic ensemble. This capability renders the hybrid thermal atom-nanophotonic platform a distinctive and valuable one for manipulating the collective effect and achieving substantial optical nonlinearities.