Although Lithium ion batteries offer numerous advantages (e.g. energy density, efficiency, etc.) over other types of batteries, recent thermally driven accidents involving consumer electronics have generated curiosity and necessitated a deeper understanding of the thermal behavior of these batteries. Thermal transport across the multilayer stacks that form prismatic and spiral-wound batteries generally hinders heat removal from the system. In cylindrical batteries, most commonly found in laptops, electric vehicles and power banks, heat must conduct through to the metallic shell through many layers of the anode-separator-cathode structure, which is of low effective thermal conductivity. Specifically, the interfacial conductance across the jellyroll of the electrodes and the metallic cases of Lithium Ion batteries has not been previously measured, and the impact of varying internal pressure is virtually unknown. This work presents thermal conductance and thermal conductivity measurements of dry 18650 cells using infrared microscopy and a miniaturized radial version of the conventional reference bar method. Two-dimensional temperature maps are captured by high resolution infrared microscopy with up to 1.7 µm spatial resolution and 0.1 K temperature resolution. The two-dimensional profiles are averaged in the direction normal to the heat flow to generate one-dimensional temperature profiles. Interfacial temperature jumps clearly indicate thermal resistances and can be separated from the thermal gradients due to conduction within a single material. Thus, we measure both cross-plane thermal conductivity of the layers that make up the battery stack (by comparing the temperature gradients within the stack) and interfacial thermal resistance. The mean thermal conductance at the interface between the jelly roll and the case averaged across all samples and all temperatures is 917 W/m² /K, with a standard deviation of 474 W/m2 /K. Thermal conductance in active batteries is expected to be higher, due to the presence of a liquid electrolyte. This work demonstrates that the low cross plane thermal conductivity of the plastic separator material is one of the limiting factors in heat dissipation.