The Sensors
Overview
MmCows is a large-scale multimodal dataset for behavior monitoring, health management, and dietary management of dairy cattle.
The dataset consists of data from 16 dairy cows collected during a 14-day real-world deployment, divided into two modality groups. The primary group includes 3D UWB location, cows' neck IMMU acceleration, air pressure, cows' CBT, ankle acceleration, multi-view RGB images, indoor THI, outdoor weather, and milk yield. The secondary group contains measured UWB distances, cows' head direction, lying behavior, and health records.
MmCows also contains 20,000 isometric-view images from multiple camera views in one day that are annotated with cows' ID and their behavior as the ground truth. The annotated cow IDs from multi-views are used to derive their 3D body location ground truth.
The Sensors
The sensor suite consists of a neck-mounted collar tag, a vaginal temperature logger, and an ankle accelerometer for each of 10 cows, as well as four stationary cameras and six environmental sensors. The neck tag records distances from the cow to eight stationary UWB anchors, the acceleration and magnetic field, and the ambient air pressure. The temperature logger measures the core body temperature, while the ankle accelerometer records the leg direction of the cow.
The cameras provide visual data for identification, localization, and ground truth labeling of all 16 cows, while the environmental sensors measure indoor ambient conditions. Additional data such as outdoor weather is recorded by a nearby weather station, and milk weight is logged by the barn staff. All sensors are synchronized to the internet time.
The Barn Setup
The cows are housed in a 20x12m pen where they only leave for milking twice daily for approximately 30 min each time. Eight stationary UWB anchors are installed around the pen that work with the neck tags to provide 3D locations of the cows. Four high-resolution cameras are mounted at four corners of the pen to capture isometric-view images of the cows.
Ground Truth
The ground truth of MmCows comprises visual cow IDs and behavior labels. Out of 4.8M image frames, 20k UWB-synced frames from all four cameras in one day are selected for cow ID annotation. We defined and annotated seven behaviors of all 16 cows at the granularity of one second during the chosen day. Details of the annotation rules can be found here.
Data Processing Flow
Some of the measured parameters are processed to derive more meaningful data that indicates the behaviors of the cows.
The heading direction related to feeding and drinking behaviors is obtained from the immu.
The lying behavior is classified from the ankle acceleration.
The 3D neck location uwb is calculated from UWB distances, while the 3D body location ground truth is derived from the ID-annotated labels.
Citation and Usage
Hien Vu, Omkar Prabhune, Unmesh Raskar, Dimuth Panditharatne, Hanwook Chung, Christopher Choi, and Younghyun Kim, "MmCows: A Multimodal Dataset for Dairy Cattle Monitoring", Advances in Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks, 2024. URL: https://github.com/neis-lab/mmcows
Manuscripts: Publisher, Main Paper, Supplementary Materials, Poster
@inproceedings{mmcows,
title = {MmCows: A Multimodal Dataset for Dairy Cattle Monitoring},
author = {Hien Vu and Omkar Prabhune and Unmesh Raskar and Dimuth Panditharatne and Hanwook Chung and Christopher Choi and Younghyun Kim},
booktitle = {Advances in Neural Information Processing Systems},
year = {2024}
}
For any inquiries or assistance, please feel free to reach out to Hien Vu at hienvu [at] purdue.edu.
MmCows is available under the CC BY-NC-SA 4.0 license. You can use, copy, and redistribute the material for non-commercial purposes, provided you credit the original source, include a link to the license, and indicate any modifications made. If you alter, adapt, or expand upon the material, your contributions must be distributed under the same license as the original. Commercial use of the material is not permitted.