构建可以检测潮热并触发动作来缓解人的设备

描述

概述

潮热是上半身突然感到温暖，通常在面部、颈部和胸部最为强烈。尽管其他医疗条件也可能导致潮热，但潮热最常见的原因是更年期。在这个项目中，我构建了一个可以用来检测潮热并触发一些动作来缓解人的设备，在这种情况下，使用红外发射器打开空调冷却系统。作为输入，它采用多维红外热传感器数据。它的输出将是一个简单的分类，如果有人被识别并且最近发生了突然的温度变化，它会通知我们。

硬件考虑

我想做一个可以读取体温数据的可穿戴设备。在做了一些研究和实施之后，我发现一直佩戴传感器，尤其是在睡觉时，在很多情况下都是不舒服的，而且是不可行的。潮热也会导致出汗，佩戴电池供电的设备需要特殊的外壳才能准确测量数据。此外，对于冷却，我研究了热电 Peltier 冷却器模块，发现它们中的大多数都需要更大的 12V 电池才能正常运行以实现最佳冷却，如果不仔细控制它也是不安全的并且可能导致灼伤。对于非侵入式解决方案，我发现低分辨率热像仪可以从远处可靠地读取温度，并且它们可以在黑暗中工作。虽然读取精度可能会受到距离的影响，但通过计算可以准确记录热图像数据的连续或突然变化。我选择 Atom Matrix 作为微控制器设备，因为它体积小、价格便宜、可以运行 TensorFlow 模型并且有一个可以用作遥控器的 IR 发射器。我选择 Wio Terminal 来捕获热像仪数据进行训练，因为它有一个带有许多按钮的 LCD 屏幕，用于捕获不同类别的数据。

训练数据收集

机器学习项目的第一步也是最重要的一步是以这样一种方式收集训练数据，即它应该涵盖给定识别任务的大多数代表性案例。Wio 终端上的 3 个按钮用于标记 3 个类别。

捕获的数据将保存到连接到 Wio 终端的微型 SD 卡上的文件中。每个热图像数据都被捕获为一个单独的文件。该文件不包含标题行，只有逗号分隔的 768 (24x32) 温度读数。示例读数文件内容如下所示。

26.47,25.97,25.85,25.72,26.90,26.12,26.60,26.86,27.00,26.68,26.90,26.74,27.78,27.21,27.75,29.12,31.29,31.50,32.24,31.95,31.72,30.80,31.29,30.69,31.18,30.86,31.46,31.37,29.21,28.23,28.18,28.03,25.83,26.33,25.55,26.56,26.59,26.90,26.52,27.38,26.94,27.39,26.85,27.21,27.32,27.66,28.81,30.45,30.97,31.74,31.55,32.14,31.37,31.03,30.63,30.69,31.03,31.52,30.85,31.14,28.80,28.57,27.81,28.39,26.43,26.24,26.67,26.71,27.13,26.99,27.63,28.07,28.59,28.39,27.80,28.19,28.11,28.25,30.91,32.15,31.78,31.46,31.82,31.33,31.10,30.43,30.37,30.06,29.77,29.84,30.58,30.45,29.28,28.42,28.34,27.76,26.31,26.62,26.38,27.24,27.27,27.91,28.94,29.11,30.11,29.73,30.25,29.53,29.59,29.22,32.01,32.70,33.17,32.00,31.15,31.52,30.59,30.46,29.87,30.07,29.43,30.09,30.01,30.68,29.10,28.91,27.99,28.34,26.59,26.60,26.99,27.49,28.68,29.64,31.88,33.14,33.41,33.02,32.48,32.83,32.60,32.60,33.58,34.16,34.79,34.58,32.43,32.15,31.07,30.77,29.84,29.89,30.01,29.48,29.99,29.63,29.33,28.47,28.23,27.90,26.26,26.26,27.13,28.21,30.50,31.41,33.23,33.70,33.44,33.40,33.38,33.18,33.31,33.12,33.65,34.33,34.81,34.93,33.96,32.50,31.29,30.82,29.37,29.93,29.13,29.93,29.29,30.07,28.76,29.00,28.38,28.69,26.83,26.55,27.36,28.68,32.50,33.12,33.78,33.40,34.17,34.14,33.80,33.56,33.84,33.35,33.85,33.54,34.63,34.45,34.68,34.49,31.76,30.94,29.87,29.38,29.67,29.20,29.45,29.68,29.02,28.42,28.63,28.49,26.29,27.30,27.51,28.71,31.96,33.70,33.86,33.76,33.87,34.32,33.60,33.94,33.41,33.39,33.74,34.04,34.32,34.95,34.24,34.63,31.81,31.15,29.59,29.78,29.19,29.62,28.77,29.72,28.88,29.12,28.51,28.79,26.45,27.15,28.06,28.72,32.45,33.05,33.89,33.84,33.60,33.39,34.02,33.69,33.64,33.29,34.11,33.81,34.68,34.55,34.75,34.15,32.55,31.42,30.25,29.76,29.56,29.42,29.64,28.59,28.87,28.66,29.10,28.79,26.76,27.06,27.39,28.97,31.63,33.46,33.78,34.22,33.97,34.11,33.58,34.20,34.01,33.97,33.69,34.16,34.54,34.90,34.33,34.63,33.09,31.81,29.81,30.15,28.89,29.81,28.96,29.39,28.70,29.35,28.54,29.07,27.18,26.59,27.31,27.98,31.52,32.56,33.37,33.52,33.72,33.72,33.52,33.22,33.75,33.27,33.93,34.13,34.69,34.65,34.26,34.12,32.70,31.12,30.58,30.45,29.91,29.42,29.72,28.82,29.66,29.10,28.91,29.04,26.26,26.72,26.66,27.70,29.47,31.74,32.43,33.58,33.48,33.45,32.63,32.79,31.94,33.32,33.36,34.14,34.42,34.55,33.99,33.87,31.15,30.99,30.15,30.84,29.81,30.02,29.24,29.61,29.15,29.18,28.82,29.32,27.21,26.60,27.01,26.83,27.83,27.64,29.27,29.08,30.97,29.96,29.96,28.30,29.66,30.77,34.03,33.60,34.23,33.26,32.36,31.42,30.61,30.63,30.44,30.50,30.45,30.46,29.70,29.37,29.70,29.34,29.17,28.83,26.28,26.53,26.36,27.31,26.68,27.55,27.46,28.27,27.83,28.53,27.72,28.01,28.14,30.84,32.27,33.38,32.09,32.70,31.33,31.32,30.03,30.53,29.86,30.86,30.35,30.88,29.78,29.66,29.58,29.62,29.06,29.77,26.63,26.16,26.65,26.85,27.10,26.79,27.12,26.67,27.47,27.14,27.15,27.15,27.93,28.86,31.42,31.62,31.87,31.27,31.11,30.65,30.47,30.46,30.62,30.17,30.45,29.94,29.73,29.38,29.33,29.33,29.18,29.06,25.92,26.74,26.20,26.93,27.03,26.89,26.71,26.92,26.90,27.08,26.74,27.46,27.06,28.62,31.36,32.15,31.96,31.97,31.03,31.01,30.20,30.71,31.08,31.04,30.51,30.21,29.31,29.81,29.18,29.43,29.07,29.56,26.80,26.61,26.85,26.61,26.86,26.81,26.86,27.13,27.21,26.94,26.84,26.77,27.24,26.68,30.82,31.45,32.56,31.93,31.30,31.13,31.15,31.29,31.83,31.84,31.02,30.16,29.38,29.19,29.56,29.13,29.41,28.83,26.54,26.56,26.58,26.95,26.81,27.11,26.44,27.07,26.81,27.04,26.77,27.05,26.85,26.94,28.29,30.73,31.36,31.84,30.73,30.88,30.73,30.90,31.61,31.92,30.55,30.41,28.76,29.27,29.02,29.64,29.32,29.59,26.79,26.32,26.78,26.93,26.85,26.80,26.95,27.14,26.90,26.59,26.60,26.76,27.19,27.05,27.35,27.16,28.37,28.20,29.84,29.51,31.84,32.69,32.54,31.55,31.03,30.36,29.18,28.63,29.54,28.92,29.18,29.26,26.28,26.71,26.30,27.09,26.60,26.93,26.71,27.05,26.46,27.39,26.90,27.22,26.64,27.33,26.61,27.39,27.15,28.14,28.76,29.57,31.26,32.76,31.79,31.63,30.77,31.15,29.09,29.45,28.78,28.99,29.07,29.83,26.96,26.68,26.70,26.47,26.79,26.53,26.86,26.56,27.08,26.60,26.86,26.50,27.11,26.74,27.29,27.29,27.16,27.50,28.66,28.47,30.35,30.02,30.71,30.36,31.65,31.08,30.13,29.60,29.28,29.08,29.37,29.02,26.45,26.86,26.63,27.07,26.53,27.05,26.85,27.21,26.48,27.28,26.80,27.12,26.54,27.20,26.52,27.19,26.77,27.61,27.91,28.58,28.91,29.42,29.19,30.28,30.07,31.03,30.25,29.85,29.17,29.61,29.30,29.79,26.88,26.71,26.97,26.73,27.01,26.79,26.83,26.99,27.51,26.93,26.96,26.79,27.22,26.88,27.34,26.87,27.20,27.23,27.70,27.45,28.61,28.18,29.45,29.06,29.99,29.77,30.61,29.50,29.72,29.00,29.41,30.15,25.94,27.25,26.77,26.97,26.82,27.24,26.67,27.18,26.75,27.16,27.01,27.37,26.91,27.49,26.81,27.24,26.89,27.97,27.36,28.21,27.73,28.77,28.52,28.88,29.20,30.00,29.95,30.37,29.49,29.48,28.84,29.91

捕获数据的可视化表示如下：

3 类热图像

 

代码部分中提到的 Github 存储库中提供了用于捕获训练数据的 Arduino 草图 (Thermal_camera_data_collection.ino)。为每个类别捕获了 100 多个样本。收集的数据已分为训练 (60%)、验证 (20%) 和测试 (20%) 数据集。由于数据是从已经校准的红外温度相机 (MLX90640) 收集的，并且它们已经在指定范围内，因此我们可以按原样使用原始数据进行训练和推理。

模型架构

我们可以将输入数据视为 24x36 像素的图像。卷积神经网络是适合识别图像和时间序列数据中的模式的最佳选择之一。前几层是带有少量其他正则化层的 2D 卷积神经网络。最后一层是具有softmax激活的全连接密集层，它输出所有3个类别的概率。该模型的总结如下。

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 24, 32, 8)         80        
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 24, 32, 8)         584       
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 12, 16, 8)         0         
_________________________________________________________________
dropout (Dropout)            (None, 12, 16, 8)         0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 12, 16, 8)         584       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 6, 8, 8)           0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 6, 8, 8)           0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 6, 8, 16)          1168      
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 3, 4, 16)          0         
_________________________________________________________________
dropout_2 (Dropout)          (None, 3, 4, 16)          0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 3, 4, 16)          2320      
_________________________________________________________________
flatten (Flatten)            (None, 192)               0         
_________________________________________________________________
dense (Dense)                (None, 64)                12352     
_________________________________________________________________
dropout_3 (Dropout)          (None, 64)                0         
_________________________________________________________________
dense_1 (Dense)              (None, 32)                2080      
_________________________________________________________________
dropout_4 (Dropout)          (None, 32)                0         
_________________________________________________________________
dense_2 (Dense)              (None, 3)                 99        
=================================================================
Total params: 19,267
Trainable params: 19,267
Non-trainable params: 0

模型训练和评估

该模型的训练是在带有 Linux 和 eGPU (NVIDIA GTX 1080Ti) 的 Intel NUC 上完成的。尽管在 CPU 上训练只需要几分钟，但在测试不同的架构和超参数时，开发过程变得非常缓慢。带有 Keras API 的 TensorFlow 2.1 用于模型创建和训练过程。我创建了一个 Jupyter notebook 用于数据处理、训练和最终模型转换。所有代码都可以在代码部分提到的 Github 存储库中找到。训练准确率为 99%，测试数据的评估准确率为 92.86%，可以通过更多的训练数据集和模型超参数调整进一步提高。

在设备上进行推理

将创建的模型转换为 TensorFlow Lite 模型，并将转换后的模型转换为 C 数组文件，以便与推理代码一起部署。TensorFlow Lite Micro SDK 用于在设备上运行推理。我创建了一个 Arduino 草图（Github 存储库中提供 Hot_flash_detector.ino），用于推断和显示结果。微控制器以 500 毫秒的间隔连续接收来自热像仪传感器的样本，并根据最近 20 个平均最高温度读数检测被识别人员的温度突然变化。

共同利益和成本的用例

这是一种易于使用的低功耗设备，可以使用电池运行数周。它可以安全地用于白天或晚上面临潮热问题的女性。该设备的成本也很低。最终工作产品（热像仪 + Atom 矩阵）的总成本远低于 50 美元，如果批量生产，还可以进一步降低。

改进范围

该设备可用于通过分析热图像时间序列数据来检测睡眠问题，并可以触发音乐系统播放一些舒缓的音乐或控制智能照明系统。此外，一些分析数据可以本地保存在 Atom Matrix 的 SPI 闪存中，并且可以使用手机应用程序通过 BLE 进行同步以进行进一步分析。