tonio/esp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
esp/docs/superpowers/plans/2026-06-24-bme680-homelab-telemetry.md
Antoine 61de768d87 first commit
2026-06-24 23:13:57 +02:00

19 KiB
Raw Permalink Blame History

BME680 Homelab Telemetry Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add MQTT publishing to the ESP32 BME680 project and stand up a homelab ingestion pipeline that stores telemetry in InfluxDB through Telegraf.

Architecture: Keep the ESP32 focused on sensor reads, Wi-Fi, and MQTT publishing only. On the homelab side, validate Mosquitto -> Telegraf -> InfluxDB first using a synthetic MQTT message, then wire the real ESP32 publisher into the same topic and payload contract.

Tech Stack: ESP-IDF, esp-idf-lib/bme680, espressif/mqtt, Wi-Fi station mode, Mosquitto, Telegraf, InfluxDB 2, Docker Compose

Repo note: /home/tonio/Workspace/perso/esp is not a git repository right now, so this plan intentionally omits commit steps.

File Map

  • Modify: bme680-demo/main/CMakeLists.txt
  • Modify: bme680-demo/main/idf_component.yml
  • Modify: bme680-demo/main/main.c
  • Create: bme680-demo/main/Kconfig.projbuild
  • Create: bme680-demo/main/wifi_mqtt.h
  • Create: bme680-demo/main/wifi_mqtt.c
  • Create: homelab/telemetry/compose.yaml
  • Create: homelab/telemetry/.env.example
  • Create: homelab/telemetry/telegraf/telegraf.conf

Task 1: Create The Homelab Telemetry Stack

Files:

  • Create: homelab/telemetry/compose.yaml

  • Create: homelab/telemetry/.env.example

  • Create: homelab/telemetry/telegraf/telegraf.conf

  • Step 1: Create the environment template for InfluxDB and Telegraf

Write homelab/telemetry/.env.example:

INFLUXDB_ADMIN_USERNAME=admin
INFLUXDB_ADMIN_PASSWORD=change-me-please
INFLUXDB_ADMIN_TOKEN=replace-with-long-random-token
INFLUXDB_ORG=homelab
INFLUXDB_BUCKET=bme680
MQTT_BROKER_URL=tcp://192.168.1.10:1883
MQTT_USERNAME=
MQTT_PASSWORD=
  • Step 2: Create the Docker Compose file for InfluxDB and Telegraf

Write homelab/telemetry/compose.yaml:

services:
  influxdb:
    image: influxdb:2
    container_name: bme680-influxdb
    restart: unless-stopped
    ports:
      - "8086:8086"
    environment:
      DOCKER_INFLUXDB_INIT_MODE: setup
      DOCKER_INFLUXDB_INIT_USERNAME: ${INFLUXDB_ADMIN_USERNAME}
      DOCKER_INFLUXDB_INIT_PASSWORD: ${INFLUXDB_ADMIN_PASSWORD}
      DOCKER_INFLUXDB_INIT_ADMIN_TOKEN: ${INFLUXDB_ADMIN_TOKEN}
      DOCKER_INFLUXDB_INIT_ORG: ${INFLUXDB_ORG}
      DOCKER_INFLUXDB_INIT_BUCKET: ${INFLUXDB_BUCKET}
    volumes:
      - ./influxdb/data:/var/lib/influxdb2
      - ./influxdb/config:/etc/influxdb2

  telegraf:
    image: telegraf:1.31
    container_name: bme680-telegraf
    restart: unless-stopped
    depends_on:
      - influxdb
    environment:
      MQTT_BROKER_URL: ${MQTT_BROKER_URL}
      MQTT_USERNAME: ${MQTT_USERNAME}
      MQTT_PASSWORD: ${MQTT_PASSWORD}
      INFLUXDB_URL: http://influxdb:8086
      INFLUXDB_ORG: ${INFLUXDB_ORG}
      INFLUXDB_BUCKET: ${INFLUXDB_BUCKET}
      INFLUXDB_TOKEN: ${INFLUXDB_ADMIN_TOKEN}
    volumes:
      - ./telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro
  • Step 3: Create the Telegraf MQTT-to-Influx configuration

Write homelab/telemetry/telegraf/telegraf.conf:

[agent]
  interval = "10s"
  flush_interval = "10s"
  omit_hostname = true

[[inputs.mqtt_consumer]]
  servers = ["${MQTT_BROKER_URL}"]
  topics = ["sensors/+/+/telemetry"]
  qos = 1
  username = "${MQTT_USERNAME}"
  password = "${MQTT_PASSWORD}"
  data_format = "json"
  tag_keys = ["device_id", "room"]
  topic_tag = "topic"
  name_override = "environment"

[[outputs.influxdb_v2]]
  urls = ["${INFLUXDB_URL}"]
  token = "${INFLUXDB_TOKEN}"
  organization = "${INFLUXDB_ORG}"
  bucket = "${INFLUXDB_BUCKET}"
  • Step 4: Validate the Compose configuration before starting containers

Run:

cp "/home/tonio/Workspace/perso/esp/homelab/telemetry/.env.example" "/home/tonio/Workspace/perso/esp/homelab/telemetry/.env" && docker compose --env-file "/home/tonio/Workspace/perso/esp/homelab/telemetry/.env" -f "/home/tonio/Workspace/perso/esp/homelab/telemetry/compose.yaml" config

Expected:

  • rendered Compose output prints successfully

  • no missing-variable or invalid-YAML errors appear

  • Step 5: Start the homelab services

Run:

docker compose --env-file "/home/tonio/Workspace/perso/esp/homelab/telemetry/.env" -f "/home/tonio/Workspace/perso/esp/homelab/telemetry/compose.yaml" up -d

Expected:

  • bme680-influxdb is running

  • bme680-telegraf is running

  • Step 6: Verify Telegraf can ingest a synthetic MQTT message before touching device code

Run:

mosquitto_pub -h 192.168.1.10 -t "sensors/office/bme680-1/telemetry" -m '{"device_id":"bme680-1","room":"office","temperature":24.8,"humidity":46.2,"pressure":1007.3,"gas_resistance":125430,"uptime_s":3812}'

Then query InfluxDB:

set -a && source "/home/tonio/Workspace/perso/esp/homelab/telemetry/.env" && set +a && docker exec bme680-influxdb influx query --org "$INFLUXDB_ORG" --token "$INFLUXDB_ADMIN_TOKEN" 'from(bucket: "'"$INFLUXDB_BUCKET"'" ) |> range(start: -10m) |> filter(fn: (r) => r._measurement == "environment") |> limit(n: 10)'

Expected:

  • query output shows environment points
  • tags include device_id=bme680-1 and room=office
  • fields include temperature, humidity, pressure, gas_resistance, and uptime_s

Task 2: Add ESP32 Telemetry Configuration And MQTT Dependency

Files:

  • Modify: bme680-demo/main/idf_component.yml

  • Create: bme680-demo/main/Kconfig.projbuild

  • Modify: bme680-demo/main/CMakeLists.txt

  • Step 1: Add the MQTT client dependency to the main component

Write bme680-demo/main/idf_component.yml:

dependencies:
  esp-idf-lib/bme680: "^1.0.7"
  espressif/mqtt: "*"
  • Step 2: Add menuconfig entries for Wi-Fi, MQTT, and topic metadata

Write bme680-demo/main/Kconfig.projbuild:

menu "BME680 Telemetry Configuration"

config BME680_WIFI_SSID
    string "Wi-Fi SSID"
    default ""

config BME680_WIFI_PASSWORD
    string "Wi-Fi password"
    default ""

config BME680_MQTT_URI
    string "MQTT broker URI"
    default "mqtt://192.168.1.10:1883"

config BME680_MQTT_USERNAME
    string "MQTT username"
    default ""

config BME680_MQTT_PASSWORD
    string "MQTT password"
    default ""

config BME680_DEVICE_ID
    string "Device ID"
    default "bme680-1"

config BME680_ROOM
    string "Room"
    default "office"

config BME680_TOPIC_PREFIX
    string "MQTT topic prefix"
    default "sensors"

endmenu
  • Step 3: Register the future Wi-Fi/MQTT source file in the component build

Write bme680-demo/main/CMakeLists.txt:

idf_component_register(SRCS "main.c" "wifi_mqtt.c"
                    INCLUDE_DIRS ".")
  • Step 4: Reconfigure the project so the new dependency and Kconfig entries are visible

Run:

source "/home/tonio/esp/esp-idf/export.sh" && idf.py -C "/home/tonio/Workspace/perso/esp/bme680-demo" reconfigure

Expected:

  • espressif/mqtt is resolved by the component manager if not already cached
  • idf.py menuconfig would now show BME680 Telemetry Configuration

Task 3: Add A Focused Wi-Fi And MQTT Transport Module

Files:

  • Create: bme680-demo/main/wifi_mqtt.h

  • Create: bme680-demo/main/wifi_mqtt.c

  • Step 1: Create the public transport interface header

Write bme680-demo/main/wifi_mqtt.h:

#pragma once

#include <stdbool.h>

#include "esp_err.h"

esp_err_t wifi_mqtt_start(void);
bool wifi_mqtt_is_ready(void);
esp_err_t wifi_mqtt_publish(const char *topic, const char *payload);
  • Step 2: Implement Wi-Fi station setup, MQTT lifecycle, and publish helper

Write bme680-demo/main/wifi_mqtt.c:

#include <string.h>

#include "esp_event.h"
#include "esp_log.h"
#include "esp_mac.h"
#include "esp_netif.h"
#include "esp_wifi.h"
#include "freertos/FreeRTOS.h"
#include "freertos/event_groups.h"
#include "mqtt_client.h"
#include "nvs_flash.h"
#include "sdkconfig.h"
#include "wifi_mqtt.h"

#define WIFI_CONNECTED_BIT BIT0
#define MQTT_CONNECTED_BIT BIT1

static const char *TAG = "wifi-mqtt";
static EventGroupHandle_t s_event_group;
static esp_mqtt_client_handle_t s_mqtt_client;
static bool s_mqtt_started;

static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data)
{
    esp_mqtt_event_handle_t event = event_data;

    if (event_id == MQTT_EVENT_CONNECTED) {
        xEventGroupSetBits(s_event_group, MQTT_CONNECTED_BIT);
        ESP_LOGI(TAG, "MQTT connected");
    } else if (event_id == MQTT_EVENT_DISCONNECTED) {
        xEventGroupClearBits(s_event_group, MQTT_CONNECTED_BIT);
        ESP_LOGW(TAG, "MQTT disconnected");
    }
}

static void wifi_event_handler(void *arg, esp_event_base_t event_base, int32_t event_id, void *event_data)
{
    if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_START) {
        esp_wifi_connect();
    } else if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_DISCONNECTED) {
        xEventGroupClearBits(s_event_group, WIFI_CONNECTED_BIT | MQTT_CONNECTED_BIT);
        ESP_LOGW(TAG, "Wi-Fi disconnected, retrying");
        esp_wifi_connect();
    } else if (event_base == IP_EVENT && event_id == IP_EVENT_STA_GOT_IP) {
        xEventGroupSetBits(s_event_group, WIFI_CONNECTED_BIT);
        ESP_LOGI(TAG, "Wi-Fi connected");
        if (!s_mqtt_started) {
            esp_mqtt_client_start(s_mqtt_client);
            s_mqtt_started = true;
        }
    }
}

esp_err_t wifi_mqtt_start(void)
{
    ESP_ERROR_CHECK(nvs_flash_init());
    ESP_ERROR_CHECK(esp_netif_init());
    ESP_ERROR_CHECK(esp_event_loop_create_default());
    esp_netif_create_default_wifi_sta();

    s_event_group = xEventGroupCreate();

    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&cfg));
    ESP_ERROR_CHECK(esp_event_handler_register(WIFI_EVENT, ESP_EVENT_ANY_ID, &wifi_event_handler, NULL));
    ESP_ERROR_CHECK(esp_event_handler_register(IP_EVENT, IP_EVENT_STA_GOT_IP, &wifi_event_handler, NULL));

    wifi_config_t wifi_config = {
        .sta = {
            .threshold.authmode = WIFI_AUTH_WPA2_PSK,
        },
    };
    strncpy((char *)wifi_config.sta.ssid, CONFIG_BME680_WIFI_SSID, sizeof(wifi_config.sta.ssid));
    strncpy((char *)wifi_config.sta.password, CONFIG_BME680_WIFI_PASSWORD, sizeof(wifi_config.sta.password));

    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA));
    ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &wifi_config));
    ESP_ERROR_CHECK(esp_wifi_start());

    esp_mqtt_client_config_t mqtt_cfg = {
        .broker.address.uri = CONFIG_BME680_MQTT_URI,
        .credentials.username = CONFIG_BME680_MQTT_USERNAME,
        .credentials.authentication.password = CONFIG_BME680_MQTT_PASSWORD,
    };

    s_mqtt_client = esp_mqtt_client_init(&mqtt_cfg);
    esp_mqtt_client_register_event(s_mqtt_client, ESP_EVENT_ANY_ID, mqtt_event_handler, NULL);

    return ESP_OK;
}

bool wifi_mqtt_is_ready(void)
{
    EventBits_t bits = xEventGroupGetBits(s_event_group);
    return (bits & WIFI_CONNECTED_BIT) && (bits & MQTT_CONNECTED_BIT);
}

esp_err_t wifi_mqtt_publish(const char *topic, const char *payload)
{
    if (!wifi_mqtt_is_ready()) {
        return ESP_ERR_INVALID_STATE;
    }

    int msg_id = esp_mqtt_client_publish(s_mqtt_client, topic, payload, 0, 1, 0);
    return msg_id >= 0 ? ESP_OK : ESP_FAIL;
}
  • Step 3: Build just enough to verify the new transport module compiles

Run:

source "/home/tonio/esp/esp-idf/export.sh" && idf.py -C "/home/tonio/Workspace/perso/esp/bme680-demo" build

Expected:

  • build passes with the new dependency and new source file
  • no undefined references to MQTT or Wi-Fi APIs remain

Task 4: Publish Real Sensor Readings From The Main Loop

Files:

  • Modify: bme680-demo/main/main.c

  • Step 1: Update the main app to start transport, build the topic, and publish JSON telemetry

Write bme680-demo/main/main.c:

#include <stdio.h>
#include <string.h>

#include "bme680.h"
#include "cJSON.h"
#include "driver/gpio.h"
#include "driver/i2c_master.h"
#include "esp_err.h"
#include "esp_log.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "i2cdev.h"
#include "sdkconfig.h"
#include "wifi_mqtt.h"

#define BME680_I2C_PORT I2C_NUM_0
#define BME680_SDA_GPIO GPIO_NUM_21
#define BME680_SCL_GPIO GPIO_NUM_22
#define BME680_ADDR BME680_I2C_ADDR_0
#define BME680_I2C_CLOCK_HZ 100000
#define BME680_INIT_RETRY_MS 2000
#define BME680_READ_INTERVAL_MS 3000

static const char *TAG = "bme680-demo";

static esp_err_t init_i2c_subsystem(void)
{
    esp_err_t err;

    do {
        err = i2cdev_init();
        if (err != ESP_OK) {
            ESP_LOGE(TAG, "I2C subsystem init failed: %s. Retrying in %d ms", esp_err_to_name(err), BME680_INIT_RETRY_MS);
            vTaskDelay(pdMS_TO_TICKS(BME680_INIT_RETRY_MS));
        }
    } while (err != ESP_OK);

    return ESP_OK;
}

static esp_err_t init_sensor(bme680_t *sensor)
{
    memset(sensor, 0, sizeof(*sensor));

    esp_err_t err = bme680_init_desc(sensor, BME680_ADDR, BME680_I2C_PORT, BME680_SDA_GPIO, BME680_SCL_GPIO);
    if (err != ESP_OK) {
        return err;
    }

    sensor->i2c_dev.cfg.master.clk_speed = BME680_I2C_CLOCK_HZ;

    err = bme680_init_sensor(sensor);
    if (err != ESP_OK) {
        bme680_free_desc(sensor);
        return err;
    }

    return bme680_set_ambient_temperature(sensor, 25);
}

static void wait_for_sensor(bme680_t *sensor)
{
    esp_err_t err;

    do {
        err = init_sensor(sensor);
        if (err != ESP_OK) {
            ESP_LOGE(TAG, "BME680 init failed on SDA=%d SCL=%d addr=0x%02x: %s. Retrying in %d ms", BME680_SDA_GPIO, BME680_SCL_GPIO, BME680_ADDR, esp_err_to_name(err), BME680_INIT_RETRY_MS);
            vTaskDelay(pdMS_TO_TICKS(BME680_INIT_RETRY_MS));
        }
    } while (err != ESP_OK);
}

static esp_err_t publish_reading(const bme680_values_float_t *values, uint32_t uptime_s)
{
    char topic[128];
    snprintf(topic, sizeof(topic), "%s/%s/%s/telemetry", CONFIG_BME680_TOPIC_PREFIX, CONFIG_BME680_ROOM, CONFIG_BME680_DEVICE_ID);

    cJSON *root = cJSON_CreateObject();
    cJSON_AddStringToObject(root, "device_id", CONFIG_BME680_DEVICE_ID);
    cJSON_AddStringToObject(root, "room", CONFIG_BME680_ROOM);
    cJSON_AddNumberToObject(root, "temperature", values->temperature);
    cJSON_AddNumberToObject(root, "humidity", values->humidity);
    cJSON_AddNumberToObject(root, "pressure", values->pressure);
    cJSON_AddNumberToObject(root, "gas_resistance", values->gas_resistance);
    cJSON_AddNumberToObject(root, "uptime_s", uptime_s);

    char *payload = cJSON_PrintUnformatted(root);
    esp_err_t err = wifi_mqtt_publish(topic, payload);

    cJSON_free(payload);
    cJSON_Delete(root);
    return err;
}

void app_main(void)
{
    bme680_t sensor;
    uint32_t uptime_s = 0;

    ESP_ERROR_CHECK(wifi_mqtt_start());
    init_i2c_subsystem();
    wait_for_sensor(&sensor);

    while (1) {
        bme680_values_float_t values;
        esp_err_t err = bme680_measure_float(&sensor, &values);

        if (err == ESP_OK) {
            ESP_LOGI(TAG, "Temp: %.2f C  Humidity: %.2f %%  Pressure: %.2f hPa  Gas: %.0f ohm", values.temperature, values.humidity, values.pressure, values.gas_resistance);

            err = publish_reading(&values, uptime_s);
            if (err != ESP_OK) {
                ESP_LOGW(TAG, "Telemetry publish skipped or failed: %s", esp_err_to_name(err));
            }
        } else {
            ESP_LOGE(TAG, "BME680 read failed: %s", esp_err_to_name(err));
        }

        uptime_s += BME680_READ_INTERVAL_MS / 1000;
        vTaskDelay(pdMS_TO_TICKS(BME680_READ_INTERVAL_MS));
    }
}
  • Step 2: Open menuconfig and set the real Wi-Fi and MQTT values

Run:

source "/home/tonio/esp/esp-idf/export.sh" && idf.py -C "/home/tonio/Workspace/perso/esp/bme680-demo" menuconfig

Set:

  • BME680 Telemetry Configuration -> Wi-Fi SSID

  • BME680 Telemetry Configuration -> Wi-Fi password

  • BME680 Telemetry Configuration -> MQTT broker URI

  • BME680 Telemetry Configuration -> MQTT username if needed

  • BME680 Telemetry Configuration -> MQTT password if needed

  • BME680 Telemetry Configuration -> Room

  • BME680 Telemetry Configuration -> Device ID

  • Step 3: Build the integrated firmware

Run:

source "/home/tonio/esp/esp-idf/export.sh" && idf.py -C "/home/tonio/Workspace/perso/esp/bme680-demo" build

Expected:

  • build passes with BME680, Wi-Fi, MQTT, and cJSON linked correctly

Task 5: Verify End-To-End Telemetry Flow

Files:

  • No file changes

  • Step 1: Subscribe to the live telemetry topic from the homelab side

Run:

mosquitto_sub -h 192.168.1.10 -t 'sensors/+/+/telemetry' -v

Expected:

  • terminal waits for live telemetry messages

  • Step 2: Flash the firmware and observe the first MQTT message

Run:

source "/home/tonio/esp/esp-idf/export.sh" && idf.py -C "/home/tonio/Workspace/perso/esp/bme680-demo" -p /dev/ttyUSB0 flash monitor

Expected on the ESP32 side:

  • Wi-Fi connect log
  • MQTT connected log
  • repeated BME680 readings
  • no crash if the broker is temporarily unavailable

Expected on the mosquitto_sub side:

sensors/office/bme680-1/telemetry {"device_id":"bme680-1","room":"office","temperature":24.8,"humidity":46.2,"pressure":1007.3,"gas_resistance":125430,"uptime_s":3812}
  • Step 3: Query InfluxDB for the live device data

Run:

set -a && source "/home/tonio/Workspace/perso/esp/homelab/telemetry/.env" && set +a && docker exec bme680-influxdb influx query --org "$INFLUXDB_ORG" --token "$INFLUXDB_ADMIN_TOKEN" 'from(bucket: "'"$INFLUXDB_BUCKET"'" ) |> range(start: -10m) |> filter(fn: (r) => r._measurement == "environment") |> filter(fn: (r) => r.device_id == "bme680-1") |> limit(n: 20)'

Expected:

  • recent rows exist for _measurement=environment

  • tags include room and device_id

  • field rows exist for temperature, humidity, pressure, gas_resistance, and uptime_s

  • Step 4: If no rows arrive, isolate the failing layer before editing code

Run these checks in order:

docker compose --env-file "/home/tonio/Workspace/perso/esp/homelab/telemetry/.env" -f "/home/tonio/Workspace/perso/esp/homelab/telemetry/compose.yaml" logs telegraf --tail 100

docker compose --env-file "/home/tonio/Workspace/perso/esp/homelab/telemetry/.env" -f "/home/tonio/Workspace/perso/esp/homelab/telemetry/compose.yaml" logs influxdb --tail 100

Interpretation:

  • message appears in mosquitto_sub but not InfluxDB: Telegraf config or credentials issue
  • message never appears in mosquitto_sub: ESP32 publish or broker reachability issue
  • Telegraf shows auth failures: fix MQTT or Influx credentials, not device code

Self-Review

  • Spec coverage: this plan adds MQTT publishing to the ESP32, validates Mosquitto -> Telegraf -> InfluxDB, uses the approved topic shape and payload fields, and keeps Postgres out of the telemetry path.
  • Placeholder scan: all file paths, commands, config content, and verification steps are explicit.
  • Type consistency: wifi_mqtt_start(), wifi_mqtt_is_ready(), and wifi_mqtt_publish() are defined once and referenced consistently; the payload keys match the approved spec exactly.