LoRa Basics Modem and LoRa Edge documentation

Use Case - IR Image Upload on Detected Motion

Introduction

The LoRaWAN® standard operates over a shared frequency space that is subject to regional limitations in packet size, spreading factor, and bandwidth of the selected channel. An end device on a LoRaWAN network must be capable of handling a varying Maximum Transmission Unit or MTU, meaning that the number of bytes that can be sent in one uplink can change over time.

When the device application knows that the packet size is likely to exceed the LoRaWAN packet size limit, the application must implement some way of breaking up larger payloads into multiple packets. The LoRa Edge™ Platform provides a tool for simplifying the upload of large blocks of data by end device applications. This is called the Large File Upload, or LFU for short.

The Event Data Transfer use case described below is suitable for addressing the sending of large blocks of data. For example, using a thermopile image collected by an end device. For the specific information on the actual process of sending the data, see the Large File Upload documentation.

Background

The example use case is an end device sensor that captures an 8x8 thermal image for counting people in a room. This is done by using a thermal image to detect “hot spots”. These hot spots represent people, who stand out against the thermal background in the environment. The end device looks like this:

../../_images/grideye-front.png

The core 8x8 thermal sensor end device is the Panasonic AMG8833 called the “Grid-EYE”. It has an 8x8 thermopile-type infrared sensor that measures temperature estimates over an 8x8 grid and produces a digital output. The readings are 12-bit with a precision of 0.25 degrees Celsius over a usable range from 0-80 degrees Celsius. The data is stored as two-bytes per pixel element, or 128 bytes for an entire thermal image. There is also a 12-bit (0.0625 degrees Celsius resolution) thermistor reading available that can provide a thermal reading for the sensor itself. This thermistor reading can be used to reduce any thermal noise that is due to heating or cooling of the sensor itself.

Note

An Aside on Data Compression:

The application on the end device in this example takes an image of 12-bit samples and finds the minimum temperature reading over all 64 samples. The end device application then subtracts the minimum temperature reading and limits the value to 8-bits/one-byte (value of 0-255) with a scale of 0.0625 degrees Celsius for a range of 16 degrees Celsius above the minimum value. This scheme was implemented to reduce the amount of data transmitted, especially since the range of thermal values is well represented in 16 degrees for indoor environments, where the sensor is typically employed. However, some applications might require the full 12-bit values and those could be sent either in a bit-packed array; every 12 bits in binary blob for 64x12=768 bits or 96 bytes.

Alternatively the 12-bit values could be sent as they are received from the Grid-EYE sensor directly, as 12-bits in 2-bytes (16-bits); for a total of 128 bytes for all 64 values in the grid array. The following analysis is based on the reduction of the original 12-bit temperature values reduced to 8-bits/one-byte using the minimum-subtraction method.

The overall end device works by using the Passive Infrared (PIR) motion sensor to determine whether there is a person in view of the Grid-EYE thermal sensor at a given time. When the PIR sensor detects motion, the capture of a thermal image by the Grid-EYE sensor is triggered. Prior to the trigger event, a quiet period is identified; for example, say the PIR motion sensor is inactive for 15 minutes. During the quiet period an 8x8 thermal image is captured as a background reference. When motion is detected with the PIR motion sensor, the 8x8 thermal imager is used to collect a triggered thermal image. This results in two thermal image measurements; each one is an 8x8 grid of one-byte thermal estimates (as reduced from the original 12-bit values as described in the aside above). Each image is 64 bytes, resulting in 128 bytes for both the motion-triggered (foreground) and background data. After converting these measurements into heatmaps, the data looks like this:

../../_images/grideye-fore-back.png

The background image measures the temperature when no one is present. The foreground in the plot above is the difference between the motion-activated image and the background (both in degrees Celsius). The purpose of the background is to establish the baseline for the difference of the foreground temperature estimate. The foreground is used to estimate the number of people in frame of the 8x8 grid image array. The server-side application performs a cluster analysis to estimate the number of distinct heat blobs captured in the foreground. The detection of a single individual is shown here:

../../_images/grideye-detect.png

Note that the overlay of the detection circle of one individual is on the original foreground thermal image (not the difference). The background data is collected over time. It can be sent on its own or saved when the trigger occurs by the detection of motion. Therefore, at a trigger event, there are 64 background bytes plus 64 foreground bytes, totaling 128 bytes. Additionally a status byte, a battery voltage value, and the minimum thermal value is sent (in a 12-bit number in two bytes) along with the image data.

At that time, the LoRaWAN bandwidth and spreading factor might be such that a data transfer of more than 128 bytes is not be possible in a single uplink and must be be broken into multiple uplink messages due to the MTU limit imposed by the current bandwidth and spreading factor settings. A tool provided by the LoRa Basics™ Modem-E functionality is Large File Upload, which can handle the breakup of large packets into sizes that can be sent over LoRaWAN uplinks. The LFU also has the ability to compensate for some level of packet loss by implementing Forward Error Correction.

Prerequisites / Dependencies

Common:

Connection to the LoRa Cloud™ services is assumed for this use case.

The total upload size of a single LFU transmission is currently 2K bytes at most. If the transmission needs to exceed that limit, the application must first break the large upload into chunks smaller than 2K bytes and send it piecemeal. The benefit of the Forward Error Correction for each of the sub-parts of the transmissions will still be available for the individual parts.

For this example, a background thermal image will be collected when no motion has been sensed by the PIR motion sensor for at least 15 minutes (adjustable based on the changes of ambient temperatures over time).

Requirement for Using Large File Upload (LFU):

  • The LoRa Edge LR1110 device should be claimed.

Prerequisites for LoRa Basics™ Modem:

Not applicable

Prerequisites for LoRa Basics™ Modem-E:

  • Based on LoRa Basics Modem-E version 1.0.7 and later

LoRaWAN® Requirements:

Step-by-Step Procedure

  1. The trigger event occurs (PIR senses motion), causing a foreground thermal image to be captured.

  2. The previously-captured background thermal image is combined in a packet with the foreground image, and a status byte.

  3. The size of this upload packet calculated.

  4. The LFU upload process is initiated.

  5. The packet data is pushed to the LoRa Basics Modem-E in chunks not to exceed the maximum transfer buffer size

  6. When the entire uplink file size is transferred to LoRa Basics Modem-E, the modem will then begin the LFU session

  7. The Large File Upload session will send file fragments to the application server until told to stop, or will send the file fragments twice each (“Timeout” status).

  8. Upon receipt of each file fragment, the application server will send the file fragments to the LoRa Cloud Modem & Geolocation Services

  9. On successful reconstruction of the upload file, the LoRa Cloud Modem & Geolocation Services will return the reconstructed file to the application server, along with a downlink message for the modem to tell it to stop sending fragments.

  10. On a failed reconstruction of the upload file, the LoRa Cloud Modem & Geolocation Services will return a downlink message to the modem, telling it to stop sending fragments.

@startumltitle Simplified Flow for\nLarge File Uploadskinparam linetype orthobox Deviceparticipant "End-Device\nApplication" as UC #99FF99participant "modem" as MODEMend boxbox Serversparticipant "Customer\nApplication Server" as AS #99FF99participant "LoRaCloud\nMGS" as MGSend boxUC -> MODEM : Initialize Upload to ModemLoop Modem-E only: Transfer file to modem    note right of UC: Until all 'size' bytes\nof the file are transferred    UC -> MODEM : Upload all data (in chunks of up to 255 bytes)end== Start of Upload Session ==UC -> MODEM ++: Upload file startLoop Modem sends file fragments    MODEM -> AS : Uplinks with file fragments    AS -> MGS : File fragmentsendMGS -> AS : Reconstructed file,\ndownlink for the modemAS -> MODEM: Downlink for the modemMODEM -> MODEM: Stop sending framentsreturn Notify the End-Device Application== End of Upload Session ==@enduml

Known Issues

None identified.