Checking Test Results

Result Output Format

Test results are generated in HTML and open in a new window.

All results are presented by a web server that is accessible from any browser.

Device Page

To view a list of all devices under test, enter the following URL:

A list of devices appears on the page.

Note: For security reasons, some information is masked in the following example:

Figure 6: Device information page

Test Case Summary Page

To view all test cases (sequences) and whether they are scheduled, ongoing or finished, enter the following URL:

A list of scheduled, ongoing and finished tests appears.

For all finished tests, the result also appears, as shown in the first three data rows below.

Figure 7: Test Case page

Test Information Page

In your browser’s address bar, enter the following URL, where X is the unique test case identifier from the Test Case List page, and is the information you want to review:

If you click the link in the list of test cases to open this page, items defaults to verification,which shows the information of different checkpoints and whether the DUT passed those checkpoints. For selected tests, some detailed information also appears with the “verification” item.

Including RSSI, the Test Bench supports a total of 23 available pages for visualizing data (marked from 1 to 23 in the list below).

1. rssi: RSSI data reported by the gateway.
2. lsnr: LoRa signal-to-noise ratio (SNR) information and statistics reported by the gateway.
3. Freq: Carrier frequency of the uplink packets.
4. Size: Physical layer payload size for uplink packets (in bytes).
5. datr: LoRa data rates used for uplink packets.
6. toa: LoRa packet time on-air information and duty-cycle information.
7. MType: Message type information.
8. FCnt: Frame count for uplink confirmed or unconfirmed data frames.
9. ACK: Acknowledgment information for both uplink and downlink data frames.
10. mac: The MAC command trace on uplink or downlink packets.
11. FPort: Application FPort information in uplink data packets.
12. FRMPayload: Application layer frame payload on uplink data frames.
13. FRMPayload_length: Application payload frame length for uplink frames (in bytes).
14. time_diff: Time difference on adjacent uplink packets.
15. FCnt_diff: Frame count difference on adjacent uplink data frames.
16. log: Detailed logging information for all packets related to the device.
17. rssi-freq: The correlation of frequency and RSSI. This is used to investigate the RSSI distribution on different frequency channels, so the quality of various channels can be ascertained. 

 Figure 8: Example of rssi-freq correlation page

In the example shown in Figure 8, the upper end of the vertical blue lines represents the 90th percentile of the RSSI grouped by the x-axis value (channel frequency). The lower end represents the 10th percentile. The black dot represents the average value.

18. lsnr-freq: The correlation of frequency and SNR. It is used to investigate the SNR distribution on different frequency channels, so the quality of various channels can be ascertained.
19. rssi-toa: The correlation of time on air (TOA) and RSSI. This is used to investigate the RSSI distribution of the packets of different on-air times. It helps to qualify the DUT hardware design.
20. lsnr-toa: The correlation between TOA and SNR. This is used to investigate the SNR distribution of the packets of different on-air time. It helps to qualify the DUT hardware design.
21. current: Current consumption during the test.

   Figure 9: Example of current consumption page

As shown in Figure 9, the current profile, distribution and CDF measurements are plotted at one-second intervals.

All the packets logged appear in the table. To view a detailed, high-resolution, current profile, click the link under Time in the grid at the bottom left of the screen. The default span for the inspection is from -5 to +10 seconds of the uplink packet.

Figure 10 provides an example of a packet current profile:

   Figure10: Example of packet current profile

22. delay: This is the network delay information. Three delays are measured:
    • Round-trip transmission time between the proxy and the test controller
    • Round-trip transmission time between the packet forwarder and the proxy
    • Round-trip transmission time between the network server and the proxy

The first graph on the Delay information page illustrates the latency over time. The second graph shows the distribution of the delays over the duration of the test.

Figure 11: The "Delay" information page

23. print: Opens the print console. Use this command if you want to print the test results page or save it as PDF. To print the test results, enter print at the end of the list of items items. To save it as a PDF, select a PDF printer in the print console. 

Note: This page supports “zooming” and “panning.”

You can cascade multiple supported items on the same page using the plus sign (+). For example, to find information about the first test on frequency, RSSI and current consumption, navigate to:

Figure 12 shows an example of the detailed test results page.

Figure12: Detailed test page (default)

Note: If items is not verification, replace the items items of one or more options, connected by the plus sign (+), to view one or multiple test data sets.

List of items Items

Note: Each items item can have one or more parameters. Multiple parameters are connected by a plus (+) sign. On the Test Information page, you can summarize and visualize different information about a items item depending on its parameters. The content of each items item determines the information that is displayed on this page. See Figure 13 for an example of the type of received signal strength indicator (RSSI) data as reported by the gateway.

Figure 13: Example of RSSI page

The value over time, histogram, and cumulative distribution function (CDF) curve are all plotted on this page. Zoom and pan features are supported.

General Packet Page

Even without predefined test cases, the Test Bench will act as a sniffer to find and save the following information:

• DevEUI: Device EUI of the device under test (DUT)
• startTime: Starting timestamp of the inspection period, as a Unix Time Stamp (float number)
• duration: Duration of the inspection period, in seconds
• items: Aspects to be presented;, see the List of items Items section

To retrieve this information, enter:

General Current Page

You can check the current consumption of the DUT without a test case definition.

While fetching the current profile, the server pulls a large amount of information from memory into the database. The time to process this data can take up to one minute.

Two sampling rates are supported:

• High sampling rate profile
• Long duration profile (Low sampling rate)

To check the profile for less than one minute use the high sampling rate mode. To check the profile for longer than one minute use the long duration mode.

High Sampling Rate Profile

For a high-resolution profile (at about 670 SPS), the Test Bench can retrieve the current profile during the transmission and reception periods. Enter the following in your browser’s address bar:

• startIme: Starting timestamp of the inspection period, as a Unix Time Stamp (float number). You can use external resources^[1] to convert the date to a time stamp.
• duration: Duration of the inspection period, in seconds

Long Duration Profile

To observe power consumption over a long span of time (1SPS), use a low-resolution profile. In the address bar of your browser enter the following URL:

• startTime: Starting timestamp of the inspection period, in Unix Time Stamp (float number)
• duration: Duration of the inspection period, in seconds

If the duration is more than 86,400 seconds (one day), the resolution is automatically decreased to one sample per minute (1/60 SPS).

Similarly, you can retrieve the latest current consumption data from either of the following URLs:

or

• time: the current profile duration to be retrieved, in seconds