How to reduce the Data Rate/Load e.g. for Remote RF Monitoring?

#1 · 10/03/2021, 13:27

Quote from AdminTC on 10/03/2021, 13:27
Lets say you want to have a long time live RF monitoring and have a bad remote connection (e.g. only a 3G connection) or simply want to reduce the data rate. How can you reduce the data rate without loosing to much time and frequency domain information?

We have set up the following experimental demo mission where you can adjust different data reduction parameters and see the results at a glance:

A SPECTRAN V6 is streaming raw I/Q data (@ 80MHz RTBW). Those data are converted to SPECTRA with an IQ Power Spectrum block which streams to a Waterfall block. The output of the Waterfall block will be send through a HTTP Server and HTTP Client connection (to measure the data rate) and then displayd on a second Waterfall block to compare the data with the original stream:

The first data reduction is done by converting the I/Q to SPECTRA data with the IQ Power Spectrum block. This will reduce the data load from around 240MBytes/s to "only" 95MBytes/s so about factor 2,5:

The next data reduction is more tricky, we use the Waterfall block to reduce the data load: Since it offers an output for the Time Compression we can highly reduce the data load without missing a signal since we are using the Max function for the Time Compression. Setting the Time Compression to 100 already reduces the data load by factor 100 to only 950kBytes/s. Even with this highly reduced data load we still get a perfect data view for the spectrum:

Even turning the Time Compression to maximum (100k) offers a perfect spectrum view and now reduces the data load again by factor 100 to a supere low 1kBytes/s:

But now you get a heavy resolution loss in the time domain. Yes you will not miss any signal but it would be nice get some more resolution within the time domain. So how could achieve this without bringing the date load up again? The only way is to reduce the FFT. So we will reduce the frequency domain resolution to get a higher time domain resolution. E.g. if we change the FFT from 1024 to 64 our time domain resolution will highly increase but the data rate is still at a very low rate of around 2kBytes/s:

At the end it is simply up to your needs how to mix those settings but as you can see reducing the data load us no magic if you know how to do and if you can live with the results.

If you wish to get an even higher data reduction you could in addition start to skip windows from the FFT e.g. skipping 10 of 11 windows will reduce the data rate by factor 10 but in this case you will lose data since there is no Max function available. But now you get an ultra low 150Bytes/s:

Want an even lower rate? You can skip even more windows or why not add a second Waterfall block? This will reduce the data rate by up to another 100k...

Lets say you want to have a long time live RF monitoring and have a bad remote connection (e.g. only a 3G connection) or simply want to reduce the data rate. How can you reduce the data rate without loosing to much time and frequency domain information?

We have set up the following experimental demo mission where you can adjust different data reduction parameters and see the results at a glance:

A SPECTRAN V6 is streaming raw I/Q data (@ 80MHz RTBW). Those data are converted to SPECTRA with an IQ Power Spectrum block which streams to a Waterfall block. The output of the Waterfall block will be send through a HTTP Server and HTTP Client connection (to measure the data rate) and then displayd on a second Waterfall block to compare the data with the original stream:

Data Reduction Mission

The first data reduction is done by converting the I/Q to SPECTRA data with the IQ Power Spectrum block. This will reduce the data load from around 240MBytes/s to "only" 95MBytes/s so about factor 2,5:

Reducing the data load by coverting I/Q data to SPECTRA

The next data reduction is more tricky, we use the Waterfall block to reduce the data load: Since it offers an output for the Time Compression we can highly reduce the data load without missing a signal since we are using the Max function for the Time Compression. Setting the Time Compression to 100 already reduces the data load by factor 100 to only 950kBytes/s. Even with this highly reduced data load we still get a perfect data view for the spectrum:

Reducing the Spectrum Data Load by factor 100!

Even turning the Time Compression to maximum (100k) offers a perfect spectrum view and now reduces the data load again by factor 100 to a supere low 1kBytes/s:

Ultra Low Data Rate for Spectrum Monitoring

But now you get a heavy resolution loss in the time domain. Yes you will not miss any signal but it would be nice get some more resolution within the time domain. So how could achieve this without bringing the date load up again? The only way is to reduce the FFT. So we will reduce the frequency domain resolution to get a higher time domain resolution. E.g. if we change the FFT from 1024 to 64 our time domain resolution will highly increase but the data rate is still at a very low rate of around 2kBytes/s:

High Time Domain Resolution with Ultra Low Data Load

At the end it is simply up to your needs how to mix those settings but as you can see reducing the data load us no magic if you know how to do and if you can live with the results.

If you wish to get an even higher data reduction you could in addition start to skip windows from the FFT e.g. skipping 10 of 11 windows will reduce the data rate by factor 10 but in this case you will lose data since there is no Max function available. But now you get an ultra low 150Bytes/s:

Lowest Possible Data Rate for RF Monitoring

Want an even lower rate? You can skip even more windows or why not add a second Waterfall block? This will reduce the data rate by up to another 100k...

Sofon has reacted to this post.