Forum
Please or Register to create posts and topics.

File Writer

File Writer block:

Stream/Write any RTSA data as there is .rtsa", ".tag" and ".dat" (I/Q, Spectra, Video, Tracking etc.) to disk:

File Writer Block | Export any RTSA Datafile to Disk

Left hand side input:

  • Stream

Right hand side output:

  • Monitor

 

The File Writer block records any incoming data stream to disk. It offers a Monitor output to check the data in real-time, which can also be used to connect additional control blocks like the Control Sequencer.

When writing raw IQ data make sure you have a well performing SSD as the data rate can exceed several hundred megabytes per second (depending on settings)! You might consider to use the IQ Demodulator to reduce the data size.

You can set a lot of useful parameter so it will perfectly fit your needs:

  • Single or Multi Stream support
  • Autostart feature
  • Automatically include index and/or timestamp in filenames
  • Auto Split file (reduce single file size)
  • Auto rotate (prevent a folder to overflow by overwriting the old data first | FIFO feature incl. adjustable file rotations)
  • Prevent overwriting old files
  • Adjustable data compression
  • Adjustable folder memory limitation
  • Adjustable record delay

 

File Writer Setup

 

BTW: We already have a nice posting helping you to setup up a Circular Buffer to prevent a full harddisk. It will limit the harddisk space by automatically overwriting the old data.

While writing the data you get some additional information e.g. a data preview via a small histogram screenshot, a compact waterfall display and some statistics as for instance the Write Rate (MBytes/s), the Buffer Fill and the Compression Ratio as graphs:

File Write streaming IQ data to disk

 

Some typical missions:

File Write typical missions

I am trying to create a series of missions that each automatically start, write a single file (based on a time duration or file size limit) and then automatically move on to the next mission.  I'm sure there must be a way to do this but have not been able to figure out the correct File Writer and Control Sequencer settings in order to accomplish this.  I have the missions chained together via control sequencer, but the operator has to actually stop the file writing or pause the Spectran to make it move on to the next mission.

For file writer, regardless of chaining missions with control sequencer, I cannot find a combination of settings that have it just write a single file of a fixed time/size and then stop.

Also, I played a little with the folder size limit settings and it did not seem to work correctly.

Starting and Stopping the FileWriter based on time just needs a Start, Delay and Stop command in the Command Sequencer:

And the use the "Start" button in the Control Sequencer to start the recording. You may need to add a few seconds extra for the device connection to be established though (delay is clock time, not measurement time). Repeat that sequence (or use a "Load Mission" command) afterwards for any additional recording (and maybe add a delay in between for safety). FileWriter doesn't need any special settings, except maybe disabling "Check Overwrite" to prevent confirmation dialogs.

Quote from nothingman1020 on 03/05/2022, 20:59

Also, I played a little with the folder size limit settings and it did not seem to work correctly.

The size check is only applied to the file set currently being recorded (when using the rotation feature and indices or timestamps), other files in the folder are not accounted for (for performance reasons).

The File Writer now offers a Discontinuities/Second chart & free disk space information.

This will help a lot to keep control over the recording:

File Writer Block

premise:IQ Rate is 92MHz;

1.When I set the compression=0,the write rate was 737MB/s,and the compression ratio is 1.0:1.The IQ data collected at this time was complete.

2.When I set the compression=1(default setting),the write rate was 250MB/s,and the compression ratio is 2.8:1.Was the IQ data collected at this time  not complete?

3.what is the meanning/use of compression and how should I  set it up correctly?

Uploaded files:
  • 微信图片_20230522145357.png

Compression does the same as in any other context: (repeated) long sequences of data are replaced by shorter versions and a lookup table, on decompression the short version is expanded again to the original long sequence. This can save significant amount of bandwidth at the expense of slightly higher CPU load and memory consumption.

It is usually safe to just leave the default value of 1: Disabling compression yields no real benefit as the way higher IO speeds will also need more CPU power as well, and the bandwidth requirements can easily exceed your systems IO capacity (even high end desktop SSDs will usually struggle to maintain transfer speeds above 1 GB/s over longer periods of time, and transfer speed requirements scale basically linear with RTBW bandwidth and/or number of streams). On the other hand higher compression settings will quickly run into diminishing returns, but that also depends quite a bit on the nature of the recorded data. Still might be useful if you're really short on disk space or IO bandwidth.

Quote from mm_dev on 22/05/2023, 20:32

Compression does the same as in any other context: (repeated) long sequences of data are replaced by shorter versions and a lookup table, on decompression the short version is expanded again to the original long sequence.

I had recorded the IQ data using File Writer ever before. but I don't pay attention that the compression level default value is "1" not "0". So how can I decompress the short version to expand again to the original long sequence on RTSA?

You don't have to do anything, File Reader / File Source blocks do that automatically.

How do I change the IQ Rate when recording Raw IQ to a file? I'm using a Spectran V6 that defaults to 92MHz, but I only need 10Mhz or so for this application.

Is .dat the "raw" format? I don't seen an option in File Writer to choose Raw IQ despite seeing that description in many blocks on the forum.

-James

Simply change the sample rate within the V6 block by changing the Span.

jamesm has reacted to this post.
jamesm