SIE file size issues

Currently, there are no user options that affect the size of the SIE file. The data types used for channels are set to minimize CPU load, which means that 64 bit floats are used for all channels where any numerical processing is required. The only exception to this are the MX channels that source data as 32 bit floats, with no numerical processing required. This includes the MX840 CAN channels. However, the MX471 CAN channels are currently stored as 64 bit floats.

NOTE
Providing user options for storing channels in other data types to consume less SIE storage area is planned for the future, but is not scheduled at this time.

Users commonly want to know how long a test can run before it fills up the SIE storage area or reaches a maximum desired SIE file size.

When the system is used in a simple “channel data logger” mode where all channels are stored using the Collect time history storage option (or one or more Time History DataModes configured to use the Always on triggering option), this can be calculated with reasonable accuracy using the following steps.

  1. There is some overhead in the SIE file for storing the metadata (e.g., SXR setup files, system and hardware information and SIE tags for channel parameters). As a rule of thumb, this overhead can be estimated as about three times the SXR file size. While this is insignificant for most testing scenarios, it can be a significant factor for short term tests and/or tests at very low sample rates like 1 S/s or less.
  2. The remainder of the SIE file will contain the channel data. For each channel, the storage rate can be computed using the following formula:

bytes_per_second = bytes_per_sample * sample_rate * overhead_factor

  1. Refer to the first section of this topic to set data_type_bytes. It will be either 4 (for 32 bit float) or 8 (for 64 bit float).
  2. The sample_rate is available in the user interface.
  3. The overhead_factor is computed as follows. There are 40 bytes of overhead used for each SIE block of data stored. The amount of data (in bytes) in any SIE block is determined by the following algorithm (provided using C based pseudo code).

if (sample_rate >= 50) block_time = 5;

else if (sample_rate >= 10) block_time = 10;

else if (sample_rate >= 2) block_time =20;

else if (sample_rate > 0.4) block_time = 40;

else block_time = 80;

bytes_per_block = sample_rate * block_time * bytes_per_sample;

if (bytes_per_block > 8000)

bytes_per_block = 8000;

else if (bytes_per_block < bytes_per_sample)

bytes_per_block = bytes_per_sample;

overhead_factor = (bytes_per_block + 40) / bytes_per_block;

NOTE
The “block_time” parameter in the above algorithm is the time between block write to the SIE file. For example, at 50 S/s, blocks are written every 5 seconds, and at 1 S/s, blocks are written every 20 seconds.

Of course, it is common for the system to be used in a more complicated mode than a simple “channel data logger”.

  1. The storage rate for CAN and GPS raw message channels is not deterministic. However, the user can usually get good estimates by running some tests that store only these channels.
  2. For histogram DataModes, the storage area is computed using the following formulas.
  1. For one dimensional histograms: bytes = (bins + 2) * 8
  2. For two dimensional histograms: bytes = (bins + 2) * (bins + 2) * 8

NOTE
The histograms are stored in DRAM memory while the test is running. They are written to the SIE file when the test is stopped. SIE storage space is reserved for this to ensure they can be stored, even if the SIE storage area is fully consumed during the test.

  1. When sequential DataModes such as the Burst History, Peak Valley and Time History with gated triggering are used, the storage rate is not deterministic and in general not feasible to estimate.