Step-by-step guide

A delayed replication is a way of replicating n times a block of data in a BUFR message by expressing the value of n in the data section itself as opposed to repeating the blocks statically in the definition of the structure. This is possible because the structure of the message is defined in section 3 through the sequence of unexpandedDescriptors while the data themselves are encoded in section 4 (data section) according to the structure defined in section 3. If in the structure definition a delayed replication is included then the value of the replication has to be decoded in the data section and is not known until the decoding of the data section is performed (in ecCodes this is happening when the key "unpack" is set to 1).

  1. To encode a new message the user has to set the structure by setting the unexpandedDescriptors to the desired list of descriptors, but the value of the keys: delayedDescriptorReplicationFactor, extendedDelayedDescriptorReplicationFactor, shortDelayedDescriptorReplicationFactor are all set to 1 and cannot be changed after the template message is created by setting the unexpandedDescriptors. The reason for having three different keys is that in BUFR we have three different type of replications with different number of bits, ranging from the short replication factor with 1 bit to the extended replication factor with 16 bits.

  2. There is the option to set the delayed descriptors keys to different values while setting the unexpandedDescriptors. This can be obtained by setting some of the following keys: inputDelayedDescriptorReplicationFactor, inputExtendedDelayedDescriptorReplicationFactor, inputShortDelayedDescriptorReplicationFactor to a list of values corresponding to the desired replication values in the message structure.

    We need to consider two different cases.

  3. Uncompressed data

    For uncompressed data the list of values for the delayed descriptor keys has to have the appropriate number of values for the number of subsets and eventually the number of times it is repeated in the structure. 

    Here follows an example using the bufr_filter, but we always remember that the same example could be implemented in Fortran or Python by setting the same keys with the same values.


    instructions.filter
    set numberOfSubsets=2;
set inputDelayedDescriptorReplicationFactor = {2,3};
set inputExtendedDelayedDescriptorReplicationFactor = {3,4};
set unexpandedDescriptors={309052};

print "/subsetNumber=1/delayedDescriptorReplicationFactor=[/subsetNumber=1/delayedDescriptorReplicationFactor]";
print "/subsetNumber=1/extendedDelayedDescriptorReplicationFactor=[/subsetNumber=1/extendedDelayedDescriptorReplicationFactor]";
print "/subsetNumber=2/delayedDescriptorReplicationFactor=[/subsetNumber=2/delayedDescriptorReplicationFactor]";
print "/subsetNumber=2/extendedDelayedDescriptorReplicationFactor=[/subsetNumber=2/extendedDelayedDescriptorReplicationFactor]";
set pressure={102400,50000,40000,30000,20000,15000,102400,50000,40000,30000,20000,15000};

set pack=1;

write;

    At line 2 and 3 the delayed descriptors factors are set with values that are appropriate for a message with 2 subsets (set at line 1). At line 4 the unexpandedDescriptors are set for an upper air message. After printing the replication factors for the two subsets the pressure, which is present in all the replicated blocks is set with a list of 12 values. The values are 12 because they are 3+2=5 for the first subset and 3+4=7 for the second subset, giving a total of 12 values. The order of the values is subset by subset. First all the values belonging to a subset and then the values belonging to the next subset.

  4. Compressed data

    For compressed data the keys to be set are the same, but the different number of values in the list for the delayed descriptor keys is due only to the possibility of having more than one replication factor in the expanded structure, not due to different values of the replication for different subsets. It is indeed an important rule for BUFR that compressed data cannot have different replication factors for different subsets. The replication factor is constant across the different subsets. This characteristics it is a constraint for the BUFR compressed data. Here follows an example using the bufr_filter.

    instructions.filter
    set localTablesVersionNumber=1;
set masterTablesVersionNumber=13;

set inputDelayedDescriptorReplicationFactor = {5};
set compressedData=1;
set numberOfSubsets=2;
set unexpandedDescriptors={312061};

set #1#windSpeedAt10M={10,20};
set #3#windSpeedAt10M={30,40};


set pack=1;
write;

    In this example we are using data which are available only in an ECMWF local table and therefore we are setting the local table version at the beginning. At line 4 the inputDelayedDescriptorReplicationFactor is set to 5. The key compressedData set to 1 means that the data are going to be in the compressed form and the number of subsets is set to 2 in the line after. At line 7 the unexpandedDescriptors are set and now we are going to have 5 windSpeedAt10M elements due to the replication factor of 5 and each of them is a vector of two elements due to the fact that we have two subsets in the message.

  5. We always need to remember to pack the data before writing.