FIX Trading Community

 

 Remember me

Register  |   Lost password?

FAST Protocol

Discussions > FAST Protocol > Calculated fields support

Calculated fields support

Complete message thread from old site

Jim Northey
2220 days ago,(2008/03/19)

On the call the need to represent calculated fields in meta data was discussed by Hanno and Greg. The feeling was, in a tunneling scenario where a FIX tag=value message is being reconstructed, there should be a standard way of indicating fields, such as checksum, message length need to be recalculated.

Hanno added that sometimes one FIX field might be segments, such as CFICode into constituent parts. Would it be useful to have meta data on how to reconstruct the data from its constituent FAST fields?

Response from Hanno:

Another example for the concatenation is again the UTCTimestamp datatype in FIX. If it can be split into date+time on the wire, then date can be coded very efficiently as a constant if it does not change during the session, requiring only time to be sent on the wire. The receiving end can put the two fields back together to create a valid FIX datatype again.

Jim Northey
2220 days ago,(2008/03/19)

Further Response from Hanno:
I agree, you could even combine the two to have a smaller integer value for msec since midnight instead of epoch. We do not need to define a single method, rather a toolbox of options to be chosen from. Greg made the case for ISE where members want to pick up the binary version whereas CME has stuck to the string representation and omitted the punctuation marks. The latter will benefit more from a concatenation approach than the former but there is no need to rule one or the other out. As you said, this is up to testing to determine the most efficient way.

The language to describe such conversions is what is needed for any of this. It is only needed in the context of standard messages where datatypes, field orders and field content are typically defined from a non-technical perspective. I believe this is a good thing. However, it does not preclude optimization on a technical level which is what FAST is intended for.

> On the call the need to represent calculated fields in meta data was
> discussed by Hanno and Greg. The feeling was, in a tunneling scenario
> where a FIX tag=value message is being reconstructed, there should be a
> standard way of indicating fields, such as checksum, message length need
> to be recalculated.
>
> Hanno added that sometimes one FIX field might be segments, such as
> CFICode into constituent parts. Would it be useful to have meta data on
> how to reconstruct the data from its constituent FAST fields?
>
> Response from Hanno:
>
> Another example for the concatenation is again the UTCTimestamp
> datatype in FIX. If it can be split into date+time on the wire, then
> date can be coded very efficiently as a constant if it does not change
> during the session, requiring only time to be sent on the wire. The
> receiving end can put the two fields back together to create a valid
> FIX datatype again.

David Rosenborg
2214 days ago,(2008/03/25)

No formal notation would be necessary to express that for example checksum and message length should be recalculated in a tunneling scenario. The FIX-over-FAST document should mention that such fields are not to be present in the FAST version of the message since they are artifacts of the tag=value encoding.

In earlier drafts of FAST there was the notion of a computed field, later renamed to implicit field, which was later droped alltogether since the construct was considered purely informational (and with very little practical value)

As discussed earlier, timestamps are best (most compact and efficiently) represented as a number of milli/nano/seconds since an EPOCH, together with the delta operator. What we need to discuss further is how to indicate the resolution and epoch used for a particular field.

Sending timestamps as strings should still be an option, but designing string de-/composition trickery seems suboptimal and defeats the purpose of FAST beeing efficient. If you want a simple solution in an otherwise string based context stick with full tsp strings and a suitable operator. If you want best overall performance, send integers with e.g. milliseconds since EPOCH.

For the timestamp handling I agree we'll need some formal notation. Ideally the next revision of FAST will include a special timestamp datatype for this purpose. In the meantime, some simple heuristcs and documentation in the FIX-over-FAST doucment would cover the most basic and immediate needs.

/David

> On the call the need to represent calculated fields in meta data was
> discussed by Hanno and Greg. The feeling was, in a tunneling scenario
> where a FIX tag=value message is being reconstructed, there should be a
> standard way of indicating fields, such as checksum, message length need
> to be recalculated.
>
> Hanno added that sometimes one FIX field might be segments, such as
> CFICode into constituent parts. Would it be useful to have meta data on
> how to reconstruct the data from its constituent FAST fields?
>
> Response from Hanno:
>
> Another example for the concatenation is again the UTCTimestamp
> datatype in FIX. If it can be split into date+time on the wire, then
> date can be coded very efficiently as a constant if it does not change
> during the session, requiring only time to be sent on the wire. The
> receiving end can put the two fields back together to create a valid
> FIX datatype again.

Hanno Klein
2214 days ago,(2008/03/25)

David,

that sounds simple for computable fields but I believe some more detail needs to be explained. If we assume a FAST decoder and a FIX engine (and nothing else) then the FAST decoder does not know it is decoding a FIX message (and hence does not produce ChkSum) and the FIX engine assumes that it receives a valid FIX message (and hence also does not produce ChkSum). Conceptually, who takes care of the missing bit? Is it a third, separate component or an integral part of one of the other two?

Regards,
Hanno.

> No formal notation would be necessary to express that for example
> checksum and message length should be recalculated in a tunneling
> scenario. The FIX-over-FAST document should mention that such fields are
> not to be present in the FAST version of the message since they are
> artifacts of the tag=value encoding.
>
> In earlier drafts of FAST there was the notion of a computed field,
> later renamed to implicit field, which was later droped alltogether
> since the construct was considered purely informational (and with very
> little practical value)
>
> As discussed earlier, timestamps are best (most compact and efficiently)
> represented as a number of milli/nano/seconds since an EPOCH, together
> with the delta operator. What we need to discuss further is how to
> indicate the resolution and epoch used for a particular field.
>
> Sending timestamps as strings should still be an option, but designing
> string de-/composition trickery seems suboptimal and defeats the purpose
> of FAST beeing efficient. If you want a simple solution in an otherwise
> string based context stick with full tsp strings and a suitable
> operator. If you want best overall performance, send integers with e.g.
> milliseconds since EPOCH.
>
> For the timestamp handling I agree we'll need some formal notation.
> Ideally the next revision of FAST will include a special timestamp
> datatype for this purpose. In the meantime, some simple heuristcs and
> documentation in the FIX-over-FAST doucment would cover the most basic
> and immediate needs.
>
> /David
>
> > On the call the need to represent calculated fields in meta data was
> > discussed by Hanno and Greg. The feeling was, in a tunneling scenario
> > where a FIX tag=value message is being reconstructed, there should be
> > a standard way of indicating fields, such as checksum, message length
> > need to be recalculated.
> >
> > Hanno added that sometimes one FIX field might be segments, such as
> > CFICode into constituent parts. Would it be useful to have meta data
> > on how to reconstruct the data from its constituent FAST fields?
> >
> > Response from Hanno:
> >
> > Another example for the concatenation is again the UTCTimestamp
> > datatype in FIX. If it can be split into date+time on the wire, then
> > date can be coded very efficiently as a constant if it does not change
> > during the session, requiring only time to be sent on the wire. The
> > receiving end can put the two fields back together to create a valid
> > FIX datatype again.

David Rosenborg
2214 days ago,(2008/03/25)

The part that is composing the tag=value message is responsible for supplying the body length and checksum fields. If that component is directly integrated with the FAST decoder or is implemented as a separate component that works off some other intermediate format that the FAST decoder produces, doesn't matter. The tag=value encoder must obviously know it is encoding the tag=value format and should have no problem supplying a checksum. No metadata should be needed to state this fact.

/David

> David,
>
> that sounds simple for computable fields but I believe some more detail
> needs to be explained. If we assume a FAST decoder and a FIX engine (and
> nothing else) then the FAST decoder does not know it is decoding a FIX
> message (and hence does not produce ChkSum) and the FIX engine assumes
> that it receives a valid FIX message (and hence also does not produce
> ChkSum). Conceptually, who takes care of the missing bit? Is it a third,
> separate component or an integral part of one of the other two?
>
> Regards, Hanno.
>
> > No formal notation would be necessary to express that for example
> > checksum and message length should be recalculated in a tunneling
> > scenario. The FIX-over-FAST document should mention that such fields
> > are not to be present in the FAST version of the message since they
> > are artifacts of the tag=value encoding.
> >
> > In earlier drafts of FAST there was the notion of a computed field,
> > later renamed to implicit field, which was later droped alltogether
> > since the construct was considered purely informational (and with very
> > little practical value)
> >
> > As discussed earlier, timestamps are best (most compact and
> > efficiently) represented as a number of milli/nano/seconds since an
> > EPOCH, together with the delta operator. What we need to discuss
> > further is how to indicate the resolution and epoch used for a
> > particular field.
> >
> > Sending timestamps as strings should still be an option, but designing
> > string de-/composition trickery seems suboptimal and defeats the
> > purpose of FAST beeing efficient. If you want a simple solution in an
> > otherwise string based context stick with full tsp strings and a
> > suitable operator. If you want best overall performance, send integers
> > with e.g. milliseconds since EPOCH.
> >
> > For the timestamp handling I agree we'll need some formal notation.
> > Ideally the next revision of FAST will include a special timestamp
> > datatype for this purpose. In the meantime, some simple heuristcs and
> > documentation in the FIX-over-FAST doucment would cover the most basic
> > and immediate needs.
> >
> > /David
> >
> > > On the call the need to represent calculated fields in meta data was
> > > discussed by Hanno and Greg. The feeling was, in a tunneling
> > > scenario where a FIX tag=value message is being reconstructed, there
> > > should be a standard way of indicating fields, such as checksum,
> > > message length need to be recalculated.
> > >
> > > Hanno added that sometimes one FIX field might be segments, such as
> > > CFICode into constituent parts. Would it be useful to have meta data
> > > on how to reconstruct the data from its constituent FAST fields?
> > >
> > > Response from Hanno:
> > >
> > > Another example for the concatenation is again the UTCTimestamp
> > > datatype in FIX. If it can be split into date+time on the wire, then
> > > date can be coded very efficiently as a constant if it does not
> > > change during the session, requiring only time to be sent on the
> > > wire. The receiving end can put the two fields back together to
> > > create a valid FIX datatype again.