Skip to content

Stream Feature View with Aggregations in Continuous Mode

For applications that require the most up-to-date feature data, "continuous mode" for Stream Feature Views can update feature values in less than a second after the event is available in the stream data source. With continuous mode, Tecton will process each event as it arrives, rather than waiting for the slide period to complete.

Please see the Stream Feature View Overview documentation for details on how a Stream Feature View Overview Feature View works.

Feature Definition Example

  • Set aggregation_mode=AggregationMode.CONTINUOUS to enable continuous event processing.
  • Optionally set instance_availability="on_demand" within the stream_cluster_config. Spot Instances may lead to feature processing delays due to spot termination or looking for an available instance, so On-demand Instances will deliver more consistent performance.

This example Feature View shows how to configure the decorator parameters.

Stream Data Source Configuration If your stream data source is Kinesis, we suggest lowering the default buffering to avoid delays in event processing.

Here are some suggested parameter values for a KinesisConfig:

  • For Databricks users:

    • maxFetchDuration="200ms"
    • maxFetchRate="2"
    • minFetchPeriod="200ms"

  • For EMR users:

    • kinesis.executor.idleTimeBetweenReadsInMs ="200"
    • kinesis.executor.maxFetchTimeInMs = "200"

This example data source shows how to configure the stream options on a KinesisConfig with Databricks.

Usage Example

See how to use a Stream Feature View with Continuous Aggregations in a notebook here.