On-Demand Feature Views that Return Struct Features

Note

This feature is available in Tecton on Databricks and Tecton on EMR.

You can include a Struct data type in the output schema of an On-Demand Feature View (ODFV). A Struct can contain multiple fields with mixed data types.

A Struct can be nested within other complex types. For example, you can have a Struct within a Struct, or an array of Structs.

Using a Struct in the output schema of an ODFV allows you to easily parse the ODFV's output when it contains multiple feature values.

Example usage: An output Struct containing two fields

The ODFV definition

from tecton import on_demand_feature_view, RequestSource
from tecton.types import Array, Field, Float64, String, Struct

request_source = RequestSource([Field("input_float", Float64)])

output_schema = [
    Field("output_struct", Struct([
        Field("string_field", String),
        Field("float64_field", Float64)
    ]))
]

@on_demand_feature_view(
    mode="python",
    sources=[request_source],
    schema=output_schema,
    description="Output a struct with two fields."
)
def simple_struct_example_odfv(request):
    input_float = request["input_float"]
    return {
        "output_struct": {
            "string_field": str(input_float * 2),
            "float64_field": input_float * 2
        }
    }

Example usage in a notebook

import tecton
import pandas

spine_df = pandas.DataFrame(data={"input_float": [1.23, 3.22]})

simple_struct_example_odfv = tecton.get_workspace('my_workspace').get_feature_view('simple_struct_example_odfv')
simple_struct_example_odfv.get_historical_features(spine_df).to_spark().show(10, False)

Output:

+-----------+-----------------------------------------+
|input_float|simple_struct_example_odfv__output_struct|
+-----------+-----------------------------------------+
|1.23       |{2.46, 2.46}                             |
|3.22       |{6.44, 6.44}                             |
+-----------+-----------------------------------------+

Example HTTP request

$ curl -X POST http://<your_cluster>.tecton.ai/api/v1/feature-service/get-features\
     -H "Authorization: Tecton-key $TECTON_API_KEY" -d\
'{
  "params": {
    "workspace_name": "my_workspace",
    "feature_view_name": "simple_struct_example_odfv",
    "request_context_map": {
      "input_float": 1.23
    },
    "metadata_options": {
      "include_names": true,
      "include_data_types": true
    }
  }
}'

Output:

{
  "result": {
    "features": [
      [
        "2.46",
        2.46
      ]
    ]
  },
  "metadata": {
    "features": [
      {
        "name": "output_struct",
        "dataType": {
          "type": "struct",
          "fields": [
            {
              "name": "string_field",
              "dataType": {
                "type": "string"
              }
            },
            {
              "name": "float64_field",
              "dataType": {
                "type": "float64"
              }
            }
          ]
        }
      }
    ]
  }
}

Example usage: An output Struct containing an array of Structs with some nulls

The ODFV definition

from tecton import on_demand_feature_view, RequestSource
from tecton.types import Array, Field, Float64, String, Struct
request_source = RequestSource([Field("input_float", Float64)])

output_schema = [
    Field("array_of_structs", Array(
        Struct([
            Field("string_field", String),
            Field("float64_field", Float64)
        ])
    )),
]

@on_demand_feature_view(
    mode="python",
    sources=[request_source],
    schema=output_schema,
    description="Output an array of Structs with some null examples."
)
def array_of_structs_example_odfv(request):
    input_float = request["input_float"]
    return {
        "array_of_structs": [
            {
                "string_field": str(input_float * 2),
                "float64_field": input_float * 2
            },
            {
                "string_field": str(input_float * 3),
                "float64_field": input_float * 3
            },
            # A Struct missing one key and setting the other explicitly to None. These are equivalent
            # was to return a "null" field.
            {
                "string_field": None,
                # "float64_field": ...
            },
            # All Tecton data types are nullable, including Structs.
            None
        ]
    }

Example usage in a notebook

array_of_structs_example_odfv = tecton.get_workspace('my_workspace').get_feature_view('array_of_structs_example_odfv')
array_of_structs_example_odfv.get_historical_features(spine_df).to_spark().show(10, False)

Output:

+-----------+------------------------------------------------+
|input_float|array_of_structs_example_odfv__array_of_structs |
+-----------+------------------------------------------------+
|1.23       |[{2.46, 2.46}, {3.69, 3.69}, {null, null}, null]|
|3.22       |[{6.44, 6.44}, {9.66, 9.66}, {null, null}, null]|
+-----------+------------------------------------------------+

Example HTTP request

$ curl -X POST http://<your_cluster>.tecton.ai/api/v1/feature-service/get-features\
     -H "Authorization: Tecton-key $TECTON_API_KEY" -d\
'{
  "params": {
    "workspace_name": "my_workspace",
    "feature_view_name": "array_of_structs_example_odfv",
    "request_context_map": {
      "input_float": 1.23
    },
    "metadata_options": {
      "include_names": true,
      "include_data_types": true
    }
  }
}'

Output:

{
  "result": {
    "features": [
      [
        [
          "2.46",
          2.46
        ],
        [
          "3.69",
          3.69
        ],
        [
          null,
          null
        ],
        null
      ]
    ]
  },
  "metadata": {
    "features": [
      {
        "name": "array_of_structs",
        "dataType": {
          "type": "array",
          "elementType": {
            "type": "struct",
            "fields": [
              {
                "name": "string_field",
                "dataType": {
                  "type": "string"
                }
              },
              {
                "name": "float64_field",
                "dataType": {
                  "type": "float64"
                }
              }
            ]
          }
        }
      }
    ]
  }
}

Note that null or missing fields are returned in the JSON response as JSON null, and that there is a difference between a Struct containing all null values and a null Struct. Both are shown in this example.