Applying Changes to a Feature Repository and a Workspace

When you run tecton apply (from the Tecton CLI), the following occurs:

The changes in your local repository, since the last run of tecton apply, are pushed to the corresponding remote repository that is managed by Tecton.
The changes are applied to the current workspace that is selected in your Tecton CLI. (To see the selected workspace, run tecton show).
Changes that have been applied become available on your Tecton instance; for example, feature pipelines run against applied features and write the results to the online and offline stores.

Making a change to a feature repository

As an example, add the file my_entity.py, which contains an Entity object, to a local Feature Repository:

# my_repo/my_entity.py
from tecton import Entity

user = Entity(
    name="user",
    join_keys=["user_id"],
    description="My first entity!"
)

It's important to declare Tecton Objects as global variables in your Python module. When plan or apply commands are run, the Tecton CLI references all Python objects instantiated in the global scope to identify objects in the Tecton Feature Repository.

Previewing changes using `tecton plan`

To preview changes made to your local feature repository, run tecton plan. These are the changes that will be applied if tecton apply is run (the next step).

$ tecton plan
Using workspace "my_workspace" on cluster https://my_app.tecton.ai
✅ Imported 1 Python module from the feature repository
✅ Collecting local feature declarations
✅ Performing server-side validation of feature declarations
 ↓↓↓↓↓↓↓↓↓↓↓↓ Plan Start ↓↓↓↓↓↓↓↓↓↓

  + Create Entity
    name:            user
    description:     My first entity!

 ↑↑↑↑↑↑↑↑↑↑↑↑ Plan End ↑↑↑↑↑↑↑↑↑↑↑↑

Pushing changes using `tecton apply`

Running tecton apply will generate the same output as tecton plan, along with a final prompt to apply the changes.

$ tecton apply
Using workspace "my_workspace" on cluster https://my_app.tecton.ai
✅ Imported 1 Python module from the feature repository
✅ Collecting local feature declarations
✅ Performing server-side validation of feature declarations
 ↓↓↓↓↓↓↓↓↓↓↓↓ Plan Start ↓↓↓↓↓↓↓↓↓↓

  + Create Entity
    name:            user
    description:     My first entity!

 ↑↑↑↑↑↑↑↑↑↑↑↑ Plan End ↑↑↑↑↑↑↑↑↑↑↑↑
Are you sure you want to apply this plan? [y/N]> y
🎉 all done!

If you see the message "all done!", as shown above, then your changes have been pushed to to the Tecton repository.

Note

If your repository contains multiple changes, tecton apply will apply all changes at once.

Running tecton plan before tecton apply is not required.

Types of repository changes

There are 5 types of changes that can be applied to a repository:

+ Create: A new object is being created for the first time.
- Delete: A previously created object is being deleted.
~ Update: A non-destructive update to an existing object (e.g. changing the description of a FeatureView).
~ Recreate: A destructive update that requires an object to be recreated in the remote feature repository. This often occurs when transformations are updated or dependencies change between objects. For example, changing a Data Source definition may require any Feature Views that depend on it to be recreated and re-materialized.
~ Upgrade: No-op updates of objects to meet the latest Tecton API version. These are sometimes observed after upgrading the Tecton SDK using pip3 and should be considered safe.

Examples

Below are examples of create, delete, and update changes:

$ tecton apply
Using workspace "prod"
✅ Imported 3 Python modules from the feature repository
✅ Collecting local feature declarations
✅ Performing server-side validation of feature declarations
↓↓↓↓↓↓↓↓↓↓↓↓ Plan Start ↓↓↓↓↓↓↓↓↓↓
  - Delete Entity
    name:            my_entity
    owner:           alice

  + Create Entity
    name:            my_new_entity
    owner:           alice

  ~ Update FeatureView
    name:            my_feature_view
    owner:           alice
    description:  -> Description of this FeatureView!
↑↑↑↑↑↑↑↑↑↑↑↑ Plan End ↑↑↑↑↑↑↑↑↑↑↑↑
Are you sure you want to apply this plan? [y/N]> y
🎉 all done!

Other actions related to `tecton apply`

Viewing the `apply` history for a Workspace

The tecton log command will display a list of previously applied ids for the remote Feature Repository in your Tecton Workspace.

$ tecton log
Using workspace "prod"
Apply ID:   006ad43e0000000000000107
Author: drake
Date:   2020-05-20 23:19:41.829000

Apply ID:   83a205340000000000000105
Author: rihanna
Date:   2020-05-20 18:00:01.858000

Apply ID:   56e8a66a00000000000000fd
Author: jayz
Date:   2020-05-19 15:13:35.083000

Apply ID:   4bfe16ea00000000000000f4
Author: alicakeys
Date:   2020-05-18 18:45:21.232000

Restoring a previous `apply`

Tecton stores a snapshot of your workspace's local Feature Repository each time tecton apply is run. The tecton restore command makes it possible to overwrite your local Feature Repository with a previous applied version.

To restore the most recently applied version, run tecton restore without an apply ID:

$ tecton restore
Using workspace "prod"
This operation may remove or modify the following files:
/Users/drake/Tecton/my-git-repo/feature_repo/entities.py
/Users/drake/Tecton/my-git-repo/feature_repo/data_sources.py
/Users/drake/Tecton/my-git-repo/feature_repo/feature_views.py
Ok? [y/N]>y

To restore previous version of your local Feature Repository, first run tecton log to determine which apply ID to restore, then run tecton restore <apply ID>.

$ tecton log
Using workspace "prod"
Apply ID:   006ad43e0000000000000107
Author: jayz
Date:   2020-05-20 23:19:41.829000

Apply ID:   **83a205340000000000000105**
Author: rihanna
Date:   2020-05-20 18:00:01.858000

Apply ID:   4bfe16ea00000000000000f4
Author: alicakeys
Date:   2020-05-18 18:45:21.232000

$ tecton restore **83a205340000000000000105**
tecton restore
Using workspace "prod"
This operation may remove or modify the following files:
/Users/drake/Tecton/my-git-repo/feature_repo/entities.py
/Users/drake/Tecton/my-git-repo/feature_repo/data_sources.py
/Users/drake/Tecton/my-git-repo/feature_repo/feature_views.py
Ok? [y/N]>y

Skipping files using `.tectonignore`

Tecton supports a .tectonignore file that can specify files or path expressions to ignore when plan or apply are run. It's similar to Git's .gitignore configuration. .tectonignore should be declared in the feature repo root directory.

For example, in the following repo all objects declared in transactions_batch.py, entities.py, fraud_detection.py, and fraudulent_transactions_count.py would be processed.

├── data_sources
│   └── transactions_batch.py
├── entities.py
├── feature_services
│   └── fraud_detection.py
└── features
    └── fraudulent_transactions_count.py

Suppose everything under feature_services/ need to be ignored temporarily. Adding .tectonignore to the repo root with the following glob expression will ignore fraud_detection.py altogether.

# Ignore everything under feature_services/
feature_services/*.py

# Alternatively, include nested directories under feature_services/
feature_services/**/*.py

# Alternatively, ignore a specific file
feature_services/fraud_detection.py

According the tecton plan, the repo contain the following files:

├── .tectonignore
├── data_sources
│   └── transactions_batch.py
├── entities.py
└── features
    └── fraudulent_transactions_count.py

Cloning an existing Workspace

Running tecton apply within a workspace simply applies the local feature repo to the Workspace. Suppose you would like to iterate on a live Workspace in an experimental development Workspace using a new git branch.

Select the existing Workspace using tecton workspace select:

$ tecton workspace select existing_workspace
Switched to workspace "existing_workspace".

(Optional) Restore the existing Workspace to your local repo
```
$ tecton restore
```

Create a new workspace

$ tecton workspace create my_workspace
Created workspace "my_workspace".
Switched to workspace "my_workspace".
You have created a new, empty workspace. Workspaces let
you create and manage an isolated feature repository.
Running "tecton plan" will compare your local repository
against the remote repository, which is initially empty.

Apply the current state to your new workspace.

The new workspace my_workspace is empty. Run tecton apply so that the new workspace reflects the current prod state, which has been captured in your local Git branch.

$ tecton apply
✅ Imported 11 Python modules from the feature repository
✅ Collecting local feature declarations
↓↓↓↓↓↓↓↓↓↓↓↓ Plan Start ↓↓↓↓↓↓↓↓↓↓
...
↑↑↑↑↑↑↑↑↑↑↑↑ Plan End ↑↑↑↑↑↑↑↑↑↑↑↑
Are you sure you want to apply this plan? [Y/n] Y
All done!

Create a new Git branch

To develop using the existing Workspace state as a base, create a new Git branch from main in which to store changes made in your workspace:
```
# Create a new git branch
git checkout -b "my_workspace_branch"
Switched to new branch "my_workspace_branch"
```

You now have a development environment with a workspace and a Git branch that reflect the current state of existing_workspace.

Applying Changes to a Feature Repository and a Workspace

Making a change to a feature repository

Previewing changes using tecton plan

Pushing changes using tecton apply