Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 07b493f

Browse files
Copilotfranciscojavierarceo
Copilot
and
franciscojavierarceo
authored
docs: Add comprehensive type system reference with Map type support (#5855)
* Initial plan * docs: Add comprehensive type system documentation with Map support - Add detailed tables for all supported types (primitive, array, map) - Include complete feature view example with all types - Add Map type usage examples with nested structures - Improve documentation structure and clarity Co-authored-by: franciscojavierarceo <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: franciscojavierarceo <[email protected]>
1 parent 7d9c06f commit 07b493f

File tree

1 file changed

+139
-3
lines changed

1 file changed

+139
-3
lines changed

‎docs/reference/type-system.md‎

Lines changed: 139 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,149 @@
33
## Motivation
44

55
Feast uses an internal type system to provide guarantees on training and serving data.
6-
Feast currently supports eight primitive types - `INT32`, `INT64`, `FLOAT32`, `FLOAT64`, `STRING`, `BYTES`, `BOOL`, and `UNIX_TIMESTAMP` - and the corresponding array types.
7-
Map type is also supported using a key of `STRING` type and any supported feast type as a value.
6+
Feast supports primitive types, array types, and map types for feature values.
87
Null types are not supported, although the `UNIX_TIMESTAMP` type is nullable.
98
The type system is controlled by [`Value.proto`](https://github.com/feast-dev/feast/blob/master/protos/feast/types/Value.proto) in protobuf and by [`types.py`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/types.py) in Python.
109
Type conversion logic can be found in [`type_map.py`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/type_map.py).
1110

12-
## Examples
11+
## Supported Types
12+
13+
Feast supports the following data types:
14+
15+
### Primitive Types
16+
17+
| Feast Type | Python Type | Description |
18+
|------------|-------------|-------------|
19+
| `Int32` | `int` | 32-bit signed integer |
20+
| `Int64` | `int` | 64-bit signed integer |
21+
| `Float32` | `float` | 32-bit floating point |
22+
| `Float64` | `float` | 64-bit floating point |
23+
| `String` | `str` | String/text value |
24+
| `Bytes` | `bytes` | Binary data |
25+
| `Bool` | `bool` | Boolean value |
26+
| `UnixTimestamp` | `datetime` | Unix timestamp (nullable) |
27+
28+
### Array Types
29+
30+
All primitive types have corresponding array (list) types:
31+
32+
| Feast Type | Python Type | Description |
33+
|------------|-------------|-------------|
34+
| `Array(Int32)` | `List[int]` | List of 32-bit integers |
35+
| `Array(Int64)` | `List[int]` | List of 64-bit integers |
36+
| `Array(Float32)` | `List[float]` | List of 32-bit floats |
37+
| `Array(Float64)` | `List[float]` | List of 64-bit floats |
38+
| `Array(String)` | `List[str]` | List of strings |
39+
| `Array(Bytes)` | `List[bytes]` | List of binary data |
40+
| `Array(Bool)` | `List[bool]` | List of booleans |
41+
| `Array(UnixTimestamp)` | `List[datetime]` | List of timestamps |
42+
43+
### Map Types
44+
45+
Map types allow storing dictionary-like data structures:
46+
47+
| Feast Type | Python Type | Description |
48+
|------------|-------------|-------------|
49+
| `Map` | `Dict[str, Any]` | Dictionary with string keys and any supported Feast type as values (including nested maps) |
50+
| `Array(Map)` | `List[Dict[str, Any]]` | List of dictionaries |
51+
52+
**Note:** Map keys must always be strings. Map values can be any supported Feast type, including primitives, arrays, or nested maps.
53+
54+
## Complete Feature View Example
55+
56+
Below is a complete example showing how to define a feature view with all supported types:
57+
58+
```python
59+
from datetime import timedelta
60+
from feast import Entity, FeatureView, Field, FileSource
61+
from feast.types import (
62+
Int32, Int64, Float32, Float64, String, Bytes, Bool, UnixTimestamp,
63+
Array, Map
64+
)
65+
66+
# Define a data source
67+
user_features_source = FileSource(
68+
path="data/user_features.parquet",
69+
timestamp_field="event_timestamp",
70+
)
71+
72+
# Define an entity
73+
user = Entity(
74+
name="user_id",
75+
description="User identifier",
76+
)
77+
78+
# Define a feature view with all supported types
79+
user_features = FeatureView(
80+
name="user_features",
81+
entities=[user],
82+
ttl=timedelta(days=1),
83+
schema=[
84+
# Primitive types
85+
Field(name="age", dtype=Int32),
86+
Field(name="account_balance", dtype=Int64),
87+
Field(name="transaction_amount", dtype=Float32),
88+
Field(name="credit_score", dtype=Float64),
89+
Field(name="username", dtype=String),
90+
Field(name="profile_picture", dtype=Bytes),
91+
Field(name="is_active", dtype=Bool),
92+
Field(name="last_login", dtype=UnixTimestamp),
93+
94+
# Array types
95+
Field(name="daily_steps", dtype=Array(Int32)),
96+
Field(name="transaction_history", dtype=Array(Int64)),
97+
Field(name="ratings", dtype=Array(Float32)),
98+
Field(name="portfolio_values", dtype=Array(Float64)),
99+
Field(name="favorite_items", dtype=Array(String)),
100+
Field(name="document_hashes", dtype=Array(Bytes)),
101+
Field(name="notification_settings", dtype=Array(Bool)),
102+
Field(name="login_timestamps", dtype=Array(UnixTimestamp)),
103+
104+
# Map types
105+
Field(name="user_preferences", dtype=Map),
106+
Field(name="metadata", dtype=Map),
107+
Field(name="activity_log", dtype=Array(Map)),
108+
],
109+
source=user_features_source,
110+
)
111+
```
112+
113+
### Map Type Usage Examples
114+
115+
Maps can store complex nested data structures:
116+
117+
```python
118+
# Simple map
119+
user_preferences = {
120+
"theme": "dark",
121+
"language": "en",
122+
"notifications_enabled": True,
123+
"font_size": 14
124+
}
125+
126+
# Nested map
127+
metadata = {
128+
"profile": {
129+
"bio": "Software engineer",
130+
"location": "San Francisco"
131+
},
132+
"stats": {
133+
"followers": 1000,
134+
"posts": 250
135+
}
136+
}
137+
138+
# List of maps
139+
activity_log = [
140+
{"action": "login", "timestamp": "2024-01-01T10:00:00", "ip": "192.168.1.1"},
141+
{"action": "purchase", "timestamp": "2024-01-01T11:30:00", "amount": 99.99},
142+
{"action": "logout", "timestamp": "2024-01-01T12:00:00"}
143+
]
144+
```
145+
146+
## Type System in Practice
147+
148+
The sections below explain how Feast uses its type system in different contexts.
13149

14150
### Feature inference
15151

0 commit comments

Comments
 (0)