AI/ML notes

tags

Tags

Great follow-up! In Metaflow, tags are lightweight metadata labels you can attach to flows, runs, steps, and tasks. They help you organize, filter, search, and track your executions, especially useful in large projects or shared environments.


๐Ÿท๏ธ Types of Tags in Metaflow

There are two main categories:

1. System Tags

  • Automatically added by Metaflow
  • Examples:
    • metaflow_version:2.10.3
    • user:alice
    • runtime:python3.10
    • origin:cli or origin:notebook
  • Useful for filtering by user, version, or run environment

2. User Tags

  • Tags you manually assign in code or via CLI
  • Examples:
    • model:resnet
    • exp:baseline
    • priority:high

๐Ÿงช How to Add Tags

๐Ÿงต A. From CLI

python my_flow.py run --tag exp:baseline --tag priority:high

These tags are applied to the entire run.


๐Ÿงต B. In Code (self.add_tag)

Inside a step:

@step
def start(self):
    self.add_tag("data:normalized")
    self.add_tag("source:csv")
    self.next(self.end)

You can also remove a tag with:

self.remove_tag("data:normalized")

๐Ÿ“ฆ How to Access Tags Programmatically

from metaflow import Flow

run = Flow('MyFlow').latest_run
print("User tags:", run.user_tags)
print("System tags:", run.system_tags)

for step in run:
    for task in step.tasks:
        print("Task tags:", task.tags)

๐Ÿ“ Why Use Tags?

Use Case Tag Example
Grouping experiments exp:augmented, exp:baseline
Model tracking model:resnet, model:svm
Debug runs debug, issue:1234
Data provenance data:v2, data:filtered
Priority scheduling priority:high (for batch runs)

๐Ÿ” How to Filter by Tags (CLI)

metaflow --help  # to list supported commands

metaflow status MyFlow --tag exp:baseline

Or programmatically:

from metaflow import Flow

runs = [run for run in Flow('MyFlow').runs() if 'exp:baseline' in run.user_tags]

โœ… Summary

Feature How to Use
Add tag via CLI --tag key:value
Add tag in code self.add_tag('key:value')
Remove tag self.remove_tag('key:value')
Access tags .user_tags, .system_tags, .tags on flow/run/task