AI/ML notes

cards

Cards

Think of a card like a “Jupyter Notebook snapshot” or “HTML report” that summarizes data, logs, metrics, plots — whatever you want — and attaches it to a specific step of a run, so you (or your team) can inspect it later without rerunning code.


🃏 What Is a Metaflow Card?

A card is:

  • An optional artifact attached to a step
  • Rendered as HTML (interactive or static)
  • Meant for reports, summaries, dashboards
  • Automatically versioned and linked to runs
  • Can include metrics, charts, text, tables, images, etc.

It means Metaflow Cards are designed to be the final human-readable output of your flow — something you or your team can use to:

✅ Understand the outcome of a run

Not by digging through logs or CLI output, but by opening a clean, structured report.

✅ Summarize key insights and artifacts

Like:

  • Metrics (e.g., accuracy, loss, AUC)
  • Tables (e.g., test results, parameter values)
  • Plots (e.g., loss curves, feature importances)
  • Textual notes (e.g., “this run used dropout=0.3 and early stopping”)
  • File previews (images, logs)
  • Any custom HTML/CSS/JS you want

✅ Provide a “dashboard” feel

Multiple runs with cards give a notebook-like record of your experiments. For instance:

  • A card on train step shows learning curves
  • A card on eval step shows metrics + confusion matrix
  • A card on deploy step logs model registry details

📊 Card = Automated Report Per Step

In other words, instead of:

🚫 Having to manually look at CLI logs
🚫 Re-running notebook cells to recreate plots
🚫 Manually compiling experiment results into Confluence or Notion

You now have:

Self-contained, auto-generated HTML reports
✅ Tied to each flow/run/step
✅ Viewable via:

metaflow card view FlowName/run_id/step_name

Absolutely! Based on everything we’ve discussed plus the official Metaflow docs (Visualizing Results — Metaflow Docs), here’s a clean, comprehensive summary of:


✅ What Cards Are Doing (Purpose)

Cards provide:

  • 📊 Auto-generated reports tied to steps in your flow
  • 🧠 A human-friendly summary of data, metrics, and artifacts
  • 🔍 Clickable dashboards when viewing in Metaflow UI (or via CLI tools)
  • 📎 Versioned, immutable attachments to flow runs
  • 🪄 A replacement for manual Notion/Excel/Jupyter summaries

You write them once, and they travel with your flow runs forever.


🧰 What You Can Do With Cards

Inside a card, you can include:

Feature Description
Markdown Add documentation, notes, and structured explanations
Tables Display pandas DataFrames, dictionaries, config settings
Plots Render Matplotlib or Plotly charts
Artifacts Log key variables, metrics, results from your steps
HTML/CSS/JS Embed advanced visuals or interactive elements
Logs Include stdout/stderr and metadata
Images Embed generated figures (e.g., confusion matrices, sample images)

And all of it can be automatically rendered per flow run and step.


🔧 Built-in Card Types

Type Description
default Basic stdout, artifacts, metadata
blank Minimal — useful for custom rendering
task_spec Shows run/step/task metadata
default_json Outputs artifacts as JSON

📌 Note: Rich card viewing and CLI tools require outerbounds-metaflow.


⚙️ UNDERLYING MECHANISM

✅ Card Rendering Pipeline

  1. At runtime, if a step is decorated with @card, Metaflow:

    • Captures all artifacts (self.var)
    • Captures stdout/stderr, step metadata
    • Identifies the card type (type='default', type='my_card', etc.)
  2. The card type is mapped to a Python class (MyCard(MetaflowCard)) which contains:

    • A render() method → outputs HTML content
    • Optional references to templates or assets
  3. The render() method receives a task object:

    • Contains .data, .stdout, .metadata, etc.
    • Your card can use these to generate markdown/HTML
  4. The generated card content is stored as:

    _cards/FlowName/RunID/StepName/CardType/
    
  5. Cards are viewed via:

    • metaflow card view
    • Web-based Metaflow UI (Outerbounds)

🗂️ Card Template Structure

A card plugin (template) typically contains:

my_card/
├── card.py          # Required: the card class and render() logic
├── card.json        # Metadata for the card (name, description, etc.)
└── assets/          # Optional: static files (CSS, JS, images)

Example card.py

from metaflow.cards import MetaflowCard

class MyCard(MetaflowCard):
    type = "my_card"

    def render(self, task):
        score = task.data.get("score", "N/A")
        return f"""
        <h1>Model Report</h1>
        <p>Final Score: {score}</p>
        <p>Run: {task.pathspec}</p>
        """

🧪 Writing and Using a Custom Card

1. Generate a dev card template

metaflow card dev init my_card

2. Implement render() in card.py

Use task to access data and return an HTML string.

3. Install your card:

metaflow card install my_card/

4. Use it in your flow:

@card(type='my_card')
@step
def report(self):
    self.score = 0.91

🔄 The Card Object API

Inside render(self, task), you get a rich task object with:

Property Description
task.data All artifacts (self.var)
task.stdout / stderr Captured output
task.metadata User/system metadata
task.pathspec Fully qualified ID
task.created_at, .finished_at Timing info
task.environment_info Runtime info (e.g. Python version)

✅ Summary Table

Concept Description
@card decorator Attach a card to a step
type='...' Specifies the card template to use
render(task) Main logic to turn a task into HTML
card.py Defines the custom card plugin
card.json Metadata for card installation
assets/ Static assets for rich cards
metaflow card install Registers a custom card
metaflow card view Displays card output (CLI)
UI support Cards are displayed in Metaflow UI (Outerbounds)

📈 Pro Tips for Custom Cards

  • ✅ Use self.set_metadata(key, value) to add structured metadata
  • ✅ Use self.print() or print() for human-readable logs
  • ✅ Include charts or external CSS/JS in assets/
  • ✅ Test cards locally using metaflow card dev render or card view

🧠 Real-World Use Cases

Use Case What the Card Shows
Model training Hyperparams, learning curves, training time
Evaluation Metrics, ROC/AUC curves, confusion matrix
Monitoring Logs, alerts, time-to-run, success/failure history
Data QA Schema checks, missing values, data previews
Experiments Side-by-side comparison of metrics/tags