Seed

Seeds load the output of a command into a table on their source before any model queries it.

The command is run with a given set of args and needs to return a well formatted with a header row to stdout. Visivo writes that CSV into table_name on the source, so every model pointed at the source can query it like any other table.

Seeds run once per source, ahead of the models that depend on it, so several models can share one seeded table without loading it more than once.

Example

EchoPython ScriptCSV FileJoining Two Seeds

Echoing all of your data is probably not a very practical example, but it does nicely demonstrate how the feature works!

sources:
  - name: local
    type: duckdb
    database: target/local.duckdb
    seeds:
      - table_name: csv
        args:
          - echo
          - |
            x,y
            1,9
            2,1
            3,2

models:
  - name: csv
    source: ${ref(local)}
    sql: select * from csv

In this example we'll use python to generate a csv of processes running on your machine and make that csv available to Visivo as a table for analysis.

created_processes_csv.py

import subprocess
import csv
import sys

# Execute the 'ps aux' command
result = subprocess.run(["ps", "aux"], stdout=subprocess.PIPE, text=True)

# Split the output into lines
lines = result.stdout.strip().split("/n")

# Write CSV to stdout
writer = csv.writer(sys.stdout)
writer.writerow(
    ["USER","PID","%CPU","%MEM","VSZ","RSS","TTY","STAT","START","TIME","COMMAND"]
)  # Header

for line in lines[1:]:  # Skip the header line from the ps output
    row = line.split(None, 10)  # Split on whitespace, but only for the first 10 columns
    writer.writerow(row)

With your script ready to go, all you have to do is convert python create_processes_csv.py into the args list format in a seed.

sources:
  - name: local
    type: duckdb
    database: target/local.duckdb
    seeds:
      - table_name: processes
        args:
          - python
          - create_processes_csv.py

One of the best use cases for a seed is to store a static csv in your project and cat it into a table. This is great because it's simple and allows you to version control your csv data.

file.csv

columns,go,up,here
1,text,more text,6
2,stuff,more stuff,7

Then just cat the csv file in a seed.

sources:
  - name: local
    type: duckdb
    database: target/local.duckdb
    seeds:
      - table_name: file_data
        args:
          - cat
          - file.csv

Seeds on the same source land in the same database, so a single model can join across them.

sources:
  - name: local
    type: duckdb
    database: target/local.duckdb
    seeds:
      - table_name: raw_orders
        args: [cat, data/orders.csv]
      - table_name: raw_items
        args: [cat, data/items.csv]

models:
  - name: orders_with_items
    source: ${ref(local)}
    sql: |
      select * from raw_orders o join raw_items i on o.id = i.order_id

The args are python subprocess list args and you can read their source documentation here.

Attributes

Field	Type	Default	Description
path	string	None	A unique path to this object
args	array	None	An array of the variables that build your command to run.
table_name	string	None	The name of the table to write the resulting csv to on the source.
allow_empty	boolean	False	Whether to allow the command to return an empty csv.
existing_table	string	skip	What to do when the seed's `table_name` already exists on the source. `skip` (default) leaves the existing table untouched and does not run the command; `append` runs the command and inserts its rows into the existing table; `overwrite` runs the command and replaces the table. When the table is absent the seed always runs and creates it.