Integrations · Databases and warehouses

Load job postings data into Snowflake

Write job posts into a Snowflake table from a pandas DataFrame with write_pandas - no stage management needed.

Get an API key Read docs

Steps

You need: Python 3.9+, pip install requests pandas "snowflake-connector-python[pandas]" and a jobdata API key.

Set your API key and Snowflake connection details.
The script flattens each job into columns and writes them with write_pandas, creating the table on first run.
Merge into a deduplicated table by ID if you append daily (see notes).
Schedule it with your orchestrator or a Snowflake-adjacent job runner.

Snowflake · python

import requests

API_KEY = "YOUR_API_KEY"


def fetch_jobs(**params):
    """Yield every job for the given filters, following pagination."""
    url = "https://jobdataapi.com/api/jobs/"
    params = {"page_size": 1000, **params}
    headers = {"Authorization": f"Api-Key {API_KEY}"}
    while url:
        resp = requests.get(url, headers=headers, params=params, timeout=60)
        resp.raise_for_status()
        data = resp.json()
        yield from data["results"]
        url, params = data["next"], None  # "next" already carries the filters


import pandas as pd
import snowflake.connector
from snowflake.connector.pandas_tools import write_pandas

df = pd.json_normalize(list(fetch_jobs(max_age=1)), sep="_")
df = df[["id", "title", "company_name", "location", "published",
         "has_remote", "application_url"]]
df.columns = [column.upper() for column in df.columns]

conn = snowflake.connector.connect(
    account="your_account", user="your_user", password="your_password",
    warehouse="COMPUTE_WH", database="JOBS", schema="PUBLIC",
)
success, chunks, rows, _ = write_pandas(conn, df, "JOB_POSTS", auto_create_table=True)
print(f"Wrote {rows} job posts")

Notes

Run it daily with max_age=1 to pick up new job posts; on access lite, drop max_age - the feed covers the past 90 days and upserting by id keeps it free of duplicates.
Append into a staging table and MERGE INTO job_posts USING staging ON job_posts.id = staging.id ... to keep one row per job.
For full-history loads, the weekly Parquet downloads (access ultra) can be copied in with COPY INTO ... FILE_FORMAT = (TYPE = PARQUET).

Ready to load real data?

All endpoints are public for limited testing - run the script without a key to see the structure, then add your key. Flat monthly plans with unlimited usage.

View pricing Explore endpoints