# Load job postings data into Snowflake

> Write job posts into a Snowflake table from a pandas DataFrame with write_pandas - no stage management needed.

Page: https://jobdataapi.com/integrations/snowflake/
Category: Databases and warehouses
You need: Python 3.9+, pip install requests pandas "snowflake-connector-python[pandas]" and a jobdata API key (https://jobdataapi.com/accounts/pricing/).

## Steps

1. Set your API key and Snowflake connection details.
2. The script flattens each job into columns and writes them with write_pandas, creating the table on first run.
3. Merge into a deduplicated table by ID if you append daily (see notes).
4. Schedule it with your orchestrator or a Snowflake-adjacent job runner.

## Code

```python
import requests

API_KEY = "YOUR_API_KEY"


def fetch_jobs(**params):
    """Yield every job for the given filters, following pagination."""
    url = "https://jobdataapi.com/api/jobs/"
    params = {"page_size": 1000, **params}
    headers = {"Authorization": f"Api-Key {API_KEY}"}
    while url:
        resp = requests.get(url, headers=headers, params=params, timeout=60)
        resp.raise_for_status()
        data = resp.json()
        yield from data["results"]
        url, params = data["next"], None  # "next" already carries the filters


import pandas as pd
import snowflake.connector
from snowflake.connector.pandas_tools import write_pandas

df = pd.json_normalize(list(fetch_jobs(max_age=1)), sep="_")
df = df[["id", "title", "company_name", "location", "published",
         "has_remote", "application_url"]]
df.columns = [column.upper() for column in df.columns]

conn = snowflake.connector.connect(
    account="your_account", user="your_user", password="your_password",
    warehouse="COMPUTE_WH", database="JOBS", schema="PUBLIC",
)
success, chunks, rows, _ = write_pandas(conn, df, "JOB_POSTS", auto_create_table=True)
print(f"Wrote {rows} job posts")
```

## Notes

- Run it daily with max_age=1 to pick up new job posts; on access lite, drop max_age - the feed covers the past 90 days and upserting by id keeps it free of duplicates.
- Append into a staging table and MERGE INTO job_posts USING staging ON job_posts.id = staging.id ... to keep one row per job.
- For full-history loads, the weekly Parquet downloads (access ultra) can be copied in with COPY INTO ... FILE_FORMAT = (TYPE = PARQUET).

## Related docs

- [Jobs API Ingestion Guide: Reliable Historical Backfills and Incremental Syncs](https://jobdataapi.com/c/jobs-api-ingestion-guide-reliable-historical-backfills-and-incremental-syncs/)
- [Date, ID, and Age Slicing Parameters Documentation](https://jobdataapi.com/c/date-id-and-age-slicing-parameters-documentation/)
- [Jobs Expired API Endpoint Documentation](https://jobdataapi.com/c/jobs-expired-api-endpoint-documentation/)
- [CSV and Parquet File Downloads Documentation](https://jobdataapi.com/c/csv-and-parquet-file-downloads-documentation/)