Integrations · Databases and warehouses

Load job postings data into DuckDB

Pull job posts into a local DuckDB file and analyze them with SQL in seconds - nested company data included.

Get an API key Read docs

Steps

You need: Python 3.9+, pip install requests duckdb and a jobdata API key.

Set your API key and the filters you need.
The script writes newline-delimited JSON and loads it with read_json_auto, keeping nested fields as structs.
Query with plain SQL, e.g. company.name for the hiring company.

DuckDB · python

import requests

API_KEY = "YOUR_API_KEY"


def fetch_jobs(**params):
    """Yield every job for the given filters, following pagination."""
    url = "https://jobdataapi.com/api/jobs/"
    params = {"page_size": 1000, **params}
    headers = {"Authorization": f"Api-Key {API_KEY}"}
    while url:
        resp = requests.get(url, headers=headers, params=params, timeout=60)
        resp.raise_for_status()
        data = resp.json()
        yield from data["results"]
        url, params = data["next"], None  # "next" already carries the filters


import json
import duckdb

with open("jobs.jsonl", "w") as f:
    for job in fetch_jobs(max_age=30, has_remote="true"):
        f.write(json.dumps(job) + "\n")

con = duckdb.connect("jobs.duckdb")
con.execute("CREATE OR REPLACE TABLE jobs AS SELECT * FROM read_json_auto('jobs.jsonl')")
print(con.sql("""
    SELECT company.name AS company, count(*) AS job_posts
    FROM jobs
    GROUP BY 1
    ORDER BY 2 DESC
    LIMIT 10
"""))

Notes

DuckDB reads the weekly Parquet downloads (access ultra) directly: SELECT * FROM 'jobs.parquet'.
Run it daily with max_age=1 to pick up new job posts; on access lite, drop max_age - the feed covers the past 90 days and upserting by id keeps it free of duplicates.

Ready to load real data?

All endpoints are public for limited testing - run the script without a key to see the structure, then add your key. Flat monthly plans with unlimited usage.

View pricing Explore endpoints