# Load job postings data into pandas

> Load job posts into a pandas DataFrame with flattened company fields, then save as Parquet for analysis or modeling.

Page: https://jobdataapi.com/integrations/pandas/
Category: Programming languages
You need: Python 3.9+, pip install requests pandas pyarrow and a jobdata API key (https://jobdataapi.com/accounts/pricing/).

## Steps

1. Set your API key and filters.
2. json_normalize flattens nested objects, so the company name becomes company_name.
3. Save to Parquet (or CSV) and analyze, chart or feed it into a model.

## Code

```python
import requests

API_KEY = "YOUR_API_KEY"


def fetch_jobs(**params):
    """Yield every job for the given filters, following pagination."""
    url = "https://jobdataapi.com/api/jobs/"
    params = {"page_size": 1000, **params}
    headers = {"Authorization": f"Api-Key {API_KEY}"}
    while url:
        resp = requests.get(url, headers=headers, params=params, timeout=60)
        resp.raise_for_status()
        data = resp.json()
        yield from data["results"]
        url, params = data["next"], None  # "next" already carries the filters


import pandas as pd

jobs = list(fetch_jobs(max_age=30, country_code="US", title="data engineer"))
df = pd.json_normalize(jobs, sep="_")
df["published"] = pd.to_datetime(df["published"])
df.to_parquet("jobs.parquet", index=False)

print(df.groupby("company_name").size().sort_values(ascending=False).head(10))
```

## Notes

- Add description_md=true for Markdown descriptions, or turn descriptions off with description_off=true for smaller, faster responses.
- Run it daily with max_age=1 to pick up new job posts; on access lite, drop max_age - the feed covers the past 90 days and upserting by id keeps it free of duplicates.

## Related docs

- [Using the jobdata API for Machine Learning with Cleaned Job Descriptions](https://jobdataapi.com/c/using-the-jobdata-api-for-machine-learning-with-cleaned-job-descriptions/)
- [Optimizing API Requests: A Guide to Efficient jobdata API Usage](https://jobdataapi.com/c/optimizing-api-requests-a-guide-to-efficient-jobdata-api-usage/)
- [Jobs API Ingestion Guide: Reliable Historical Backfills and Incremental Syncs](https://jobdataapi.com/c/jobs-api-ingestion-guide-reliable-historical-backfills-and-incremental-syncs/)