# Integrating the jobdata API with n8n

Use n8n to pull job listings from the jobdata API, transform them, and route them into databases, spreadsheets, alerts, or downstream automation.

---

The combination is a strong fit for scheduled market monitoring, job board syncs, lead enrichment, and analytics pipelines.

## Prerequisites

- An n8n instance (self-hosted or n8n Cloud)
- A jobdata API key from your [dashboard](/accounts/dashboard/)
- A target destination (for example PostgreSQL, Google Sheets, Airtable, Slack, email)

## API Basics You Should Know First

- Jobs endpoint: `https://jobdataapi.com/api/jobs/` (use trailing slash)
- Auth header format: `Authorization: Api-Key YOUR_API_KEY`
- Jobs response is paginated and returns:
  - `count`, `next`, `previous`, `results`
- `/api/jobs/` defaults to `max_age=90` if no explicit slicing/age filter is provided
- Some filters/features are plan-gated and may return `403`
- Unknown query params return `400`, so only use documented parameters

## Recommended n8n Architecture

For most production use cases, use this pattern:

1. **Schedule Trigger** (or another trigger)
2. **HTTP Request** (jobdata API call)
3. **Split Out** or **Code** (turn `results` array into 1 item per job)
4. **Optional transform/filter/dedupe nodes**
5. **Destination nodes** (DB, sheet, alert, etc.)
6. **Optional error workflow** using **Error Trigger**

## Step 1: Create Workflow and Trigger

Create a new workflow, then add a trigger:

- For periodic ingestion: **Schedule Trigger**
  - Configure minute/hour/day/week/month or custom cron
- For ad-hoc runs: **Manual Trigger**

Tip: Start with a low frequency (for example every 15-60 minutes), validate output, then tune cadence.

## Step 2: Configure HTTP Request Node

Add an **HTTP Request** node and configure:

- **Method**: `GET`
- **URL**: `https://jobdataapi.com/api/jobs/`
- **Authentication**: `Generic Credential Type` -> `Header Auth`
  - Header name: `Authorization`
  - Header value: `Api-Key YOUR_API_KEY`

### Query Parameters (example)

Turn on **Send Query Parameters** and add, for example:

- `title = python`
- `has_remote = true`
- `max_age = 7`
- `page_size = 200`

This keeps payloads smaller and usually improves reliability and speed.

## Step 3: Convert `results` Array to n8n Items

The API returns one object with `results` as an array. Most destination nodes expect one n8n item per row/object.

You have two good options:

### Option A: Split Out node (no code)

- Add **Split Out** node after HTTP Request
- Configure it to split the `results` field
- Output becomes one item per job

### Option B: Code node

If you need custom shaping, add a **Code** node with:

```javascript
const jobs = $json.results ?? [];

return jobs.map((job) => ({
  json: {
    id: job.id,
    title: job.title,
    location: job.location,
    published: job.published,
    application_url: job.application_url,
    company_id: job.company?.id,
    company_name: job.company?.name,
  },
}));
```

Note: n8n now uses the **Code** node (instead of older Function/Function Item naming).

## Step 4: Handle Pagination (Important)

If one page is not enough, use HTTP Request node pagination.

In **HTTP Request** -> **Add Option** -> **Pagination**:

### Mode 1: Response Contains Next URL

Use when API returns next-page URL (jobdata does, via `next`):

- **Pagination Mode**: `Response Contains Next URL`
- **Next URL** expression:

```javascript
{{ $response.body.next }}
```

### Mode 2: Update a Parameter in Each Request

Use page-number-based APIs:

- **Pagination Mode**: `Update a Parameter in Each Request`
- **Type**: `Query`
- **Name**: `page`
- **Value** expression:

```javascript
{{ $pageCount + 1 }}
```

You can also set query param `page_size` (for example 200) in the main node configuration.

## Step 5: Add Destination Logic

Common targets:

- **Postgres/MySQL**: insert/upsert jobs
- **Google Sheets/Airtable**: append rows
- **Slack/Email**: notify on matching jobs
- **Data warehouse**: pass to ETL ingestion workflows

Best practice:

- Add a dedupe key (`id` and/or `application_url`)
- Upsert rather than blind insert when possible

## Step 6: Add Reliability Controls

### Retry on Fail

In node **Settings**, enable **Retry On Fail** for API and destination nodes. Use a wait interval aligned with your tolerance for retries.

### Batching for rate limits

HTTP Request supports **Batching**:

- **Items per Batch**
- **Batch Interval (ms)**

Use this if downstream APIs or your target systems throttle aggressively.

### Error workflow

Create a separate workflow with **Error Trigger** as the first node, then send alerts (Slack/email) with execution details.

In each production workflow, set this error workflow in **Workflow Settings**.

## Step 7: Test, Debug, and Activate

1. Execute step-by-step in editor
2. Confirm expected payload shape at each node
3. Validate destination writes
4. Activate workflow

For debugging production issues:

- Use Executions view
- Use **Debug in editor** / copy execution data to replay fixes faster
- Pin sample data where useful during development

## Security Best Practices

- Store API keys in n8n credentials, not hardcoded in node text fields
- Restrict workflow/credential access with project permissions
- Rotate API keys periodically
- Avoid exposing keys in logs or notifications

## Performance and Cost Tips

- Pull only what you need using filters (`title`, `country_code`, `max_age`, etc.)
- Keep page size reasonable and paginate intentionally
- Prefer incremental polling windows over full-history pulls
- Batch destination writes when supported
- Keep heavy transformations in one controlled step (Code or dedicated transform nodes)

## Common Failure Modes

- **301 redirect**: URL missing trailing slash (`/api/jobs/`)
- **400**: invalid query parameter/value
- **401**: malformed/missing auth header
- **403**: plan-gated endpoint/filter
- **429**: request rate too high; add batching/wait/retry
- **5xx/timeouts**: transient upstream issues; retry and reduce request complexity

## Example Minimal Workflow

1. Schedule Trigger (every hour)
2. HTTP Request (`/api/jobs/`, filtered query, header auth)
3. Split Out (`results`)
4. Filter (optional business criteria)
5. Postgres upsert (or Google Sheets append)
6. Slack summary message (optional)

This gives you a reliable baseline that can be extended with pagination, dedupe, and error workflows.

## Conclusion

n8n + jobdata API provides a flexible and production-friendly way to automate job data ingestion and processing. Start with a simple scheduled pull, shape `results` into itemized records, then layer in pagination, retries, and dedicated error workflows as you scale.