# CSV and Parquet File Downloads Documentation Weekly CSV and Parquet downloads for complete job data, company context, and analytical workflows. --- ## Introduction The jobdata API **access ultra** and **access ultra+** packages provide complete weekly data downloads for bulk backfills, warehouse loading, offline analysis, and reporting. Downloads are available in compressed CSV files and in one denormalized Parquet file for customers who need fast analytical scans. The download snapshot contains all jobs in the database, including historical and expired jobs. It is not limited to the API's default age window. The export is refreshed weekly, normally on Sunday around 13:00 UTC. The CSV files preserve the existing export fields and relational model. Large files sourced from the `Job` table are segmented by the job's `published` year, while reference and dimension tables remain single non-annual files. The Parquet file combines essential job fields with basic company data in one row per job. ## Access and download links File downloads are available only to **access ultra** and **access ultra+** customers. Staff accounts may also have administrative access. After subscribing, open the [dashboard](/accounts/dashboard/) and select **Generate new links** in the Download access section. The dashboard shows the files currently available, friendly labels, file sizes, annual groupings, and individual copy buttons. **Copy all available links** copies the complete current link set. Download URLs use a separate download access key, not an API key: ```text https://jobdataapi.com/download// ``` Keep the download key private. Generating a new download key invalidates the previous key and its associated download URLs. Do not commit download URLs to source repositories, embed them in public frontend code, or expose them in public logs. Files are large and are intended for server-side or command-line downloads. For example: ```sh curl -L -o jobs_2025.csv.gz https://jobdataapi.com/download//jobs_2025.csv.gz gzip -dc jobs_2025.csv.gz | head ``` The dashboard file size is the compressed download size. Decompressed CSV files require substantially more disk space. ## CSV file organization ### Non-annual reference files The following files remain single files and are not segmented by year: - `industries.csv.gz` - `company_types.csv.gz` - `companies.csv.gz` - `currency_rates.csv.gz` - `job_types.csv.gz` - `job_regions.csv.gz` - `job_countries.csv.gz` - `job_states.csv.gz` - `job_cities.csv.gz` - `tags.csv.gz` These files contain dimensions and reference data used to interpret IDs in job and relation files. Company counts and open-job counts are snapshots taken at export time. ### Annual job files Every CSV generated directly from the `Job` table is segmented by the UTC publication year: - `jobs_.csv.gz`: job records with the original HTML `description` field. - `jobs_md_.csv.gz`: the same job records with `description_md` instead of HTML `description`. - `jobs_em_.csv.gz`: job ID, company ID, and the 768-dimensional `embedding_3sh` vector. For example, a 2025 export contains `jobs_2025.csv.gz`, `jobs_md_2025.csv.gz`, and `jobs_em_2025.csv.gz`. ### Annual relation files The many-to-many relation tables are segmented using the publication year of their related job. Each row still has the same two-column structure as before, and each relation row appears in the same year as its job: - `job_job_types_.csv.gz`: `id`, `types` - `job_job_cities_.csv.gz`: `id`, `cities` - `job_job_states_.csv.gz`: `id`, `states` - `job_job_countries_.csv.gz`: `id`, `countries` - `job_job_regions_.csv.gz`: `id`, `regions` - `job_job_tags_.csv.gz`: `id`, `tags` ### Compatibility filenames The historical unsuffixed job filenames remain available as compatibility aliases: - `jobs.csv.gz` - `jobs_md.csv.gz` - `jobs_em.csv.gz` - `job_job_types.csv.gz` - `job_job_cities.csv.gz` - `job_job_states.csv.gz` - `job_job_countries.csv.gz` - `job_job_regions.csv.gz` - `job_job_tags.csv.gz` These aliases point to the **current publication year only**. They are not full historical snapshots. Use the year-suffixed files for a complete export. New integrations should use explicit annual filenames. ## CSV field reference The annual files retain the established field names and values. The year suffix changes only the filename, not the row or column format. ### `jobs_.csv.gz` - `id`: Unique job identifier. - `ext_id`: External job identifier. - `company_id`: ID of the company offering the job. - `published`: Publication date/time. - `expired`: Expiration date/time, when available. - `title`: Job title. - `location`: Original job location value. - `location_string`: Normalized job location value, when available. - `description`: Sanitized HTML job description. - `application_url`: Application link. - `experience_level`: Classified experience level. - `language`: Two-letter language code. - `has_remote`: Whether remote work is indicated. - `work_mode`: Work mode classification. - `salary_min`, `salary_max`: Original annual salary range. - `salary_currency`: Original salary currency code. - `salary_min_est`, `salary_max_est`: Estimated annual salary range. - `salary_min_usd`, `salary_max_usd`: Annual salary converted or estimated in USD. - `salary_min_eur`, `salary_max_eur`: Annual salary converted or estimated in EUR. ### `jobs_md_.csv.gz` This file has the same fields as `jobs_.csv.gz`, except that `description_md` replaces the HTML `description` field. Markdown is useful for text processing, search indexing, and systems that do not want to parse HTML. ### `jobs_em_.csv.gz` - `id`: Job identifier. - `company_id`: Company identifier. - `embedding_3sh`: 768-dimensional half-precision vector embedding. Some recent jobs may have an empty embedding while downstream embedding processing completes. ## Denormalized Parquet export `jobs_companies.parquet` is a single compressed Parquet file designed for fast analytical reads with DuckDB, Polars, PyArrow, Spark, and similar tools. It contains one row per job and includes basic company attributes in the same row, avoiding a company join for common analysis. The file contains all jobs, including historical and expired jobs. Use `published_year` to filter annual analyses. The file is compressed with Zstandard and written in batches. A replacement is published atomically, so the previous complete file remains downloadable while a new weekly file is being generated. Embeddings are intentionally not included in Parquet. The Markdown description is included, while the original HTML description is omitted. The following fields are also omitted from the Parquet schema to keep analytical scans lean: job creation and update timestamps, `experience_level_predicted`, and denormalized job-level company flags. ### Parquet job fields - `job_id`, `job_ext_id`: Job identifiers. - `published_at`, `expired_at`: Publication and expiration timestamps in UTC. - `published_year`: UTC publication year. - `job_title`, `job_location`, `job_location_string`: Job title and location values. - `job_description_md`: Markdown job description. - `job_application_url`: Application link. - `job_language`: Language code. - `job_has_remote`: Remote-work indicator. - `job_work_mode`: Work mode classification. - `job_experience_level`: Experience-level classification. - `job_salary_min`, `job_salary_max`: Original annual salary range. - `job_salary_currency`: Original salary currency. - `job_salary_min_est`, `job_salary_max_est`: Estimated annual salary range. - `job_salary_min_usd`, `job_salary_max_usd`: USD salary range. - `job_salary_min_eur`, `job_salary_max_eur`: EUR salary range. ### Parquet relation fields The following fields are native Parquet lists, preserving many-to-many relationships without duplicating the job row: - `job_cities`: List of structs containing `id`, `name`, `state_name`, `country_code`, `latitude`, and `longitude`. - `job_states`: List of structs containing `id`, `name`, `code`, and `country_code`. - `job_countries`: List of structs containing `id`, `code`, `name`, and `region_name`. - `job_regions`: List of structs containing `id` and `name`. - `job_types`: List of structs containing `id` and `name`. - `job_tags`: List of structs containing `id`, `tag_type`, `name`, and `parent_id`. List values can be unnested in analytical SQL. A job with multiple locations remains one job row, so job counts do not require deduplication unless a query intentionally unnests a relation. ### Parquet company fields - `company_id`, `company_meta_id`: Company identifiers. - `company_source_id`, `company_source_name`: Source identifiers and source name. - `company_name`: Company name. - `company_website_url`, `company_linkedin_url`: Company web and LinkedIn URLs. - `company_info_hq`, `company_info_hq_string`: Headquarters values. - `company_info_size`, `company_info_founded`: Company size and founding year. - `company_info_specialties`: Company specialties text. - `company_industry_id`, `company_industry_name`: Industry classification. - `company_type_id`, `company_type_name`: Company type classification. - `company_num_jobs_open`, `company_num_jobs_total`: Company job-count snapshots at export time. ### DuckDB examples ```python import duckdb path = 'jobs_companies.parquet' con = duckdb.connect() rows = con.execute(''' SELECT published_year, COUNT(*) AS jobs, COUNT(*) FILTER (WHERE job_has_remote) AS remote_jobs, ROUND(AVG(job_salary_min_usd), 0) AS avg_min_salary_usd FROM read_parquet(?) WHERE published_year IS NOT NULL GROUP BY published_year ORDER BY published_year ''', [path]).fetchall() for row in rows: print(row) ``` To query nested country relations without changing the one-row-per-job grain: ```python rows = con.execute(''' SELECT country.code AS country_code, country.name AS country_name, COUNT(DISTINCT jobs.job_id) AS jobs FROM read_parquet(?) AS jobs CROSS JOIN UNNEST(jobs.job_countries) AS countries(country) GROUP BY country.code, country.name ORDER BY jobs DESC LIMIT 20 ''', [path]).fetchall() for row in rows: print(row) ``` ## Sample files and SQLite demo The public sample files contain recent information for 100 companies and their latest 5,000 jobs. They are intended to demonstrate the CSV field formats and the relational SQLite schema; they are not the complete weekly customer export and they are not annualized. The sample database excludes the Markdown and embedding files, as before. Sample files: - [Industries](/media/data/industries.csv) - [Company types](/media/data/company_types.csv) - [Companies](/media/data/companies.csv) - [Currency rates](/media/data/currency_rates.csv) - [Job types](/media/data/job_types.csv) - [Job regions](/media/data/job_regions.csv) - [Job countries](/media/data/job_countries.csv) - [Job states](/media/data/job_states.csv) - [Job cities](/media/data/job_cities.csv) - [Jobs](/media/data/jobs.csv) - [Jobs in Markdown](/media/data/jobs_md.csv) - [Job embeddings](/media/data/jobs_em.csv) - [Job type relations](/media/data/job_job_types.csv) - [Job city relations](/media/data/job_job_cities.csv) - [Job state relations](/media/data/job_job_states.csv) - [Job country relations](/media/data/job_job_countries.csv) - [Job region relations](/media/data/job_job_regions.csv) - [Job tag relations](/media/data/job_job_tags.csv) - [Tags](/media/data/tags.csv) The [SQLite demo database](https://jobdataapi.com/media/data/demo.sqlite3) contains the sample data without Markdown descriptions or embeddings. The [SQLite schema](/static/data/sqlite_export_schema.sql) shows how to join the relational CSV tables. ## Examples Data downloads are particularly useful for data scientists and analysts who need complete datasets for advanced analytics, reporting, and repeatable offline workflows. They avoid the repeated pagination, joining, and retry logic required for a large API backfill. The examples below use an explicit publication year because the customer job CSVs are annualized. Replace `2025` with each year you need; download `companies.csv.gz` once because reference files are not annualized. ### Use Case 1: Market analysis **Objective:** Analyze job market trends to identify high-demand skills and emerging industries. #### 1. Download and load data ```sh YEAR=2025 curl -L -o "jobs_${YEAR}.csv.gz" "https://jobdataapi.com/download//jobs_${YEAR}.csv.gz" curl -L -o companies.csv.gz "https://jobdataapi.com/download//companies.csv.gz" gzip -dc "jobs_${YEAR}.csv.gz" > "jobs_${YEAR}.csv" gzip -dc companies.csv.gz > companies.csv ``` #### 2. Analyze job postings ```python import pandas as pd year = 2025 jobs = pd.read_csv(f'jobs_{year}.csv') companies = pd.read_csv('companies.csv') job_data = jobs.merge( companies[['id', 'name', 'info_industry_id', 'info_type_id']], left_on='company_id', right_on='id', how='left', suffixes=('', '_company'), ) skills = ['Python', 'JavaScript', 'SQL', 'Java', 'AWS'] description = job_data['description'].fillna('') skill_counts = { skill: description.str.contains(skill, case=False, regex=False).sum() for skill in skills } print(skill_counts) print(job_data.groupby('info_industry_id').size().sort_values(ascending=False).head(10)) ``` #### 3. Visualize results ```python import matplotlib.pyplot as plt plt.bar(skill_counts.keys(), skill_counts.values()) plt.xlabel('Skill') plt.ylabel('Job count') plt.title(f'High-demand skills in {year}') plt.show() ``` ### Use Case 2: Competitive analysis **Objective:** Analyze hiring patterns and job distribution for a known set of companies. #### 1. Download and load data ```sh YEAR=2025 curl -L -o "jobs_${YEAR}.csv.gz" "https://jobdataapi.com/download//jobs_${YEAR}.csv.gz" curl -L -o companies.csv.gz "https://jobdataapi.com/download//companies.csv.gz" gzip -dc "jobs_${YEAR}.csv.gz" > "jobs_${YEAR}.csv" gzip -dc companies.csv.gz > companies.csv ``` #### 2. Analyze competitor hiring patterns ```python import pandas as pd year = 2025 jobs = pd.read_csv(f'jobs_{year}.csv') companies = pd.read_csv('companies.csv') competitor_ids = [101, 102, 103] # Example company IDs competitor_jobs = jobs[jobs['company_id'].isin(competitor_ids)] competitor_counts = competitor_jobs['company_id'].value_counts() company_names = companies.set_index('id')['name'].to_dict() competitor_counts.index = competitor_counts.index.map(company_names) print(competitor_counts) ``` #### 3. Visualize results ```python import matplotlib.pyplot as plt competitor_counts.sort_values().plot(kind='barh') plt.xlabel('Job count') plt.ylabel('Company') plt.title(f'Competitor hiring in {year}') plt.show() ``` ### Use Case 3: Salary analysis **Objective:** Analyze salary distributions across job titles or other job attributes. #### 1. Download and load data ```sh YEAR=2025 curl -L -o "jobs_${YEAR}.csv.gz" "https://jobdataapi.com/download//jobs_${YEAR}.csv.gz" gzip -dc "jobs_${YEAR}.csv.gz" > "jobs_${YEAR}.csv" ``` #### 2. Analyze salary data ```python import pandas as pd year = 2025 jobs = pd.read_csv(f'jobs_{year}.csv') salary_data = jobs.dropna(subset=['salary_min', 'salary_max']).copy() salary_data['salary_min'] = pd.to_numeric(salary_data['salary_min']) salary_data['salary_max'] = pd.to_numeric(salary_data['salary_max']) salary_data['average_salary'] = ( salary_data['salary_min'] + salary_data['salary_max'] ) / 2 title_salary = ( salary_data.groupby('title')['average_salary'] .mean() .sort_values(ascending=False) ) print(title_salary.head(10)) ``` #### 3. Visualize results ```python import matplotlib.pyplot as plt title_salary.head(10).sort_values().plot(kind='barh') plt.xlabel('Average annual salary') plt.ylabel('Job title') plt.title(f'Highest average advertised salaries in {year}') plt.show() ``` For cross-currency comparisons, use the `salary_min_usd` and `salary_max_usd` fields, or use the estimated salary fields when original salary values are incomplete. ### Use Case 4: Job posting trends over time **Objective:** Compare job posting volume across months or years. Download the annual files for the period of interest. The reference `companies.csv.gz` file is not required for this analysis: ```sh for YEAR in 2023 2024 2025; do curl -L -o "jobs_${YEAR}.csv.gz" "https://jobdataapi.com/download//jobs_${YEAR}.csv.gz" gzip -dc "jobs_${YEAR}.csv.gz" > "jobs_${YEAR}.csv" done ``` Load and combine the annual files without relying on the current-year compatibility alias: ```python from pathlib import Path import pandas as pd files = sorted(Path('.').glob('jobs_20*.csv')) jobs = pd.concat( [pd.read_csv(path) for path in files], ignore_index=True, ) jobs['published'] = pd.to_datetime(jobs['published'], errors='coerce', utc=True) monthly_trends = ( jobs.dropna(subset=['published']) .groupby(jobs['published'].dt.to_period('M')) .size() ) print(monthly_trends) ``` Visualize the resulting time series: ```python import matplotlib.pyplot as plt monthly_trends.plot(kind='line') plt.xlabel('Month') plt.ylabel('Job count') plt.title('Job posting trends over time') plt.show() ``` For large analytical scans, use `jobs_companies.parquet` instead of loading multiple CSV files. It combines essential job fields with basic company data, includes a `published_year` column, and supports column-pruned reads in DuckDB and other Parquet-compatible tools. Use the CSV files when you need the established relational tables, original HTML descriptions, or the separate embedding export. These examples are starting points. Adjust the year range, filters, salary fields, and classification dimensions to match your analysis, and process annual files incrementally when local memory is limited. For API field definitions and endpoint behavior, see the relevant [API endpoint documentation](/c/docs/list/).