Upwork job scraper

This workflow automatically scrapes Upwork job listings, filters them for relevance using AI, and organizes qualified leads in Airtable with personalized icebreaker messages. It eliminates manual job hunting by intelligently analyzing job descriptions against your service criteria and generating ready-to-use proposal intros.

Who is it for?

This template is perfect for:

  • Freelance Service Providers on Upwork
  • Agencies looking for new clients on Upwork
  • Automation specialists, CRM consultants, or no-code developers
  • Anyone who wants to automate their Upwork job discovery and qualification process

What it does

  1. Scrape Upwork Search Results: Fetches job listings from a specific Upwork search URL (searches for automation jobs with $20+ hourly rate, top-tier contractors).

  2. Extract Job Links: Uses regex to parse all job listing URLs from the search results page.

  3. Set Job URL Variable: Constructs the full Upwork job URL for each listing.

  4. Sleep 2 Seconds: Adds a delay between requests to avoid rate limiting.

  5. Fetch Individual Job Page: Retrieves the full HTML for each job posting.

  6. Extract Job Title: Parses the page title using regex to get job title and metadata.

  7. Extract Job Description: Parses the meta description tag to capture the job description.

  8. Parse Title Components: Splits the title to extract job type, time commitment, and contract type.

  9. AI Job Filtering: Uses ChatGPT with a detailed prompt and few-shot examples to:

    • Filter jobs for relevance based on your service offerings (CRM systems, automation, no-code tools like Airtable, ClickUp, Make, Zapier, Monday.com)
    • Exclude irrelevant jobs (full-stack dev, Python, chatbots, Telegram, WhatsApp, virtual assistant work, etc.)
    • Generate a personalized icebreaker message for relevant jobs

  10. Parse AI Response: Extracts the JSON result containing the relevance flag, reason, and icebreaker.

  11. Search Existing Jobs: Checks Airtable to see if this job has already been logged (prevents duplicates).

  12. Add New Job to Airtable: Creates a new record for relevant, non-duplicate jobs with:

  13. Job title and description

  14. Status set to "Intake"

  15. AI-generated icebreaker introduction

  16. Static proposal template

  17. Job URL

  18. Date posted

How to set up

  1. Upwork Search URL:

    • Update the URL in the first "GET Upwork Jobs" module with your custom Upwork search criteria.

  2. OpenAI Connection:

  3. Connect your OpenAI account in the "Filter Job + Icebreaker" module.

  4. Airtable Connection:

  5. Connect your Airtable account in both the "Search Existing Job" and "Add New Job" modules.

  6. Update the Base ID to your Airtable base.

  7. Update the Table ID to your jobs table.

  8. Ensure your Airtable table has these fields: Title, Description, Status, Date Posted, Introduction, Template, Link, Type.

  9. Duplicate Check Formula:

  10. Update the search formula in "Search Existing Job" module to match against your job title field.

← Previous
Next →