Skip to content
Ahmad Humayun
← All case studies

Public Data Enrichment

SAM.gov Opportunities Enrichment Pipeline

Python pipeline for collecting public opportunity records and enriching them with PSC/NAICS-style classification context for reporting.

Public Data Workflow · Solo · Selected work

My role & contribution

Engineering work for this individual freelance engagement.

The problem

Built a public-data enrichment pipeline that collected opportunity records and added classification context so the resulting dataset was easier to search, classify, and report on. The work includes extraction, description/detail helpers, PSC/NAICS-style processing, and BigQuery-oriented loading patterns.

SAM.gov Opportunities Enrichment Pipeline
Illustrative project visual. Any figures shown are examples, not a statement of client results.

System flow

  1. 01

    Collection

    Python scripts collect public opportunity records and detail pages from source endpoints/pages.

  2. 02

    Enrichment

    PSC and NAICS-style taxonomy helpers add classification context for search and reporting.

  3. 03

    Loading

    Cleaned opportunity records are prepared for BigQuery-style destination tables.

  4. 04

    Validation

    Counts, duplicates, taxonomy coverage, and destination rows are checked after refreshes.

Difficult problems

  • Public records needed enrichment before they were useful for search and reporting
  • Classification joins could miss records or create stale categories
  • Duplicate records and changing descriptions needed validation
  • Filters and destination details needed clear ownership and validation

Work contributed

  • Built extraction and enrichment scripts for opportunity-style records
  • Added PSC/NAICS-style classification helpers
  • Prepared output for a warehouse or dashboard destination
  • Defined validation around counts, taxonomy coverage, duplicates, and destination rows

Supported outcome

  • Built extraction and enrichment scripts for opportunity-style records
  • Added PSC/NAICS-style classification helpers
  • Prepared output for a warehouse or dashboard destination
  • Defined validation around counts, taxonomy coverage, duplicates, and destination rows

Technologies in context

Python · pandas · requests · BeautifulSoup · BigQuery · JSON · Classification Data

Related services & work

Technical reading

    Working through a similar problem?

    Share the workflow and the output your team needs. We can identify a focused starting point.

    Discuss your workflow