job-research/docs/superpowers/specs/2026-05-26-candidate-profile-ingestion-design.md

Candidate Profile Ingestion Design

This spec covers the first executable slice of the job discovery system: producing a trustworthy local Candidate Profile and saved default Search Preferences from a PDF CV, a structured markdown profile, and explicit YAML overrides. It stops before scraping, ranking, or OpenCode-driven re-ranking.

Scope

This slice must let the user:

• provide a PDF CV,
• provide a structured markdown profile with some freeform notes,
• generate deterministic local profile state,
• review and resolve factual conflicts explicitly,
• materialize a final normalized Candidate Profile,
• save default Search Preferences separately from career facts,
• inspect the resulting local state confidently.

Out of scope:

• job scraping,
• ranking,
• LLM explanations of job fit,
• session-level chat overrides,
• browser or web UI flows.

Architecture

The design uses a layered state pipeline.

Inputs:

• PDF CV for raw career history
• structured markdown profile for curated interpretation and goals
• YAML overrides for explicit user corrections

Outputs:

• generated state from deterministic extraction and merge
• override state authored by the user
• final merged normalized state for downstream consumers
• saved default Search Preferences stored separately

The key trust rule is that factual conflicts must never be resolved silently. When the PDF CV and markdown profile disagree on factual history, the system writes the disagreement into a review-required section and stops short of claiming a trustworthy final profile until explicit override data resolves it.

Components

CV Extractor

Responsibility:

• read the PDF CV,
• extract text deterministically,
• derive structured career-history candidates.

Expected fields include:

• identity basics,
• experience entries,
• education entries,
• skill mentions,
• language mentions.

Behavior:

• prefer explicit extraction over inference,
• leave uncertainty visible instead of guessing,
• avoid LLM use unless deterministic extraction is incomplete or ambiguous enough to block useful structuring.

Markdown Profile Parser

Responsibility:

• read a required structured markdown template,
• parse known sections into normalized authored signals,
• preserve some freeform notes for nuance.

Expected content includes:

• strengths,
• target roles,
• tool and stack emphasis,
• interpretation of prior experience,
• narrative clarifications not obvious from the CV.

Merge and Conflict Analyzer

Responsibility:

• combine CV-derived and markdown-derived state into generated YAML,
• apply source authority rules,
• identify factual conflicts and unresolved ambiguities.

Authority rules:

• PDF CV is the default source of truth for raw career history,
• markdown profile is the source of truth for curated interpretation and goals,
• factual disagreements are surfaced and require explicit resolution,
• missing non-critical data may remain unresolved if it is clearly marked.

Override Applier

Responsibility:

• read user-authored YAML overrides,
• resolve review-required conflicts,
• materialize the final canonical Candidate Profile.

This unit is the only place where explicit user correction can replace generated factual values.

Search Preferences Manager

Responsibility:

• validate and persist saved default Search Preferences in a separate YAML file,
• keep them separate from the durable Candidate Profile.

This slice stores only saved defaults, not temporary session overrides.

Data Flow

1. The user provides a PDF CV.
2. The user provides a structured markdown profile.
3. The system extracts CV-derived structured candidates.
4. The system parses markdown-derived authored signals.
5. The merge and conflict analyzer writes generated YAML.
6. If factual conflicts or blocking ambiguities exist, the system writes them into a review-required section and exits with review-required status.
7. The user edits an override YAML file.
8. The override applier materializes the final normalized Candidate Profile.
9. The user saves validated default Search Preferences separately.

The system should support rerunning generation without losing authored overrides.

File Layout

data/
  profiles/
    cv/
      raw-text.txt
    authored/
      profile.md
    generated/
      candidate-profile.generated.yaml
    overrides/
      candidate-profile.override.yaml
    final/
      candidate-profile.yaml
  preferences/
    search-preferences.yaml

File semantics:

• candidate-profile.generated.yaml is machine-produced and replaceable
• candidate-profile.override.yaml is user-owned
• candidate-profile.yaml is the canonical merged state for downstream consumers
• search-preferences.yaml is a separate document because Search Preferences are not durable career facts

JSON export may be supported for debugging or automation, but YAML is the primary local format.

Validation and Error Handling

Hard Failures

These should fail immediately and avoid producing a claimed final profile:

• unreadable or missing PDF input,
• malformed structured markdown profile,
• invalid YAML override format,
• invalid Search Preferences schema.

Review-Required State

These should produce generated state plus a clear review-required status:

• factual conflicts between CV and markdown profile,
• blocking ambiguity that would make the final profile untrustworthy if auto-resolved.

Allowed Ambiguity

These may remain visible without blocking final state if clearly recorded:

• incomplete optional fields,
• non-critical uncertainty that does not alter durable career facts,
• missing interpretation details that can be added later.

The system must prefer explicit unresolved ambiguity over silent incorrect certainty.

Command Surface

Proposed CLI commands for this slice:

• ingest-cv <cv.pdf>
• ingest-profile <profile.md>
• build-profile --cv <cv.pdf> --profile <profile.md>
• apply-overrides <override.yaml>
• show-profile --final
• set-preferences <preferences.yaml>
• show-preferences

Expected behavior:

• build-profile generates deterministic state and signals review-required conditions
• apply-overrides resolves explicit user corrections into final canonical state
• set-preferences validates and persists saved default Search Preferences

LLM Usage Boundaries

This slice is deterministic-first.

Allowed LLM use:

• fallback interpretation when deterministic extraction is too ambiguous to structure usefully,
• optional structuring help where the result is still auditable in generated YAML.

Disallowed LLM use:

• replacing deterministic extraction as the default path,
• silently deciding factual conflicts,
• acting as the source of truth for final persisted profile state.

Testing Strategy

The first slice should be tested around trust and repeatability.

Core test cases:

• PDF-only ingestion produces stable generated state where possible
• structured markdown parsing maps required sections correctly
• CV and markdown merge cleanly when there is no factual conflict
• factual conflict produces review-required output and no false success signal
• overrides resolve conflicts into the final canonical profile correctly
• final YAML serialization is stable and readable
• invalid overrides fail with actionable validation feedback
• Search Preferences validation and persistence behave independently from the profile pipeline

Acceptance Criteria

This slice is done when:

• a real PDF CV and structured markdown profile can produce generated YAML locally,
• factual conflicts are surfaced explicitly and never resolved silently,
• override YAML can resolve those conflicts into a final canonical Candidate Profile,
• the final profile is trustworthy enough to become the source for later ranking work,
• saved default Search Preferences are validated and stored separately from the Candidate Profile,
• the workflow is repeatable from the CLI without requiring scraping, ranking, or UI work.

Non-Goals for This Slice

This design intentionally does not solve:

• job board integration,
• listing normalization,
• score computation,
• Stretch Opportunity classification,
• dismissed listing behavior,
• OpenCode-driven session override flows.

Those belong to later slices after profile state is trustworthy.