commit 08df32da53cdd7f6d96330776a71a9694bdee8fd Author: Antoine Date: Tue May 26 23:10:04 2026 +0200 docs: add candidate profile ingestion design diff --git a/docs/superpowers/specs/2026-05-26-candidate-profile-ingestion-design.md b/docs/superpowers/specs/2026-05-26-candidate-profile-ingestion-design.md new file mode 100644 index 0000000..4f91f01 --- /dev/null +++ b/docs/superpowers/specs/2026-05-26-candidate-profile-ingestion-design.md @@ -0,0 +1,258 @@ +# Candidate Profile Ingestion Design + +This spec covers the first executable slice of the job discovery system: producing a trustworthy local `Candidate Profile` and saved default `Search Preferences` from a PDF CV, a structured markdown profile, and explicit YAML overrides. It stops before scraping, ranking, or OpenCode-driven re-ranking. + +## Scope + +This slice must let the user: + +- provide a PDF CV, +- provide a structured markdown profile with some freeform notes, +- generate deterministic local profile state, +- review and resolve factual conflicts explicitly, +- materialize a final normalized `Candidate Profile`, +- save default `Search Preferences` separately from career facts, +- inspect the resulting local state confidently. + +Out of scope: + +- job scraping, +- ranking, +- LLM explanations of job fit, +- session-level chat overrides, +- browser or web UI flows. + +## Architecture + +The design uses a layered state pipeline. + +Inputs: + +- PDF CV for raw career history +- structured markdown profile for curated interpretation and goals +- YAML overrides for explicit user corrections + +Outputs: + +- generated state from deterministic extraction and merge +- override state authored by the user +- final merged normalized state for downstream consumers +- saved default `Search Preferences` stored separately + +The key trust rule is that factual conflicts must never be resolved silently. When the PDF CV and markdown profile disagree on factual history, the system writes the disagreement into a review-required section and stops short of claiming a trustworthy final profile until explicit override data resolves it. + +## Components + +### CV Extractor + +Responsibility: + +- read the PDF CV, +- extract text deterministically, +- derive structured career-history candidates. + +Expected fields include: + +- identity basics, +- experience entries, +- education entries, +- skill mentions, +- language mentions. + +Behavior: + +- prefer explicit extraction over inference, +- leave uncertainty visible instead of guessing, +- avoid LLM use unless deterministic extraction is incomplete or ambiguous enough to block useful structuring. + +### Markdown Profile Parser + +Responsibility: + +- read a required structured markdown template, +- parse known sections into normalized authored signals, +- preserve some freeform notes for nuance. + +Expected content includes: + +- strengths, +- target roles, +- tool and stack emphasis, +- interpretation of prior experience, +- narrative clarifications not obvious from the CV. + +### Merge and Conflict Analyzer + +Responsibility: + +- combine CV-derived and markdown-derived state into generated YAML, +- apply source authority rules, +- identify factual conflicts and unresolved ambiguities. + +Authority rules: + +- PDF CV is the default source of truth for raw career history, +- markdown profile is the source of truth for curated interpretation and goals, +- factual disagreements are surfaced and require explicit resolution, +- missing non-critical data may remain unresolved if it is clearly marked. + +### Override Applier + +Responsibility: + +- read user-authored YAML overrides, +- resolve review-required conflicts, +- materialize the final canonical `Candidate Profile`. + +This unit is the only place where explicit user correction can replace generated factual values. + +### Search Preferences Manager + +Responsibility: + +- validate and persist saved default `Search Preferences` in a separate YAML file, +- keep them separate from the durable `Candidate Profile`. + +This slice stores only saved defaults, not temporary session overrides. + +## Data Flow + +1. The user provides a PDF CV. +2. The user provides a structured markdown profile. +3. The system extracts CV-derived structured candidates. +4. The system parses markdown-derived authored signals. +5. The merge and conflict analyzer writes generated YAML. +6. If factual conflicts or blocking ambiguities exist, the system writes them into a review-required section and exits with review-required status. +7. The user edits an override YAML file. +8. The override applier materializes the final normalized `Candidate Profile`. +9. The user saves validated default `Search Preferences` separately. + +The system should support rerunning generation without losing authored overrides. + +## File Layout + +```text +data/ + profiles/ + cv/ + raw-text.txt + authored/ + profile.md + generated/ + candidate-profile.generated.yaml + overrides/ + candidate-profile.override.yaml + final/ + candidate-profile.yaml + preferences/ + search-preferences.yaml +``` + +File semantics: + +- `candidate-profile.generated.yaml` is machine-produced and replaceable +- `candidate-profile.override.yaml` is user-owned +- `candidate-profile.yaml` is the canonical merged state for downstream consumers +- `search-preferences.yaml` is a separate document because `Search Preferences` are not durable career facts + +JSON export may be supported for debugging or automation, but YAML is the primary local format. + +## Validation and Error Handling + +### Hard Failures + +These should fail immediately and avoid producing a claimed final profile: + +- unreadable or missing PDF input, +- malformed structured markdown profile, +- invalid YAML override format, +- invalid `Search Preferences` schema. + +### Review-Required State + +These should produce generated state plus a clear review-required status: + +- factual conflicts between CV and markdown profile, +- blocking ambiguity that would make the final profile untrustworthy if auto-resolved. + +### Allowed Ambiguity + +These may remain visible without blocking final state if clearly recorded: + +- incomplete optional fields, +- non-critical uncertainty that does not alter durable career facts, +- missing interpretation details that can be added later. + +The system must prefer explicit unresolved ambiguity over silent incorrect certainty. + +## Command Surface + +Proposed CLI commands for this slice: + +- `ingest-cv ` +- `ingest-profile ` +- `build-profile --cv --profile ` +- `apply-overrides ` +- `show-profile --final` +- `set-preferences ` +- `show-preferences` + +Expected behavior: + +- `build-profile` generates deterministic state and signals review-required conditions +- `apply-overrides` resolves explicit user corrections into final canonical state +- `set-preferences` validates and persists saved default `Search Preferences` + +## LLM Usage Boundaries + +This slice is deterministic-first. + +Allowed LLM use: + +- fallback interpretation when deterministic extraction is too ambiguous to structure usefully, +- optional structuring help where the result is still auditable in generated YAML. + +Disallowed LLM use: + +- replacing deterministic extraction as the default path, +- silently deciding factual conflicts, +- acting as the source of truth for final persisted profile state. + +## Testing Strategy + +The first slice should be tested around trust and repeatability. + +Core test cases: + +- PDF-only ingestion produces stable generated state where possible +- structured markdown parsing maps required sections correctly +- CV and markdown merge cleanly when there is no factual conflict +- factual conflict produces review-required output and no false success signal +- overrides resolve conflicts into the final canonical profile correctly +- final YAML serialization is stable and readable +- invalid overrides fail with actionable validation feedback +- `Search Preferences` validation and persistence behave independently from the profile pipeline + +## Acceptance Criteria + +This slice is done when: + +- a real PDF CV and structured markdown profile can produce generated YAML locally, +- factual conflicts are surfaced explicitly and never resolved silently, +- override YAML can resolve those conflicts into a final canonical `Candidate Profile`, +- the final profile is trustworthy enough to become the source for later ranking work, +- saved default `Search Preferences` are validated and stored separately from the `Candidate Profile`, +- the workflow is repeatable from the CLI without requiring scraping, ranking, or UI work. + +## Non-Goals for This Slice + +This design intentionally does not solve: + +- job board integration, +- listing normalization, +- score computation, +- `Stretch Opportunity` classification, +- dismissed listing behavior, +- OpenCode-driven session override flows. + +Those belong to later slices after profile state is trustworthy.