tonio/job-research
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: simplify candidate profile ingestion design

Browse Source
This commit is contained in:
Antoine 2026-05-28 12:11:19 +02:00
parent 89434e35b4
commit 654bd2810b
1 changed files with 93 additions and 130 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

223
docs/superpowers/specs/2026-05-26-candidate-profile-ingestion-design.md
View File

@ -1,45 +1,41 @@
# Candidate Profile Ingestion Design
This spec covers the first executable slice of the job discovery system: producing a trustworthy local `Candidate Profile` and saved default `Search Preferences` from a PDF CV, a structured markdown profile, and explicit YAML overrides. It stops before scraping, ranking, or OpenCode-driven re-ranking.
This spec covers the simplified first executable slice of the job discovery system: take a PDF CV and a light-template markdown profile, then produce one usable `candidate-profile.yaml` that the user can inspect and edit directly. It stops before scraping, ranking, separate `Search Preferences`, or OpenCode-driven session overrides.
## Scope
This slice must let the user:
- provide a PDF CV,
- provide a structured markdown profile with some freeform notes,
- generate deterministic local profile state,
- review and resolve factual conflicts explicitly,
- materialize a final normalized `Candidate Profile`,
- save default `Search Preferences` separately from career facts,
- inspect the resulting local state confidently.
- provide a light structured markdown profile,
- run one main CLI command,
- produce one canonical normalized YAML profile,
- see warnings when CV facts and markdown intent disagree or when extraction is uncertain,
- edit the resulting YAML directly if they want to refine or correct it.
Out of scope:
- job scraping,
- ranking,
- LLM explanations of job fit,
- session-level chat overrides,
- separate saved `Search Preferences`,
- multi-step override workflows,
- hard-stop review gates,
- browser or web UI flows.
## Architecture
The design uses a layered state pipeline.
The design is a single-step profile normalizer.
Inputs:
- PDF CV for raw career history
- structured markdown profile for curated interpretation and goals
- YAML overrides for explicit user corrections
- PDF CV for raw career history,
- light-template markdown profile for curated interpretation, job intent, and constraints.
Outputs:
Output:
- generated state from deterministic extraction and merge
- override state authored by the user
- final merged normalized state for downstream consumers
- saved default `Search Preferences` stored separately
- one canonical `candidate-profile.yaml`.
The key trust rule is that factual conflicts must never be resolved silently. When the PDF CV and markdown profile disagree on factual history, the system writes the disagreement into a review-required section and stops short of claiming a trustworthy final profile until explicit override data resolves it.
The trust rule is simpler than the earlier layered design: the system always writes the output file, but it never hides uncertainty. When the CV and markdown profile disagree on facts, or when extraction confidence is weak, the output YAML includes explicit warnings. The user can then edit the canonical YAML directly.
## Components
@ -63,145 +59,115 @@ Behavior:
- prefer explicit extraction over inference,
- leave uncertainty visible instead of guessing,
- avoid LLM use unless deterministic extraction is incomplete or ambiguous enough to block useful structuring.
- use LLM assistance only when deterministic extraction is too weak to structure useful output.
### Markdown Profile Parser
Responsibility:
- read a required structured markdown template,
- parse known sections into normalized authored signals,
- preserve some freeform notes for nuance.
- read a light required markdown template,
- parse known sections into structured authored intent.
Expected content includes:
Required sections:
- strengths,
- target roles,
- tool and stack emphasis,
- interpretation of prior experience,
- narrative clarifications not obvious from the CV.
- `Summary`
- `Target Roles`
- `Strengths`
- `Skills To Emphasize`
- `Constraints`
- `Notes`
### Merge and Conflict Analyzer
This file carries both durable interpretation and current job intent for the first slice.
### Profile Merger
Responsibility:
- combine CV-derived and markdown-derived state into generated YAML,
- apply source authority rules,
- identify factual conflicts and unresolved ambiguities.
- combine CV-derived facts and markdown-derived intent,
- normalize them into one canonical profile document,
- record warnings for factual conflicts or ambiguity.
Authority rules:
- PDF CV is the default source of truth for raw career history,
- markdown profile is the source of truth for curated interpretation and goals,
- factual disagreements are surfaced and require explicit resolution,
- missing non-critical data may remain unresolved if it is clearly marked.
### Override Applier
Responsibility:
- read user-authored YAML overrides,
- resolve review-required conflicts,
- materialize the final canonical `Candidate Profile`.
This unit is the only place where explicit user correction can replace generated factual values.
### Search Preferences Manager
Responsibility:
- validate and persist saved default `Search Preferences` in a separate YAML file,
- keep them separate from the durable `Candidate Profile`.
This slice stores only saved defaults, not temporary session overrides.
- CV is the default source for raw career history,
- markdown profile is the source for interpretation, emphasis, and job intent,
- factual disagreement becomes a warning, not a blocked workflow,
- the final YAML is user-editable and becomes the local source of truth after generation.
## Data Flow
1. The user provides a PDF CV.
2. The user provides a structured markdown profile.
2. The user provides a light-template markdown profile.
3. The system extracts CV-derived structured candidates.
4. The system parses markdown-derived authored signals.
5. The merge and conflict analyzer writes generated YAML.
6. If factual conflicts or blocking ambiguities exist, the system writes them into a review-required section and exits with review-required status.
7. The user edits an override YAML file.
8. The override applier materializes the final normalized `Candidate Profile`.
9. The user saves validated default `Search Preferences` separately.
The system should support rerunning generation without losing authored overrides.
4. The system parses markdown-derived authored intent.
5. The system merges both into one normalized profile document.
6. The system writes `candidate-profile.yaml`.
7. The system includes warnings inside the YAML when facts conflict or extraction is uncertain.
8. The user edits `candidate-profile.yaml` directly if they want to refine the final result.
## File Layout
```text
data/
profiles/
cv/
raw-text.txt
authored/
profile.md
generated/
candidate-profile.generated.yaml
overrides/
candidate-profile.override.yaml
final/
candidate-profile.yaml
preferences/
search-preferences.yaml
candidate-profile.yaml
```
File semantics:
Optional debug artifacts may be added later, but the first slice should optimize for a single obvious output file.
- `candidate-profile.generated.yaml` is machine-produced and replaceable
- `candidate-profile.override.yaml` is user-owned
- `candidate-profile.yaml` is the canonical merged state for downstream consumers
- `search-preferences.yaml` is a separate document because `Search Preferences` are not durable career facts
## Output Shape
JSON export may be supported for debugging or automation, but YAML is the primary local format.
The canonical YAML should include:
- identity and summary,
- target roles and job intent,
- skills and technologies,
- experience entries,
- education entries,
- constraints,
- notes,
- warnings.
The warnings section should be human-readable and should point out:
- factual conflicts,
- missing important fields,
- low-confidence extraction areas.
## Validation and Error Handling
### Hard Failures
These should fail immediately and avoid producing a claimed final profile:
These should fail immediately and avoid producing misleading output:
- unreadable or missing PDF input,
- malformed structured markdown profile,
- invalid YAML override format,
- invalid `Search Preferences` schema.
- malformed markdown template missing required headings,
- invalid command arguments.
### Review-Required State
### Non-Blocking Warnings
These should produce generated state plus a clear review-required status:
These should still produce `candidate-profile.yaml`:
- factual conflicts between CV and markdown profile,
- blocking ambiguity that would make the final profile untrustworthy if auto-resolved.
### Allowed Ambiguity
These may remain visible without blocking final state if clearly recorded:
- incomplete optional fields,
- non-critical uncertainty that does not alter durable career facts,
- missing interpretation details that can be added later.
- low-confidence extraction,
- unclear dates or inferred experience totals.
The system must prefer explicit unresolved ambiguity over silent incorrect certainty.
The system must prefer explicit warnings over silent incorrect certainty.
## Command Surface
Proposed CLI commands for this slice:
The first slice should expose one main command:
- `ingest-cv <cv.pdf>`
- `ingest-profile <profile.md>`
- `build-profile --cv <cv.pdf> --profile <profile.md>`
- `apply-overrides <override.yaml>`
- `show-profile --final`
- `set-preferences <preferences.yaml>`
- `show-preferences`
- `build-profile --cv <cv.pdf> --profile <profile.md> --out data/candidate-profile.yaml`
Optional helper commands may be added later, but they are not required for this slice.
Expected behavior:
- `build-profile` generates deterministic state and signals review-required conditions
- `apply-overrides` resolves explicit user corrections into final canonical state
- `set-preferences` validates and persists saved default `Search Preferences`
- read both inputs,
- produce one canonical YAML file,
- print a success message,
- print whether warnings were included.
## LLM Usage Boundaries
@ -209,40 +175,36 @@ This slice is deterministic-first.
Allowed LLM use:
- fallback interpretation when deterministic extraction is too ambiguous to structure usefully,
- optional structuring help where the result is still auditable in generated YAML.
- fallback structuring help when deterministic extraction alone cannot produce a usable normalized profile.
Disallowed LLM use:
- replacing deterministic extraction as the default path,
- silently deciding factual conflicts,
- acting as the source of truth for final persisted profile state.
- acting as the only source of truth for the final persisted profile.
## Testing Strategy
The first slice should be tested around trust and repeatability.
The first slice should be tested around usefulness and transparency.
Core test cases:
- PDF-only ingestion produces stable generated state where possible
- structured markdown parsing maps required sections correctly
- CV and markdown merge cleanly when there is no factual conflict
- factual conflict produces review-required output and no false success signal
- overrides resolve conflicts into the final canonical profile correctly
- final YAML serialization is stable and readable
- invalid overrides fail with actionable validation feedback
- `Search Preferences` validation and persistence behave independently from the profile pipeline
- valid CV plus valid markdown produces `candidate-profile.yaml`,
- missing required markdown sections fails clearly,
- factual conflict still writes output and records warnings,
- final YAML contains both extracted facts and markdown intent,
- output serialization is stable and readable,
- command-line usage is repeatable.
## Acceptance Criteria
This slice is done when:
- a real PDF CV and structured markdown profile can produce generated YAML locally,
- factual conflicts are surfaced explicitly and never resolved silently,
- override YAML can resolve those conflicts into a final canonical `Candidate Profile`,
- the final profile is trustworthy enough to become the source for later ranking work,
- saved default `Search Preferences` are validated and stored separately from the `Candidate Profile`,
- the workflow is repeatable from the CLI without requiring scraping, ranking, or UI work.
- a real PDF CV and light-template markdown profile can produce one local `candidate-profile.yaml`,
- the YAML is readable and useful enough for the user to trust and edit directly,
- factual conflicts and uncertainty appear as warnings rather than hidden behavior,
- the workflow is one obvious command rather than a multi-step state machine,
- the result is good enough to become the input to later scraping and ranking work.
## Non-Goals for This Slice
@ -253,6 +215,7 @@ This design intentionally does not solve:
- score computation,
- `Stretch Opportunity` classification,
- dismissed listing behavior,
- OpenCode-driven session override flows.
- separate persisted `Search Preferences`,
- session-level conversational overrides.
Those belong to later slices after profile state is trustworthy.
Those belong to later slices after the system can reliably understand the candidate.