Compare commits
68 Commits
89434e35b4
...
b4182c9686
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b4182c9686 | ||
|
|
e9db6b48d9 | ||
|
|
85c4278db3 | ||
|
|
ed1af201bc | ||
|
|
fa2000abf0 | ||
|
|
c218a9040e | ||
|
|
1dfaea3508 | ||
|
|
53b4ac0ea3 | ||
|
|
a998f1d968 | ||
|
|
40b3619578 | ||
|
|
cdae23e303 | ||
|
|
3768bf9b3c | ||
|
|
207d5c51c3 | ||
|
|
352dfcd6ce | ||
|
|
31c0488f60 | ||
|
|
86b5cf2d7b | ||
|
|
450134c808 | ||
|
|
ce73787f39 | ||
|
|
c28f804e23 | ||
|
|
47f912ce8c | ||
|
|
ac1e0fb269 | ||
|
|
c2ca1f457e | ||
|
|
1c267520f0 | ||
|
|
2da39613c7 | ||
|
|
52b21b65c5 | ||
|
|
97ed98c5de | ||
|
|
47619abc08 | ||
|
|
5fefa3aab1 | ||
|
|
debaab0947 | ||
|
|
96b4cd37ec | ||
|
|
96a7dcbe94 | ||
|
|
95b17a7c50 | ||
|
|
00f3717995 | ||
|
|
96e2a7b178 | ||
|
|
e86257cbcc | ||
|
|
ac78bee74b | ||
|
|
4d57af0b86 | ||
|
|
2765add571 | ||
|
|
80f308356a | ||
|
|
cfbd1943ec | ||
|
|
ad36de0a3f | ||
|
|
fb150dbe69 | ||
|
|
0f6e641a99 | ||
|
|
c4b9fc13ce | ||
|
|
8d232060a0 | ||
|
|
d27c58014d | ||
|
|
1879de68a8 | ||
|
|
3c331ef687 | ||
|
|
042feab4fd | ||
|
|
49cc4a9959 | ||
|
|
19a1731349 | ||
|
|
77d2f0f5c6 | ||
|
|
a4c1ff3c0b | ||
|
|
0eac9856da | ||
|
|
fe201831ba | ||
|
|
c57d9b0275 | ||
|
|
544b5892fa | ||
|
|
df140c3c77 | ||
|
|
ddd3c9d78a | ||
|
|
17800a5f07 | ||
|
|
0ce4ca0ee5 | ||
|
|
1b4e901afe | ||
|
|
a0522c316e | ||
|
|
ac9616080b | ||
|
|
4b2ce3b465 | ||
|
|
e171228626 | ||
|
|
309aa2f4be | ||
|
|
654bd2810b |
3
.gitignore
vendored
3
.gitignore
vendored
@ -8,3 +8,6 @@ wheels/
|
||||
|
||||
# Virtual environments
|
||||
.venv
|
||||
|
||||
# Local git worktrees
|
||||
.worktrees/
|
||||
|
||||
22
docs/profile-template.md
Normal file
22
docs/profile-template.md
Normal file
@ -0,0 +1,22 @@
|
||||
# Candidate Profile
|
||||
|
||||
## Summary
|
||||
One short paragraph describing your target profile.
|
||||
|
||||
## Target Roles
|
||||
- Data Engineer
|
||||
|
||||
## Strengths
|
||||
- Python
|
||||
- SQL
|
||||
|
||||
## Skills To Emphasize
|
||||
- GCP
|
||||
- BigQuery
|
||||
|
||||
## Constraints
|
||||
- CDI only
|
||||
- France only
|
||||
|
||||
## Notes
|
||||
- Anything the CV parser might miss but future ranking should understand.
|
||||
File diff suppressed because it is too large
Load Diff
612
docs/superpowers/plans/2026-06-01-apec-ingestion.md
Normal file
612
docs/superpowers/plans/2026-06-01-apec-ingestion.md
Normal file
@ -0,0 +1,612 @@
|
||||
# Apec Ingestion Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build one `fetch-apec` command that reads `data/candidate-profile.yaml`, derives deterministic Apec searches, fetches up to 50 public listings, stores raw HTML snapshots, and writes a normalized `listings.yaml` file plus run metadata.
|
||||
|
||||
**Architecture:** The implementation is a fetch-and-normalize pipeline with explicit artifacts by run. Profile-driven query derivation feeds an Apec adapter, successful detail pages are persisted as raw snapshots, and a small normalizer plus within-run deduper writes inspectable YAML outputs for later ranking work.
|
||||
|
||||
**Tech Stack:** Python 3.13, Typer, Playwright for Python, BeautifulSoup4, Pydantic v2, PyYAML, pytest
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
- Modify: `pyproject.toml` — add Playwright dependency if missing
|
||||
- Modify: `src/job_research/cli.py` — add `fetch-apec` command
|
||||
- Modify: `src/job_research/models.py` — add normalized listing and run metadata models
|
||||
- Modify: `src/job_research/storage.py` — add helpers for per-run artifact paths and YAML writes
|
||||
- Create: `src/job_research/apec/__init__.py` — Apec package marker
|
||||
- Create: `src/job_research/apec/query_derivation.py` — deterministic query derivation from candidate profile
|
||||
- Create: `src/job_research/apec/adapter.py` — public Apec search and detail-page fetching
|
||||
- Create: `src/job_research/apec/normalize.py` — normalize Apec detail pages into listing records
|
||||
- Create: `src/job_research/apec/dedupe.py` — minimal within-run deduplication
|
||||
- Create: `tests/apec/test_query_derivation.py` — profile-driven query tests
|
||||
- Create: `tests/apec/test_normalize.py` — normalized listing extraction tests
|
||||
- Create: `tests/apec/test_dedupe.py` — within-run dedupe tests
|
||||
- Create: `tests/test_apec_cli.py` — CLI integration tests for `fetch-apec`
|
||||
- Create: `tests/test_apec_storage.py` — run artifact persistence tests
|
||||
|
||||
## Task 0: Dependencies for Apec Ingestion
|
||||
|
||||
**Files:**
|
||||
- Modify: `pyproject.toml`
|
||||
|
||||
- [ ] **Step 1: Write the failing import check**
|
||||
|
||||
Run: `uv run python -c "import playwright, bs4"`
|
||||
Expected: FAIL with missing dependency errors
|
||||
|
||||
- [ ] **Step 2: Add the minimal dependencies for this slice**
|
||||
|
||||
```toml
|
||||
# pyproject.toml
|
||||
[project]
|
||||
dependencies = [
|
||||
"beautifulsoup4>=4.12,<5",
|
||||
"playwright>=1.52,<2",
|
||||
"pydantic>=2.7,<3",
|
||||
"pypdf>=5.0,<6",
|
||||
"pyyaml>=6.0,<7",
|
||||
"typer>=0.12,<1",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Sync and verify the imports work**
|
||||
|
||||
Run: `uv sync && uv run python -c "import playwright, bs4"`
|
||||
Expected: PASS with no output
|
||||
|
||||
- [ ] **Step 4: Commit the dependency update**
|
||||
|
||||
```bash
|
||||
git add pyproject.toml uv.lock
|
||||
git commit -m "chore: add Apec ingestion dependencies"
|
||||
```
|
||||
|
||||
## Task 1: Listing and Run Artifact Models
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/job_research/models.py`
|
||||
- Create: `tests/test_apec_storage.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing model serialization test**
|
||||
|
||||
```python
|
||||
# tests/test_apec_storage.py
|
||||
from job_research.models import ApecListing, ApecRunMeta, ListingWarning
|
||||
|
||||
|
||||
def test_apec_models_serialize_expected_listing_shape() -> None:
|
||||
listing = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="123",
|
||||
url="https://example.test/job/123",
|
||||
title="Data Engineer",
|
||||
company="Example",
|
||||
location="Paris",
|
||||
contract_type="CDI",
|
||||
description_text="Build pipelines",
|
||||
published_at="2026-06-01",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
warnings=[ListingWarning(field="location", message="Location inferred from page text")],
|
||||
)
|
||||
run_meta = ApecRunMeta(
|
||||
derived_queries=["Data Engineer"],
|
||||
fetched_count=1,
|
||||
normalized_count=1,
|
||||
deduplicated_count=1,
|
||||
failed_count=0,
|
||||
listing_errors=[],
|
||||
)
|
||||
|
||||
assert listing.model_dump()["source"] == "apec"
|
||||
assert run_meta.model_dump()["derived_queries"] == ["Data Engineer"]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the model test to verify it fails**
|
||||
|
||||
Run: `uv run pytest tests/test_apec_storage.py::test_apec_models_serialize_expected_listing_shape -v`
|
||||
Expected: FAIL with `ImportError` or `AttributeError` for missing Apec models
|
||||
|
||||
- [ ] **Step 3: Add normalized listing and run metadata models**
|
||||
|
||||
```python
|
||||
# src/job_research/models.py
|
||||
class ListingWarning(BaseModel):
|
||||
field: str
|
||||
message: str
|
||||
|
||||
|
||||
class ListingError(BaseModel):
|
||||
url: str
|
||||
stage: str
|
||||
message: str
|
||||
|
||||
|
||||
class ApecListing(BaseModel):
|
||||
source: str
|
||||
source_job_id: str | None = None
|
||||
url: str
|
||||
title: str | None = None
|
||||
company: str | None = None
|
||||
location: str | None = None
|
||||
contract_type: str | None = None
|
||||
description_text: str | None = None
|
||||
published_at: str | None = None
|
||||
fetched_at: str
|
||||
warnings: list[ListingWarning] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ApecRunMeta(BaseModel):
|
||||
derived_queries: list[str] = Field(default_factory=list)
|
||||
fetched_count: int = 0
|
||||
normalized_count: int = 0
|
||||
deduplicated_count: int = 0
|
||||
failed_count: int = 0
|
||||
listing_errors: list[ListingError] = Field(default_factory=list)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the model test to verify it passes**
|
||||
|
||||
Run: `uv run pytest tests/test_apec_storage.py::test_apec_models_serialize_expected_listing_shape -v`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit the models**
|
||||
|
||||
```bash
|
||||
git add src/job_research/models.py tests/test_apec_storage.py
|
||||
git commit -m "feat: add Apec listing artifact models"
|
||||
```
|
||||
|
||||
## Task 2: Run Artifact Storage Layout
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/job_research/storage.py`
|
||||
- Modify: `tests/test_apec_storage.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing run-path test**
|
||||
|
||||
```python
|
||||
# tests/test_apec_storage.py
|
||||
from pathlib import Path
|
||||
|
||||
from job_research.storage import apec_run_paths
|
||||
|
||||
|
||||
def test_apec_run_paths_builds_expected_layout(tmp_path: Path) -> None:
|
||||
paths = apec_run_paths(tmp_path, run_id="2026-06-01T10-00-00Z")
|
||||
|
||||
assert paths["run_dir"] == tmp_path / "apec" / "runs" / "2026-06-01T10-00-00Z"
|
||||
assert paths["listings"] == tmp_path / "apec" / "runs" / "2026-06-01T10-00-00Z" / "listings.yaml"
|
||||
assert paths["run_meta"] == tmp_path / "apec" / "runs" / "2026-06-01T10-00-00Z" / "run-meta.yaml"
|
||||
assert paths["snapshots"] == tmp_path / "apec" / "runs" / "2026-06-01T10-00-00Z" / "snapshots"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the run-path test to verify it fails**
|
||||
|
||||
Run: `uv run pytest tests/test_apec_storage.py::test_apec_run_paths_builds_expected_layout -v`
|
||||
Expected: FAIL because `apec_run_paths` does not exist yet
|
||||
|
||||
- [ ] **Step 3: Implement run-path helpers and artifact writes**
|
||||
|
||||
```python
|
||||
# src/job_research/storage.py
|
||||
def apec_run_paths(data_root: Path, run_id: str) -> dict[str, Path]:
|
||||
run_dir = data_root / "apec" / "runs" / run_id
|
||||
return {
|
||||
"run_dir": run_dir,
|
||||
"listings": run_dir / "listings.yaml",
|
||||
"run_meta": run_dir / "run-meta.yaml",
|
||||
"snapshots": run_dir / "snapshots",
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the run-path test to verify it passes**
|
||||
|
||||
Run: `uv run pytest tests/test_apec_storage.py::test_apec_run_paths_builds_expected_layout -v`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit the storage layout helper**
|
||||
|
||||
```bash
|
||||
git add src/job_research/storage.py tests/test_apec_storage.py
|
||||
git commit -m "feat: add Apec run artifact paths"
|
||||
```
|
||||
|
||||
## Task 3: Deterministic Query Derivation
|
||||
|
||||
**Files:**
|
||||
- Create: `src/job_research/apec/__init__.py`
|
||||
- Create: `src/job_research/apec/query_derivation.py`
|
||||
- Create: `tests/apec/test_query_derivation.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing query derivation test**
|
||||
|
||||
```python
|
||||
# tests/apec/test_query_derivation.py
|
||||
from job_research.apec.query_derivation import derive_apec_queries
|
||||
from job_research.models import CandidateProfileOutput
|
||||
|
||||
|
||||
def test_derive_apec_queries_from_candidate_profile() -> None:
|
||||
profile = CandidateProfileOutput(
|
||||
target_roles=["Data Engineer", "Analytics Engineer"],
|
||||
strengths=["Python", "SQL"],
|
||||
skills_to_emphasize=["BigQuery", "GCP"],
|
||||
constraints=["CDI only", "France only"],
|
||||
)
|
||||
|
||||
queries = derive_apec_queries(profile)
|
||||
|
||||
assert "Data Engineer" in queries
|
||||
assert "Analytics Engineer" in queries
|
||||
assert len(queries) <= 5
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the query test to verify it fails**
|
||||
|
||||
Run: `uv run pytest tests/apec/test_query_derivation.py::test_derive_apec_queries_from_candidate_profile -v`
|
||||
Expected: FAIL with missing module or function
|
||||
|
||||
- [ ] **Step 3: Implement deterministic query derivation**
|
||||
|
||||
```python
|
||||
# src/job_research/apec/query_derivation.py
|
||||
from job_research.models import CandidateProfileOutput
|
||||
|
||||
|
||||
def derive_apec_queries(profile: CandidateProfileOutput) -> list[str]:
|
||||
queries: list[str] = []
|
||||
for title in profile.target_roles:
|
||||
if title not in queries:
|
||||
queries.append(title)
|
||||
return queries[:5]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the query test to verify it passes**
|
||||
|
||||
Run: `uv run pytest tests/apec/test_query_derivation.py::test_derive_apec_queries_from_candidate_profile -v`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit query derivation**
|
||||
|
||||
```bash
|
||||
git add src/job_research/apec/__init__.py src/job_research/apec/query_derivation.py tests/apec/test_query_derivation.py
|
||||
git commit -m "feat: add deterministic Apec query derivation"
|
||||
```
|
||||
|
||||
## Task 4: Listing Normalization and Within-Run Deduplication
|
||||
|
||||
**Files:**
|
||||
- Create: `src/job_research/apec/normalize.py`
|
||||
- Create: `src/job_research/apec/dedupe.py`
|
||||
- Create: `tests/apec/test_normalize.py`
|
||||
- Create: `tests/apec/test_dedupe.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing normalization and dedupe tests**
|
||||
|
||||
```python
|
||||
# tests/apec/test_normalize.py
|
||||
from job_research.apec.normalize import normalize_apec_listing
|
||||
|
||||
|
||||
def test_normalize_apec_listing_extracts_minimal_shape() -> None:
|
||||
html = """
|
||||
<html>
|
||||
<body>
|
||||
<h1>Data Engineer</h1>
|
||||
<div class="company">Example Corp</div>
|
||||
<div class="location">Paris</div>
|
||||
<div class="contract">CDI</div>
|
||||
<div class="description">Build pipelines</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
listing = normalize_apec_listing(url="https://example.test/job/123", html=html, fetched_at="2026-06-01T10:00:00Z")
|
||||
|
||||
assert listing.title == "Data Engineer"
|
||||
assert listing.company == "Example Corp"
|
||||
assert listing.contract_type == "CDI"
|
||||
```
|
||||
|
||||
```python
|
||||
# tests/apec/test_dedupe.py
|
||||
from job_research.apec.dedupe import dedupe_apec_listings
|
||||
from job_research.models import ApecListing
|
||||
|
||||
|
||||
def test_dedupe_apec_listings_by_url() -> None:
|
||||
listings = [
|
||||
ApecListing(source="apec", url="https://example.test/job/1", fetched_at="2026-06-01T10:00:00Z"),
|
||||
ApecListing(source="apec", url="https://example.test/job/1", fetched_at="2026-06-01T10:01:00Z"),
|
||||
]
|
||||
|
||||
deduped = dedupe_apec_listings(listings)
|
||||
|
||||
assert len(deduped) == 1
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the normalization and dedupe tests to verify they fail**
|
||||
|
||||
Run: `uv run pytest tests/apec/test_normalize.py tests/apec/test_dedupe.py -v`
|
||||
Expected: FAIL with missing modules/functions
|
||||
|
||||
- [ ] **Step 3: Implement minimal normalization and dedupe**
|
||||
|
||||
```python
|
||||
# src/job_research/apec/normalize.py
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
from job_research.models import ApecListing
|
||||
|
||||
|
||||
def normalize_apec_listing(url: str, html: str, fetched_at: str) -> ApecListing:
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
title = soup.find("h1")
|
||||
company = soup.select_one(".company")
|
||||
location = soup.select_one(".location")
|
||||
contract = soup.select_one(".contract")
|
||||
description = soup.select_one(".description")
|
||||
|
||||
return ApecListing(
|
||||
source="apec",
|
||||
url=url,
|
||||
title=title.get_text(strip=True) if title else None,
|
||||
company=company.get_text(strip=True) if company else None,
|
||||
location=location.get_text(strip=True) if location else None,
|
||||
contract_type=contract.get_text(strip=True) if contract else None,
|
||||
description_text=description.get_text(" ", strip=True) if description else None,
|
||||
fetched_at=fetched_at,
|
||||
)
|
||||
```
|
||||
|
||||
```python
|
||||
# src/job_research/apec/dedupe.py
|
||||
from job_research.models import ApecListing
|
||||
|
||||
|
||||
def dedupe_apec_listings(listings: list[ApecListing]) -> list[ApecListing]:
|
||||
seen: set[str] = set()
|
||||
deduped: list[ApecListing] = []
|
||||
for listing in listings:
|
||||
if listing.url in seen:
|
||||
continue
|
||||
seen.add(listing.url)
|
||||
deduped.append(listing)
|
||||
return deduped
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the normalization and dedupe tests to verify they pass**
|
||||
|
||||
Run: `uv run pytest tests/apec/test_normalize.py tests/apec/test_dedupe.py -v`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit normalization and dedupe**
|
||||
|
||||
```bash
|
||||
git add src/job_research/apec/normalize.py src/job_research/apec/dedupe.py tests/apec/test_normalize.py tests/apec/test_dedupe.py
|
||||
git commit -m "feat: add Apec normalization and dedupe"
|
||||
```
|
||||
|
||||
## Task 5: Public Apec Adapter and Snapshot Persistence
|
||||
|
||||
**Files:**
|
||||
- Create: `src/job_research/apec/adapter.py`
|
||||
- Modify: `tests/test_apec_storage.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing snapshot persistence test**
|
||||
|
||||
```python
|
||||
# tests/test_apec_storage.py
|
||||
from pathlib import Path
|
||||
|
||||
from job_research.storage import apec_run_paths, load_yaml
|
||||
|
||||
|
||||
def test_apec_run_artifacts_include_snapshot_and_meta(tmp_path: Path) -> None:
|
||||
paths = apec_run_paths(tmp_path, run_id="2026-06-01T10-00-00Z")
|
||||
paths["snapshots"].mkdir(parents=True, exist_ok=True)
|
||||
snapshot = paths["snapshots"] / "job-123.html"
|
||||
snapshot.write_text("<html>snapshot</html>", encoding="utf-8")
|
||||
|
||||
assert snapshot.exists()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the snapshot test to verify it fails if needed**
|
||||
|
||||
Run: `uv run pytest tests/test_apec_storage.py::test_apec_run_artifacts_include_snapshot_and_meta -v`
|
||||
Expected: PASS or minimal failure if path handling needs adjustment
|
||||
|
||||
- [ ] **Step 3: Implement the Apec adapter skeleton and snapshot write helpers**
|
||||
|
||||
```python
|
||||
# src/job_research/apec/adapter.py
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass
|
||||
class ApecSearchResult:
|
||||
url: str
|
||||
source_job_id: str | None = None
|
||||
|
||||
|
||||
class ApecAdapter:
|
||||
def __init__(self, max_listings: int = 50) -> None:
|
||||
self.max_listings = max_listings
|
||||
|
||||
def search(self, queries: list[str]) -> list[ApecSearchResult]:
|
||||
return []
|
||||
|
||||
def fetch_listing_html(self, url: str) -> str:
|
||||
return ""
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the snapshot test and any adapter-adjacent tests**
|
||||
|
||||
Run: `uv run pytest tests/test_apec_storage.py -v`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit the adapter scaffold**
|
||||
|
||||
```bash
|
||||
git add src/job_research/apec/adapter.py tests/test_apec_storage.py
|
||||
git commit -m "feat: add Apec adapter scaffold"
|
||||
```
|
||||
|
||||
## Task 6: fetch-apec Command Orchestration
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/job_research/cli.py`
|
||||
- Create: `tests/test_apec_cli.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing CLI orchestration tests**
|
||||
|
||||
```python
|
||||
# tests/test_apec_cli.py
|
||||
from pathlib import Path
|
||||
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from job_research.cli import app
|
||||
|
||||
|
||||
def test_fetch_apec_reads_profile_and_writes_run_artifacts(monkeypatch, tmp_path: Path) -> None:
|
||||
data_dir = tmp_path / "data"
|
||||
data_dir.mkdir()
|
||||
(data_dir / "candidate-profile.yaml").write_text(
|
||||
"target_roles:\n - Data Engineer\nstrengths:\n - Python\nskills_to_emphasize:\n - BigQuery\nconstraints:\n - CDI only\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
result = CliRunner().invoke(app, ["fetch-apec", "--data-root", str(data_dir)])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "normalized listing count" in result.stdout.lower()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the CLI orchestration test to verify it fails**
|
||||
|
||||
Run: `uv run pytest tests/test_apec_cli.py::test_fetch_apec_reads_profile_and_writes_run_artifacts -v`
|
||||
Expected: FAIL because `fetch-apec` does not exist yet
|
||||
|
||||
- [ ] **Step 3: Implement fetch-apec command orchestration**
|
||||
|
||||
```python
|
||||
# src/job_research/cli.py
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import typer
|
||||
|
||||
from job_research.apec.adapter import ApecAdapter
|
||||
from job_research.apec.dedupe import dedupe_apec_listings
|
||||
from job_research.apec.normalize import normalize_apec_listing
|
||||
from job_research.apec.query_derivation import derive_apec_queries
|
||||
from job_research.models import ApecRunMeta, CandidateProfileOutput, ListingError
|
||||
from job_research.storage import apec_run_paths, load_yaml, save_yaml
|
||||
|
||||
@app.command("fetch-apec")
|
||||
def fetch_apec(
|
||||
data_root: Path = typer.Option(Path("data")),
|
||||
) -> None:
|
||||
profile_payload = load_yaml(data_root / "candidate-profile.yaml")
|
||||
profile = CandidateProfileOutput.model_validate(profile_payload)
|
||||
queries = derive_apec_queries(profile)
|
||||
if not queries:
|
||||
raise typer.BadParameter("No usable Apec queries could be derived from candidate-profile.yaml")
|
||||
|
||||
run_id = datetime.now(UTC).strftime("%Y-%m-%dT%H-%M-%SZ")
|
||||
paths = apec_run_paths(data_root, run_id)
|
||||
paths["snapshots"].mkdir(parents=True, exist_ok=True)
|
||||
|
||||
adapter = ApecAdapter(max_listings=50)
|
||||
search_results = adapter.search(queries)
|
||||
listings = []
|
||||
errors: list[ListingError] = []
|
||||
|
||||
for result in search_results[:50]:
|
||||
try:
|
||||
html = adapter.fetch_listing_html(result.url)
|
||||
snapshot_path = paths["snapshots"] / f"{(result.source_job_id or 'listing').replace('/', '-')}.html"
|
||||
snapshot_path.write_text(html, encoding="utf-8")
|
||||
listings.append(normalize_apec_listing(url=result.url, html=html, fetched_at=run_id))
|
||||
except Exception as exc:
|
||||
errors.append(ListingError(url=result.url, stage="fetch_or_normalize", message=str(exc)))
|
||||
|
||||
deduped = dedupe_apec_listings(listings)
|
||||
run_meta = ApecRunMeta(
|
||||
derived_queries=queries,
|
||||
fetched_count=len(search_results[:50]),
|
||||
normalized_count=len(listings),
|
||||
deduplicated_count=len(deduped),
|
||||
failed_count=len(errors),
|
||||
listing_errors=errors,
|
||||
)
|
||||
|
||||
save_yaml(paths["listings"], {"listings": [listing.model_dump(mode="json") for listing in deduped]})
|
||||
save_yaml(paths["run_meta"], run_meta.model_dump(mode="json"))
|
||||
|
||||
typer.echo(f"Query count: {len(queries)}")
|
||||
typer.echo(f"Fetched listing count: {run_meta.fetched_count}")
|
||||
typer.echo(f"Normalized listing count: {run_meta.normalized_count}")
|
||||
typer.echo(f"Deduplicated count: {run_meta.deduplicated_count}")
|
||||
typer.echo(f"Failed listing count: {run_meta.failed_count}")
|
||||
```
|
||||
|
||||
Implementation requirements:
|
||||
- load `data/candidate-profile.yaml`
|
||||
- validate into `CandidateProfileOutput`
|
||||
- derive queries
|
||||
- create a run id and run paths
|
||||
- invoke adapter search/fetch flow
|
||||
- persist snapshots, listings.yaml, run-meta.yaml
|
||||
- print summary counts
|
||||
|
||||
- [ ] **Step 4: Run the CLI orchestration test to verify it passes**
|
||||
|
||||
Run: `uv run pytest tests/test_apec_cli.py::test_fetch_apec_reads_profile_and_writes_run_artifacts -v`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit the fetch-apec command**
|
||||
|
||||
```bash
|
||||
git add src/job_research/cli.py tests/test_apec_cli.py
|
||||
git commit -m "feat: add fetch-apec command"
|
||||
```
|
||||
|
||||
## Task 7: Full Regression and Manual Smoke Test
|
||||
|
||||
**Files:**
|
||||
- Modify: none
|
||||
|
||||
- [ ] **Step 1: Run the full test suite**
|
||||
|
||||
Run: `uv run pytest tests -v`
|
||||
Expected: PASS with all Apec-slice and profile-slice tests green
|
||||
|
||||
- [ ] **Step 2: Run a manual fetch-apec smoke test with mocked or safe local input**
|
||||
|
||||
Run: `uv run job-research fetch-apec --help`
|
||||
Expected: command help shows the Apec fetch workflow
|
||||
|
||||
- [ ] **Step 3: Commit validated Apec ingestion slice**
|
||||
|
||||
```bash
|
||||
git add pyproject.toml src/job_research tests
|
||||
git commit -m "feat: complete Apec ingestion slice"
|
||||
```
|
||||
|
||||
## Spec Coverage Check
|
||||
|
||||
- Explicit `fetch-apec` command: covered by Task 6
|
||||
- Read `data/candidate-profile.yaml`: covered by Task 6
|
||||
- Deterministic query derivation: covered by Task 3
|
||||
- 50-listing cap and adapter behavior: covered by Task 5 and Task 6
|
||||
- Raw HTML snapshot persistence: covered by Task 2, Task 5, and Task 6
|
||||
- Normalized YAML listing output: covered by Task 1, Task 4, and Task 6
|
||||
- Minimal within-run deduplication: covered by Task 4
|
||||
- Partial-success metadata and run summary: covered by Task 1, Task 2, and Task 6
|
||||
@ -1,45 +1,41 @@
|
||||
# Candidate Profile Ingestion Design
|
||||
|
||||
This spec covers the first executable slice of the job discovery system: producing a trustworthy local `Candidate Profile` and saved default `Search Preferences` from a PDF CV, a structured markdown profile, and explicit YAML overrides. It stops before scraping, ranking, or OpenCode-driven re-ranking.
|
||||
This spec covers the simplified first executable slice of the job discovery system: take a PDF CV and a light-template markdown profile, then produce one usable `candidate-profile.yaml` that the user can inspect and edit directly. It stops before scraping, ranking, separate `Search Preferences`, or OpenCode-driven session overrides.
|
||||
|
||||
## Scope
|
||||
|
||||
This slice must let the user:
|
||||
|
||||
- provide a PDF CV,
|
||||
- provide a structured markdown profile with some freeform notes,
|
||||
- generate deterministic local profile state,
|
||||
- review and resolve factual conflicts explicitly,
|
||||
- materialize a final normalized `Candidate Profile`,
|
||||
- save default `Search Preferences` separately from career facts,
|
||||
- inspect the resulting local state confidently.
|
||||
- provide a light structured markdown profile,
|
||||
- run one main CLI command,
|
||||
- produce one canonical normalized YAML profile,
|
||||
- see warnings when CV facts and markdown intent disagree or when extraction is uncertain,
|
||||
- edit the resulting YAML directly if they want to refine or correct it.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- job scraping,
|
||||
- ranking,
|
||||
- LLM explanations of job fit,
|
||||
- session-level chat overrides,
|
||||
- separate saved `Search Preferences`,
|
||||
- multi-step override workflows,
|
||||
- hard-stop review gates,
|
||||
- browser or web UI flows.
|
||||
|
||||
## Architecture
|
||||
|
||||
The design uses a layered state pipeline.
|
||||
The design is a single-step profile normalizer.
|
||||
|
||||
Inputs:
|
||||
|
||||
- PDF CV for raw career history
|
||||
- structured markdown profile for curated interpretation and goals
|
||||
- YAML overrides for explicit user corrections
|
||||
- PDF CV for raw career history,
|
||||
- light-template markdown profile for curated interpretation, job intent, and constraints.
|
||||
|
||||
Outputs:
|
||||
Output:
|
||||
|
||||
- generated state from deterministic extraction and merge
|
||||
- override state authored by the user
|
||||
- final merged normalized state for downstream consumers
|
||||
- saved default `Search Preferences` stored separately
|
||||
- one canonical `candidate-profile.yaml`.
|
||||
|
||||
The key trust rule is that factual conflicts must never be resolved silently. When the PDF CV and markdown profile disagree on factual history, the system writes the disagreement into a review-required section and stops short of claiming a trustworthy final profile until explicit override data resolves it.
|
||||
The trust rule is simpler than the earlier layered design: the system always writes the output file, but it never hides uncertainty. When the CV and markdown profile disagree on facts, or when extraction confidence is weak, the output YAML includes explicit warnings. The user can then edit the canonical YAML directly.
|
||||
|
||||
## Components
|
||||
|
||||
@ -63,145 +59,115 @@ Behavior:
|
||||
|
||||
- prefer explicit extraction over inference,
|
||||
- leave uncertainty visible instead of guessing,
|
||||
- avoid LLM use unless deterministic extraction is incomplete or ambiguous enough to block useful structuring.
|
||||
- use LLM assistance only when deterministic extraction is too weak to structure useful output.
|
||||
|
||||
### Markdown Profile Parser
|
||||
|
||||
Responsibility:
|
||||
|
||||
- read a required structured markdown template,
|
||||
- parse known sections into normalized authored signals,
|
||||
- preserve some freeform notes for nuance.
|
||||
- read a light required markdown template,
|
||||
- parse known sections into structured authored intent.
|
||||
|
||||
Expected content includes:
|
||||
Required sections:
|
||||
|
||||
- strengths,
|
||||
- target roles,
|
||||
- tool and stack emphasis,
|
||||
- interpretation of prior experience,
|
||||
- narrative clarifications not obvious from the CV.
|
||||
- `Summary`
|
||||
- `Target Roles`
|
||||
- `Strengths`
|
||||
- `Skills To Emphasize`
|
||||
- `Constraints`
|
||||
- `Notes`
|
||||
|
||||
### Merge and Conflict Analyzer
|
||||
This file carries both durable interpretation and current job intent for the first slice.
|
||||
|
||||
### Profile Merger
|
||||
|
||||
Responsibility:
|
||||
|
||||
- combine CV-derived and markdown-derived state into generated YAML,
|
||||
- apply source authority rules,
|
||||
- identify factual conflicts and unresolved ambiguities.
|
||||
- combine CV-derived facts and markdown-derived intent,
|
||||
- normalize them into one canonical profile document,
|
||||
- record warnings for factual conflicts or ambiguity.
|
||||
|
||||
Authority rules:
|
||||
|
||||
- PDF CV is the default source of truth for raw career history,
|
||||
- markdown profile is the source of truth for curated interpretation and goals,
|
||||
- factual disagreements are surfaced and require explicit resolution,
|
||||
- missing non-critical data may remain unresolved if it is clearly marked.
|
||||
|
||||
### Override Applier
|
||||
|
||||
Responsibility:
|
||||
|
||||
- read user-authored YAML overrides,
|
||||
- resolve review-required conflicts,
|
||||
- materialize the final canonical `Candidate Profile`.
|
||||
|
||||
This unit is the only place where explicit user correction can replace generated factual values.
|
||||
|
||||
### Search Preferences Manager
|
||||
|
||||
Responsibility:
|
||||
|
||||
- validate and persist saved default `Search Preferences` in a separate YAML file,
|
||||
- keep them separate from the durable `Candidate Profile`.
|
||||
|
||||
This slice stores only saved defaults, not temporary session overrides.
|
||||
- CV is the default source for raw career history,
|
||||
- markdown profile is the source for interpretation, emphasis, and job intent,
|
||||
- factual disagreement becomes a warning, not a blocked workflow,
|
||||
- the final YAML is user-editable and becomes the local source of truth after generation.
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. The user provides a PDF CV.
|
||||
2. The user provides a structured markdown profile.
|
||||
2. The user provides a light-template markdown profile.
|
||||
3. The system extracts CV-derived structured candidates.
|
||||
4. The system parses markdown-derived authored signals.
|
||||
5. The merge and conflict analyzer writes generated YAML.
|
||||
6. If factual conflicts or blocking ambiguities exist, the system writes them into a review-required section and exits with review-required status.
|
||||
7. The user edits an override YAML file.
|
||||
8. The override applier materializes the final normalized `Candidate Profile`.
|
||||
9. The user saves validated default `Search Preferences` separately.
|
||||
|
||||
The system should support rerunning generation without losing authored overrides.
|
||||
4. The system parses markdown-derived authored intent.
|
||||
5. The system merges both into one normalized profile document.
|
||||
6. The system writes `candidate-profile.yaml`.
|
||||
7. The system includes warnings inside the YAML when facts conflict or extraction is uncertain.
|
||||
8. The user edits `candidate-profile.yaml` directly if they want to refine the final result.
|
||||
|
||||
## File Layout
|
||||
|
||||
```text
|
||||
data/
|
||||
profiles/
|
||||
cv/
|
||||
raw-text.txt
|
||||
authored/
|
||||
profile.md
|
||||
generated/
|
||||
candidate-profile.generated.yaml
|
||||
overrides/
|
||||
candidate-profile.override.yaml
|
||||
final/
|
||||
candidate-profile.yaml
|
||||
preferences/
|
||||
search-preferences.yaml
|
||||
candidate-profile.yaml
|
||||
```
|
||||
|
||||
File semantics:
|
||||
Optional debug artifacts may be added later, but the first slice should optimize for a single obvious output file.
|
||||
|
||||
- `candidate-profile.generated.yaml` is machine-produced and replaceable
|
||||
- `candidate-profile.override.yaml` is user-owned
|
||||
- `candidate-profile.yaml` is the canonical merged state for downstream consumers
|
||||
- `search-preferences.yaml` is a separate document because `Search Preferences` are not durable career facts
|
||||
## Output Shape
|
||||
|
||||
JSON export may be supported for debugging or automation, but YAML is the primary local format.
|
||||
The canonical YAML should include:
|
||||
|
||||
- identity and summary,
|
||||
- target roles and job intent,
|
||||
- skills and technologies,
|
||||
- experience entries,
|
||||
- education entries,
|
||||
- constraints,
|
||||
- notes,
|
||||
- warnings.
|
||||
|
||||
The warnings section should be human-readable and should point out:
|
||||
|
||||
- factual conflicts,
|
||||
- missing important fields,
|
||||
- low-confidence extraction areas.
|
||||
|
||||
## Validation and Error Handling
|
||||
|
||||
### Hard Failures
|
||||
|
||||
These should fail immediately and avoid producing a claimed final profile:
|
||||
These should fail immediately and avoid producing misleading output:
|
||||
|
||||
- unreadable or missing PDF input,
|
||||
- malformed structured markdown profile,
|
||||
- invalid YAML override format,
|
||||
- invalid `Search Preferences` schema.
|
||||
- malformed markdown template missing required headings,
|
||||
- invalid command arguments.
|
||||
|
||||
### Review-Required State
|
||||
### Non-Blocking Warnings
|
||||
|
||||
These should produce generated state plus a clear review-required status:
|
||||
These should still produce `candidate-profile.yaml`:
|
||||
|
||||
- factual conflicts between CV and markdown profile,
|
||||
- blocking ambiguity that would make the final profile untrustworthy if auto-resolved.
|
||||
|
||||
### Allowed Ambiguity
|
||||
|
||||
These may remain visible without blocking final state if clearly recorded:
|
||||
|
||||
- incomplete optional fields,
|
||||
- non-critical uncertainty that does not alter durable career facts,
|
||||
- missing interpretation details that can be added later.
|
||||
- low-confidence extraction,
|
||||
- unclear dates or inferred experience totals.
|
||||
|
||||
The system must prefer explicit unresolved ambiguity over silent incorrect certainty.
|
||||
The system must prefer explicit warnings over silent incorrect certainty.
|
||||
|
||||
## Command Surface
|
||||
|
||||
Proposed CLI commands for this slice:
|
||||
The first slice should expose one main command:
|
||||
|
||||
- `ingest-cv <cv.pdf>`
|
||||
- `ingest-profile <profile.md>`
|
||||
- `build-profile --cv <cv.pdf> --profile <profile.md>`
|
||||
- `apply-overrides <override.yaml>`
|
||||
- `show-profile --final`
|
||||
- `set-preferences <preferences.yaml>`
|
||||
- `show-preferences`
|
||||
- `build-profile --cv <cv.pdf> --profile <profile.md> --out data/candidate-profile.yaml`
|
||||
|
||||
Optional helper commands may be added later, but they are not required for this slice.
|
||||
|
||||
Expected behavior:
|
||||
|
||||
- `build-profile` generates deterministic state and signals review-required conditions
|
||||
- `apply-overrides` resolves explicit user corrections into final canonical state
|
||||
- `set-preferences` validates and persists saved default `Search Preferences`
|
||||
- read both inputs,
|
||||
- produce one canonical YAML file,
|
||||
- print a success message,
|
||||
- print whether warnings were included.
|
||||
|
||||
## LLM Usage Boundaries
|
||||
|
||||
@ -209,40 +175,36 @@ This slice is deterministic-first.
|
||||
|
||||
Allowed LLM use:
|
||||
|
||||
- fallback interpretation when deterministic extraction is too ambiguous to structure usefully,
|
||||
- optional structuring help where the result is still auditable in generated YAML.
|
||||
- fallback structuring help when deterministic extraction alone cannot produce a usable normalized profile.
|
||||
|
||||
Disallowed LLM use:
|
||||
|
||||
- replacing deterministic extraction as the default path,
|
||||
- silently deciding factual conflicts,
|
||||
- acting as the source of truth for final persisted profile state.
|
||||
- acting as the only source of truth for the final persisted profile.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
The first slice should be tested around trust and repeatability.
|
||||
The first slice should be tested around usefulness and transparency.
|
||||
|
||||
Core test cases:
|
||||
|
||||
- PDF-only ingestion produces stable generated state where possible
|
||||
- structured markdown parsing maps required sections correctly
|
||||
- CV and markdown merge cleanly when there is no factual conflict
|
||||
- factual conflict produces review-required output and no false success signal
|
||||
- overrides resolve conflicts into the final canonical profile correctly
|
||||
- final YAML serialization is stable and readable
|
||||
- invalid overrides fail with actionable validation feedback
|
||||
- `Search Preferences` validation and persistence behave independently from the profile pipeline
|
||||
- valid CV plus valid markdown produces `candidate-profile.yaml`,
|
||||
- missing required markdown sections fails clearly,
|
||||
- factual conflict still writes output and records warnings,
|
||||
- final YAML contains both extracted facts and markdown intent,
|
||||
- output serialization is stable and readable,
|
||||
- command-line usage is repeatable.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
This slice is done when:
|
||||
|
||||
- a real PDF CV and structured markdown profile can produce generated YAML locally,
|
||||
- factual conflicts are surfaced explicitly and never resolved silently,
|
||||
- override YAML can resolve those conflicts into a final canonical `Candidate Profile`,
|
||||
- the final profile is trustworthy enough to become the source for later ranking work,
|
||||
- saved default `Search Preferences` are validated and stored separately from the `Candidate Profile`,
|
||||
- the workflow is repeatable from the CLI without requiring scraping, ranking, or UI work.
|
||||
- a real PDF CV and light-template markdown profile can produce one local `candidate-profile.yaml`,
|
||||
- the YAML is readable and useful enough for the user to trust and edit directly,
|
||||
- factual conflicts and uncertainty appear as warnings rather than hidden behavior,
|
||||
- the workflow is one obvious command rather than a multi-step state machine,
|
||||
- the result is good enough to become the input to later scraping and ranking work.
|
||||
|
||||
## Non-Goals for This Slice
|
||||
|
||||
@ -253,6 +215,7 @@ This design intentionally does not solve:
|
||||
- score computation,
|
||||
- `Stretch Opportunity` classification,
|
||||
- dismissed listing behavior,
|
||||
- OpenCode-driven session override flows.
|
||||
- separate persisted `Search Preferences`,
|
||||
- session-level conversational overrides.
|
||||
|
||||
Those belong to later slices after profile state is trustworthy.
|
||||
Those belong to later slices after the system can reliably understand the candidate.
|
||||
|
||||
236
docs/superpowers/specs/2026-06-01-apec-ingestion-design.md
Normal file
236
docs/superpowers/specs/2026-06-01-apec-ingestion-design.md
Normal file
@ -0,0 +1,236 @@
|
||||
# Apec Ingestion Design
|
||||
|
||||
This spec covers the next executable slice of the job discovery system: use the canonical `Candidate Profile` to fetch public Apec listings, store raw listing snapshots, and produce a first normalized YAML listing file for inspection. It stops before ranking, explanations, and multi-source support.
|
||||
|
||||
## Scope
|
||||
|
||||
This slice must let the user:
|
||||
|
||||
- run one explicit command,
|
||||
- read `data/candidate-profile.yaml`,
|
||||
- derive deterministic Apec search queries from that profile,
|
||||
- fetch up to 50 public Apec listings,
|
||||
- store raw listing detail-page HTML and crawl metadata,
|
||||
- produce one normalized YAML file of listings,
|
||||
- tolerate partial failure by keeping successful listings and recording per-listing errors.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- ranking,
|
||||
- LLM explanation,
|
||||
- non-Apec sources,
|
||||
- cross-run or cross-source deduplication,
|
||||
- database storage,
|
||||
- authenticated sessions.
|
||||
|
||||
## Architecture
|
||||
|
||||
The design is a fetch-and-normalize pipeline centered on one explicit command:
|
||||
|
||||
- `fetch-apec`
|
||||
|
||||
Inputs:
|
||||
|
||||
- `data/candidate-profile.yaml`
|
||||
|
||||
Outputs:
|
||||
|
||||
- raw Apec listing snapshots,
|
||||
- per-run metadata,
|
||||
- one normalized `listings.yaml` file.
|
||||
|
||||
The key boundary is that this slice acquires and structures listing data but does not score it. It should preserve enough raw evidence to debug extraction and enough normalized structure to support the future ranking slice.
|
||||
|
||||
## Components
|
||||
|
||||
### Profile Query Derivation
|
||||
|
||||
Responsibility:
|
||||
|
||||
- read the canonical `Candidate Profile`,
|
||||
- derive a small deterministic set of Apec search queries from profile content.
|
||||
|
||||
This should use explicit profile fields such as:
|
||||
|
||||
- target roles,
|
||||
- strengths,
|
||||
- skills to emphasize,
|
||||
- constraints.
|
||||
|
||||
It must be deterministic and reproducible, not LLM-generated.
|
||||
|
||||
### Apec Search Adapter
|
||||
|
||||
Responsibility:
|
||||
|
||||
- execute public Apec searches,
|
||||
- gather listing URLs and lightweight result metadata,
|
||||
- paginate as needed,
|
||||
- respect the 50-listing cap across the full run.
|
||||
|
||||
### Listing Snapshot Fetcher
|
||||
|
||||
Responsibility:
|
||||
|
||||
- visit each public Apec listing detail page,
|
||||
- store:
|
||||
- listing URL,
|
||||
- raw HTML,
|
||||
- crawl timestamp,
|
||||
- basic fetch metadata,
|
||||
- per-listing error information on failure.
|
||||
|
||||
### Normalizer
|
||||
|
||||
Responsibility:
|
||||
|
||||
- transform successful Apec detail pages into a first normalized listing shape,
|
||||
- extract obvious fields without overreaching.
|
||||
|
||||
### Within-Run Deduper
|
||||
|
||||
Responsibility:
|
||||
|
||||
- remove obvious duplicates within the current run only,
|
||||
- use Apec URL or source job ID when available.
|
||||
|
||||
This slice does not attempt cross-run or cross-source deduplication.
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. The user runs `fetch-apec`.
|
||||
2. The system reads `data/candidate-profile.yaml`.
|
||||
3. It derives a small deterministic set of Apec search queries.
|
||||
4. It fetches search results from Apec.
|
||||
5. It caps the full run at 50 listings.
|
||||
6. For each listing URL:
|
||||
- fetch the public detail page,
|
||||
- store raw HTML and crawl metadata,
|
||||
- normalize successful pages into listing records,
|
||||
- record per-listing errors when fetch or normalization fails.
|
||||
7. It performs minimal within-run deduplication.
|
||||
8. It writes:
|
||||
- `listings.yaml`
|
||||
- `run-meta.yaml`
|
||||
- snapshot HTML files
|
||||
9. It prints a concise run summary.
|
||||
|
||||
## File Layout
|
||||
|
||||
```text
|
||||
data/
|
||||
candidate-profile.yaml
|
||||
apec/
|
||||
runs/
|
||||
<timestamp>/
|
||||
listings.yaml
|
||||
run-meta.yaml
|
||||
snapshots/
|
||||
<listing-id-or-slug>.html
|
||||
```
|
||||
|
||||
File roles:
|
||||
|
||||
- `listings.yaml` contains normalized deduplicated listings for that run
|
||||
- `run-meta.yaml` contains derived queries, counts, crawl metadata, and per-listing errors
|
||||
- `snapshots/*.html` contains raw listing detail pages for debugging and later extraction improvements
|
||||
|
||||
## Command Surface
|
||||
|
||||
This slice should expose one explicit command:
|
||||
|
||||
- `uv run job-research fetch-apec`
|
||||
|
||||
Expected behavior:
|
||||
|
||||
- load the default candidate profile,
|
||||
- derive queries automatically,
|
||||
- fetch public Apec listings,
|
||||
- write run artifacts,
|
||||
- print a summary including:
|
||||
- query count,
|
||||
- fetched listing count,
|
||||
- normalized listing count,
|
||||
- deduplicated count,
|
||||
- failed listing count.
|
||||
|
||||
## Normalized Listing Shape
|
||||
|
||||
The first normalized listing shape should include:
|
||||
|
||||
- `source`
|
||||
- `source_job_id` when available
|
||||
- `url`
|
||||
- `title`
|
||||
- `company`
|
||||
- `location`
|
||||
- `contract_type`
|
||||
- `description_text`
|
||||
- `published_at` when available
|
||||
- `fetched_at`
|
||||
- `warnings`
|
||||
|
||||
This is intentionally minimal. Salary, remote mode, seniority inference, and richer tagging belong to later slices unless extraction proves stable enough to add them cheaply.
|
||||
|
||||
## Validation and Error Handling
|
||||
|
||||
### Hard Failures
|
||||
|
||||
These should fail the run before or during crawling:
|
||||
|
||||
- missing or unreadable `data/candidate-profile.yaml`,
|
||||
- no usable derived search queries,
|
||||
- Apec unreachable for the whole run,
|
||||
- every listing fetch failing.
|
||||
|
||||
### Partial-Success Behavior
|
||||
|
||||
These should still produce output artifacts:
|
||||
|
||||
- some listing detail pages fail to load,
|
||||
- some listing pages load but normalize incompletely,
|
||||
- some duplicates are removed during within-run deduplication.
|
||||
|
||||
When partial success occurs, the run should:
|
||||
|
||||
- keep successful normalized listings,
|
||||
- record per-listing failures in `run-meta.yaml`,
|
||||
- preserve available snapshots.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
The slice should be tested around correctness of ingestion, resilience, and artifact shape.
|
||||
|
||||
Core test cases:
|
||||
|
||||
- deterministic query derivation from `candidate-profile.yaml`,
|
||||
- 50-listing cap behavior,
|
||||
- snapshot persistence for successful detail pages,
|
||||
- partial-success handling with per-listing errors,
|
||||
- minimal within-run deduplication,
|
||||
- normalized YAML output shape,
|
||||
- run summary metadata.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
This slice is done when:
|
||||
|
||||
- one command can fetch public Apec listings from the canonical candidate profile,
|
||||
- raw detail-page HTML snapshots are preserved,
|
||||
- one normalized `listings.yaml` file is produced,
|
||||
- partial failures are visible rather than silent,
|
||||
- the resulting listing artifact is good enough to feed the later ranking slice.
|
||||
|
||||
## Non-Goals for This Slice
|
||||
|
||||
This design intentionally does not solve:
|
||||
|
||||
- listing ranking,
|
||||
- fit explanations,
|
||||
- source-agnostic ingestion,
|
||||
- multi-board aggregation,
|
||||
- database-backed run history,
|
||||
- login-required crawling,
|
||||
- long-term deduplication across runs.
|
||||
|
||||
Those belong to later slices after raw listing acquisition is stable.
|
||||
@ -1,7 +1,30 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=69"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "job-research"
|
||||
version = "0.1.0"
|
||||
description = "Add your description here"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.13"
|
||||
dependencies = []
|
||||
dependencies = [
|
||||
"beautifulsoup4>=4.12,<5",
|
||||
"playwright>=1.52,<2",
|
||||
"pydantic>=2.7,<3",
|
||||
"pypdf>=5.0,<6",
|
||||
"pyyaml>=6.0,<7",
|
||||
"typer>=0.12,<1",
|
||||
]
|
||||
|
||||
[dependency-groups]
|
||||
dev = ["pytest>=8.2,<9"]
|
||||
|
||||
[project.scripts]
|
||||
job-research = "job_research.cli:main"
|
||||
|
||||
[tool.setuptools]
|
||||
package-dir = {"" = "src"}
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
3
src/job_research/__init__.py
Normal file
3
src/job_research/__init__.py
Normal file
@ -0,0 +1,3 @@
|
||||
__all__ = ["__version__"]
|
||||
|
||||
__version__ = "0.1.0"
|
||||
3
src/job_research/apec/__init__.py
Normal file
3
src/job_research/apec/__init__.py
Normal file
@ -0,0 +1,3 @@
|
||||
from job_research.apec.query_derivation import derive_apec_queries
|
||||
|
||||
__all__ = ["derive_apec_queries"]
|
||||
317
src/job_research/apec/adapter.py
Normal file
317
src/job_research/apec/adapter.py
Normal file
@ -0,0 +1,317 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from urllib.parse import parse_qsl, urlencode, urlparse, urlsplit, urlunsplit
|
||||
|
||||
from playwright.sync_api import Error as PlaywrightError
|
||||
from playwright.sync_api import TimeoutError as PlaywrightTimeoutError
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
from job_research.models import ListingError
|
||||
|
||||
|
||||
_SEARCH_URL = "https://www.apec.fr/candidat/recherche-emploi.html/emploi"
|
||||
_FRANCE_LOCATION_ID = "799"
|
||||
_CDI_CONTRACT_ID = "101888"
|
||||
_SEARCH_INPUT_SELECTOR = 'input[name="keywords"]'
|
||||
_RESULT_LINK_SELECTOR = "a[href*='/candidat/recherche-emploi.html/emploi/detail-offre/']"
|
||||
_ZERO_RESULTS_URL_FRAGMENT = "/recherche-avancee"
|
||||
_DETAIL_JOB_ID_PATTERN = re.compile(r"/detail-offre/([^/?#]+)")
|
||||
_APEC_HOSTS = {"apec.fr", "www.apec.fr"}
|
||||
_MAX_PAGES_PER_QUERY = 50
|
||||
_MAX_CONSECUTIVE_NO_PROGRESS_PAGES = 10
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class ApecSearchResult:
|
||||
url: str
|
||||
source_job_id: str | None = None
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class ApecSearchFilters:
|
||||
location: str | None = None
|
||||
contract_type: str | None = None
|
||||
|
||||
|
||||
class ApecSearchError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _open_public_page():
|
||||
with sync_playwright() as playwright:
|
||||
browser = playwright.chromium.launch(headless=True)
|
||||
try:
|
||||
page = browser.new_page()
|
||||
page.set_default_timeout(15_000)
|
||||
yield page
|
||||
finally:
|
||||
browser.close()
|
||||
|
||||
|
||||
def _extract_source_job_id(url: str) -> str | None:
|
||||
match = _DETAIL_JOB_ID_PATTERN.search(url)
|
||||
if match is None:
|
||||
return None
|
||||
|
||||
return match.group(1)
|
||||
|
||||
|
||||
def _search_results_url(base_url: str, page_number: int) -> str:
|
||||
parsed_url = urlsplit(base_url)
|
||||
params = parse_qsl(parsed_url.query, keep_blank_values=True)
|
||||
filtered_params = [(key, value) for key, value in params if key != "page"]
|
||||
filtered_params.append(("page", str(page_number)))
|
||||
return urlunsplit(parsed_url._replace(query=urlencode(filtered_params, doseq=True)))
|
||||
|
||||
|
||||
def _search_url(query: str, search_filters: ApecSearchFilters, page_number: int = 0) -> str:
|
||||
params = [
|
||||
("motsCles", query),
|
||||
("page", str(page_number)),
|
||||
]
|
||||
|
||||
if search_filters.location == "France":
|
||||
params.insert(1, ("lieux", _FRANCE_LOCATION_ID))
|
||||
|
||||
if search_filters.contract_type == "CDI":
|
||||
params.insert(2 if search_filters.location == "France" else 1, ("typesContrat", _CDI_CONTRACT_ID))
|
||||
|
||||
return f"{_SEARCH_URL}?{urlencode(params)}"
|
||||
|
||||
|
||||
def _accept_cookies_if_present(page) -> None:
|
||||
try:
|
||||
page.locator('input[name="cguAcceptees"]').check(timeout=2_000)
|
||||
except (AttributeError, PlaywrightError, PlaywrightTimeoutError):
|
||||
pass
|
||||
|
||||
for button_name in ("ACCEPTER", "Accepter tous les cookies"):
|
||||
try:
|
||||
page.get_by_role("button", name=button_name).click(timeout=2_000)
|
||||
return
|
||||
except (AttributeError, PlaywrightError, PlaywrightTimeoutError):
|
||||
continue
|
||||
|
||||
|
||||
def _goto_and_wait(page, url: str) -> bool:
|
||||
try:
|
||||
page.goto(url, wait_until="domcontentloaded")
|
||||
page.wait_for_load_state("domcontentloaded")
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def _is_public_apec_detail_url(url: str) -> bool:
|
||||
parsed_url = urlparse(url)
|
||||
return (
|
||||
parsed_url.scheme == "https"
|
||||
and parsed_url.hostname in _APEC_HOSTS
|
||||
and re.fullmatch(r"/candidat/recherche-emploi\.html/emploi/detail-offre/[^/?#]+", parsed_url.path)
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
class ApecAdapter:
|
||||
def __init__(self, max_listings: int = 50) -> None:
|
||||
self.max_listings = max_listings
|
||||
self.search_errors: list[ListingError] = []
|
||||
self._browser_context = None
|
||||
|
||||
@contextmanager
|
||||
def browser_session(self):
|
||||
if self._browser_context is not None:
|
||||
yield
|
||||
return
|
||||
|
||||
with sync_playwright() as playwright:
|
||||
browser = playwright.chromium.launch(headless=True)
|
||||
browser_context = browser.new_context()
|
||||
self._browser_context = browser_context
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
self._browser_context = None
|
||||
browser.close()
|
||||
|
||||
@contextmanager
|
||||
def _open_page(self):
|
||||
if self._browser_context is None:
|
||||
with _open_public_page() as page:
|
||||
yield page
|
||||
return
|
||||
|
||||
page = self._browser_context.new_page()
|
||||
page.set_default_timeout(15_000)
|
||||
try:
|
||||
yield page
|
||||
finally:
|
||||
page.close()
|
||||
|
||||
def _record_search_error(
|
||||
self,
|
||||
query: str,
|
||||
search_filters: ApecSearchFilters,
|
||||
message: str,
|
||||
*,
|
||||
url: str | None = None,
|
||||
) -> None:
|
||||
self.search_errors.append(
|
||||
ListingError(url=url or _search_url(query, search_filters), stage="search", message=message)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _is_zero_results_page(page) -> bool:
|
||||
return _ZERO_RESULTS_URL_FRAGMENT in page.url and "error=true" in page.url
|
||||
|
||||
def search(self, queries: list[str], search_filters: ApecSearchFilters) -> list[ApecSearchResult]:
|
||||
results: list[ApecSearchResult] = []
|
||||
seen_keys: set[str] = set()
|
||||
usable_search_page_seen = False
|
||||
self.search_errors = []
|
||||
|
||||
with self._open_page() as page:
|
||||
for query in queries:
|
||||
if not query.strip():
|
||||
continue
|
||||
|
||||
if len(results) >= self.max_listings:
|
||||
break
|
||||
|
||||
if not _goto_and_wait(page, _search_url(query, search_filters)):
|
||||
self._record_search_error(query, search_filters, "search page navigation failed")
|
||||
continue
|
||||
|
||||
_accept_cookies_if_present(page)
|
||||
|
||||
try:
|
||||
page.wait_for_selector(_SEARCH_INPUT_SELECTOR, timeout=5_000)
|
||||
except PlaywrightTimeoutError:
|
||||
self._record_search_error(query, search_filters, "search input did not render")
|
||||
continue
|
||||
|
||||
if self._is_zero_results_page(page):
|
||||
usable_search_page_seen = True
|
||||
continue
|
||||
|
||||
try:
|
||||
page.wait_for_selector(_RESULT_LINK_SELECTOR, timeout=5_000)
|
||||
except PlaywrightTimeoutError:
|
||||
self._record_search_error(query, search_filters, "search results did not render")
|
||||
continue
|
||||
|
||||
usable_search_page_seen = True
|
||||
|
||||
result_page_url = page.url
|
||||
seen_page_urls: set[str] = {result_page_url}
|
||||
no_progress_pages = 0
|
||||
|
||||
for page_number in range(_MAX_PAGES_PER_QUERY):
|
||||
if len(results) >= self.max_listings:
|
||||
break
|
||||
|
||||
if page_number > 0:
|
||||
next_page_url = _search_results_url(result_page_url, page_number)
|
||||
if next_page_url in seen_page_urls:
|
||||
break
|
||||
|
||||
if not _goto_and_wait(page, next_page_url):
|
||||
self._record_search_error(
|
||||
query,
|
||||
search_filters,
|
||||
f"page {page_number} navigation failed",
|
||||
url=next_page_url,
|
||||
)
|
||||
break
|
||||
|
||||
try:
|
||||
page.wait_for_selector(_RESULT_LINK_SELECTOR, timeout=5_000)
|
||||
except PlaywrightTimeoutError:
|
||||
self._record_search_error(
|
||||
query,
|
||||
search_filters,
|
||||
f"page {page_number} results did not render",
|
||||
url=next_page_url,
|
||||
)
|
||||
break
|
||||
|
||||
current_page_url = page.url
|
||||
if page_number > 0 and current_page_url in seen_page_urls:
|
||||
break
|
||||
|
||||
seen_page_urls.add(current_page_url)
|
||||
|
||||
try:
|
||||
hrefs = page.locator(_RESULT_LINK_SELECTOR).evaluate_all(
|
||||
"nodes => nodes.map(node => node.href)"
|
||||
)
|
||||
except Exception:
|
||||
self._record_search_error(
|
||||
query,
|
||||
search_filters,
|
||||
f"page {page_number} result links could not be evaluated",
|
||||
url=current_page_url,
|
||||
)
|
||||
break
|
||||
|
||||
if not hrefs:
|
||||
no_progress_pages += 1
|
||||
if no_progress_pages >= _MAX_CONSECUTIVE_NO_PROGRESS_PAGES:
|
||||
break
|
||||
|
||||
continue
|
||||
|
||||
added_any_result = False
|
||||
for href in hrefs:
|
||||
source_job_id = _extract_source_job_id(href)
|
||||
dedupe_key = source_job_id or href
|
||||
if dedupe_key in seen_keys:
|
||||
continue
|
||||
|
||||
seen_keys.add(dedupe_key)
|
||||
results.append(ApecSearchResult(url=href, source_job_id=source_job_id))
|
||||
added_any_result = True
|
||||
|
||||
if len(results) >= self.max_listings:
|
||||
break
|
||||
|
||||
if added_any_result:
|
||||
no_progress_pages = 0
|
||||
else:
|
||||
no_progress_pages += 1
|
||||
if no_progress_pages >= _MAX_CONSECUTIVE_NO_PROGRESS_PAGES:
|
||||
break
|
||||
|
||||
if not usable_search_page_seen:
|
||||
raise ApecSearchError("Apec search page was not reachable for any query")
|
||||
|
||||
return results
|
||||
|
||||
def fetch_listing_html(self, url: str) -> str:
|
||||
if not _is_public_apec_detail_url(url):
|
||||
raise ValueError("ApecAdapter only fetches public Apec URLs")
|
||||
|
||||
with self._open_page() as page:
|
||||
page.goto(url, wait_until="domcontentloaded")
|
||||
_accept_cookies_if_present(page)
|
||||
page.wait_for_function(
|
||||
"""
|
||||
() => {
|
||||
const title = document.querySelector('.container-details-offer h1, h1');
|
||||
const reference = document.querySelector('.ref-offre');
|
||||
const offerList = document.querySelector('.details-offer-list');
|
||||
return !!title && !!reference && !!offerList;
|
||||
}
|
||||
""",
|
||||
polling=1000,
|
||||
timeout=15_000,
|
||||
)
|
||||
final_url = page.url
|
||||
if not _is_public_apec_detail_url(final_url):
|
||||
raise ValueError(f"ApecAdapter landed on an unexpected URL after redirects: {final_url}")
|
||||
return page.content()
|
||||
92
src/job_research/apec/dedupe.py
Normal file
92
src/job_research/apec/dedupe.py
Normal file
@ -0,0 +1,92 @@
|
||||
from job_research.models import ApecListing
|
||||
|
||||
|
||||
_MERGEABLE_FIELDS = (
|
||||
"source_job_id",
|
||||
"title",
|
||||
"company",
|
||||
"location",
|
||||
"contract_type",
|
||||
"description_text",
|
||||
"published_at",
|
||||
"refreshed_at",
|
||||
)
|
||||
|
||||
|
||||
def _merge_listing_metadata(survivor: ApecListing, source: ApecListing) -> None:
|
||||
for field_name in _MERGEABLE_FIELDS:
|
||||
if getattr(survivor, field_name) is None:
|
||||
value = getattr(source, field_name)
|
||||
if value is not None:
|
||||
setattr(survivor, field_name, value)
|
||||
|
||||
for warning in source.warnings:
|
||||
if warning not in survivor.warnings:
|
||||
survivor.warnings.append(warning)
|
||||
|
||||
|
||||
def _register_listing(
|
||||
url_to_listing: dict[str, ApecListing],
|
||||
source_job_id_to_listing: dict[str, ApecListing],
|
||||
listing: ApecListing,
|
||||
survivor: ApecListing,
|
||||
) -> None:
|
||||
url_to_listing[listing.url] = survivor
|
||||
|
||||
if listing.source_job_id is not None:
|
||||
source_job_id_to_listing[listing.source_job_id] = survivor
|
||||
|
||||
|
||||
def _repoint_listing_aliases(
|
||||
url_to_listing: dict[str, ApecListing],
|
||||
source_job_id_to_listing: dict[str, ApecListing],
|
||||
removed: ApecListing,
|
||||
survivor: ApecListing,
|
||||
) -> None:
|
||||
for mapping in (url_to_listing, source_job_id_to_listing):
|
||||
for alias, listing in list(mapping.items()):
|
||||
if listing is removed:
|
||||
mapping[alias] = survivor
|
||||
|
||||
|
||||
def dedupe_apec_listings(listings: list[ApecListing]) -> list[ApecListing]:
|
||||
url_to_listing: dict[str, ApecListing] = {}
|
||||
source_job_id_to_listing: dict[str, ApecListing] = {}
|
||||
survivor_order: dict[int, int] = {}
|
||||
next_order = 0
|
||||
deduped: list[ApecListing] = []
|
||||
|
||||
for listing in listings:
|
||||
source_job_id = listing.source_job_id
|
||||
matches: list[ApecListing] = []
|
||||
|
||||
url_match = url_to_listing.get(listing.url)
|
||||
if url_match is not None:
|
||||
matches.append(url_match)
|
||||
|
||||
if source_job_id is not None:
|
||||
source_job_id_match = source_job_id_to_listing.get(source_job_id)
|
||||
if source_job_id_match is not None and source_job_id_match not in matches:
|
||||
matches.append(source_job_id_match)
|
||||
|
||||
if not matches:
|
||||
deduped.append(listing)
|
||||
survivor_order[id(listing)] = next_order
|
||||
next_order += 1
|
||||
_register_listing(url_to_listing, source_job_id_to_listing, listing, listing)
|
||||
continue
|
||||
|
||||
survivor = min(matches, key=lambda candidate: survivor_order[id(candidate)])
|
||||
for other in matches:
|
||||
if other is survivor:
|
||||
continue
|
||||
|
||||
_merge_listing_metadata(survivor, other)
|
||||
deduped[:] = [item for item in deduped if item is not other]
|
||||
_repoint_listing_aliases(url_to_listing, source_job_id_to_listing, other, survivor)
|
||||
survivor_order.pop(id(other), None)
|
||||
|
||||
_merge_listing_metadata(survivor, listing)
|
||||
_register_listing(url_to_listing, source_job_id_to_listing, listing, survivor)
|
||||
|
||||
return deduped
|
||||
328
src/job_research/apec/normalize.py
Normal file
328
src/job_research/apec/normalize.py
Normal file
@ -0,0 +1,328 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import unicodedata
|
||||
from datetime import datetime
|
||||
|
||||
from bs4 import BeautifulSoup
|
||||
from bs4.element import NavigableString
|
||||
|
||||
from job_research.models import ApecListing, ListingWarning
|
||||
|
||||
|
||||
_PUBLISHED_AT_PATTERN = re.compile(r"Publi[ée]e le (\d{2}/\d{2}/\d{4})")
|
||||
_REFRESHED_AT_PATTERN = re.compile(r"Actualis[ée]e le (\d{2}/\d{2}/\d{4})")
|
||||
_DETAIL_JOB_ID_PATTERN = re.compile(r"/detail-offre/([^/?#]+)")
|
||||
_SOURCE_JOB_ID_PATTERN = re.compile(r"Ref\. Apec\s*:\s*([A-Z0-9]+)")
|
||||
_CONTRACT_PATTERN = re.compile(r"\b(CDI|CDD|Alternance|Intérim|Stage|Freelance|Indépendant)\b")
|
||||
_HEADING_TAG_NAMES = {"h1", "h2", "h3", "h4", "h5", "h6"}
|
||||
_PLACEHOLDER_TEXT_TOKENS = {
|
||||
"na",
|
||||
"nr",
|
||||
"none",
|
||||
"null",
|
||||
"unknown",
|
||||
"tbd",
|
||||
"nonrenseigne",
|
||||
"nonrenseignee",
|
||||
"nondisponible",
|
||||
}
|
||||
|
||||
|
||||
def _clean_text(value: str | None) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
|
||||
cleaned = " ".join(value.split())
|
||||
return cleaned or None
|
||||
|
||||
|
||||
def _text_token(value: str) -> str:
|
||||
normalized = unicodedata.normalize("NFKD", value)
|
||||
return re.sub(r"[^a-z0-9]+", "", normalized.casefold())
|
||||
|
||||
|
||||
def _has_useful_text(value: str | None) -> bool:
|
||||
cleaned = _clean_text(value)
|
||||
if cleaned is None:
|
||||
return False
|
||||
|
||||
token = _text_token(cleaned)
|
||||
return bool(token) and token not in _PLACEHOLDER_TEXT_TOKENS
|
||||
|
||||
|
||||
def _text_before_heading(node) -> str | None:
|
||||
if isinstance(node, NavigableString):
|
||||
return _clean_text(str(node))
|
||||
|
||||
if getattr(node, "name", None) in _HEADING_TAG_NAMES:
|
||||
return None
|
||||
|
||||
pieces: list[str] = []
|
||||
for child in getattr(node, "children", []):
|
||||
if getattr(child, "name", None) in _HEADING_TAG_NAMES:
|
||||
break
|
||||
|
||||
text = _text_before_heading(child)
|
||||
cleaned = _clean_text(text)
|
||||
if cleaned:
|
||||
pieces.append(cleaned)
|
||||
|
||||
return _clean_text(" ".join(pieces))
|
||||
|
||||
|
||||
def _extract_section_text(block, label: str) -> str | None:
|
||||
heading = block.find(lambda tag: getattr(tag, "name", None) in _HEADING_TAG_NAMES and _clean_text(tag.get_text(" ", strip=True)) == label)
|
||||
if heading is None:
|
||||
return None
|
||||
|
||||
pieces: list[str] = []
|
||||
for sibling in heading.next_siblings:
|
||||
if getattr(sibling, "name", None) in _HEADING_TAG_NAMES:
|
||||
break
|
||||
|
||||
text = _text_before_heading(sibling)
|
||||
if text:
|
||||
pieces.append(text)
|
||||
|
||||
return _clean_text(" ".join(pieces))
|
||||
|
||||
|
||||
def _detail_block_text(soup: BeautifulSoup, label: str) -> str | None:
|
||||
for block in soup.select(".details-post"):
|
||||
if block.find("h4") is None:
|
||||
continue
|
||||
|
||||
extracted = _extract_section_text(block, label)
|
||||
if extracted is not None:
|
||||
return extracted
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _warning(field: str, message: str) -> ListingWarning:
|
||||
return ListingWarning(field=field, message=message)
|
||||
|
||||
|
||||
def _extract_source_job_id_from_url(url: str) -> str | None:
|
||||
match = _DETAIL_JOB_ID_PATTERN.search(url)
|
||||
if match is None:
|
||||
return None
|
||||
|
||||
return match.group(1)
|
||||
|
||||
|
||||
def _extract_listing_date(
|
||||
soup: BeautifulSoup,
|
||||
pattern: re.Pattern[str],
|
||||
*,
|
||||
field: str,
|
||||
missing_message: str | None = None,
|
||||
invalid_message: str,
|
||||
warnings: list[ListingWarning],
|
||||
warn_on_missing: bool,
|
||||
) -> str | None:
|
||||
card_offer = soup.select_one(".card-offer")
|
||||
if card_offer is None:
|
||||
if warn_on_missing and missing_message is not None:
|
||||
warnings.append(_warning(field, missing_message))
|
||||
return None
|
||||
|
||||
match = pattern.search(card_offer.get_text(" ", strip=True))
|
||||
if match is None:
|
||||
if warn_on_missing and missing_message is not None:
|
||||
warnings.append(_warning(field, missing_message))
|
||||
return None
|
||||
|
||||
try:
|
||||
return datetime.strptime(match.group(1), "%d/%m/%Y").date().isoformat()
|
||||
except ValueError:
|
||||
warnings.append(_warning(field, invalid_message))
|
||||
return None
|
||||
|
||||
|
||||
def _extract_source_job_id(soup: BeautifulSoup) -> str | None:
|
||||
ref = soup.select_one(".ref-offre")
|
||||
if ref is None:
|
||||
return None
|
||||
|
||||
match = _SOURCE_JOB_ID_PATTERN.search(ref.get_text(" ", strip=True))
|
||||
if match is None:
|
||||
return None
|
||||
|
||||
return match.group(1)
|
||||
|
||||
|
||||
def _extract_contract_type(details_offer_list) -> str | None:
|
||||
contract_item = details_offer_list.select_one("li:nth-of-type(2)")
|
||||
if contract_item is None:
|
||||
return None
|
||||
|
||||
span = contract_item.find("span")
|
||||
if span is not None:
|
||||
return _clean_text(span.get_text(" ", strip=True))
|
||||
|
||||
match = _CONTRACT_PATTERN.search(contract_item.get_text(" ", strip=True))
|
||||
if match is None:
|
||||
return None
|
||||
|
||||
return match.group(1)
|
||||
|
||||
|
||||
def _extract_company(soup: BeautifulSoup, details_offer_list) -> str | None:
|
||||
for selector in (".card-ents .ents-name", ".card-ents-quote"):
|
||||
company = soup.select_one(selector)
|
||||
if company is not None:
|
||||
text = _clean_text(company.get_text(" ", strip=True))
|
||||
if text is not None:
|
||||
return text
|
||||
|
||||
if details_offer_list is not None:
|
||||
company = details_offer_list.select_one("li:first-of-type")
|
||||
if company is not None:
|
||||
text = _clean_text(company.get_text(" ", strip=True))
|
||||
if text is not None:
|
||||
return text
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def normalize_apec_listing(
|
||||
url: str,
|
||||
html: str,
|
||||
fetched_at: str,
|
||||
*,
|
||||
source_job_id: str | None = None,
|
||||
published_at: str | None = None,
|
||||
refreshed_at: str | None = None,
|
||||
) -> ApecListing:
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
warnings: list[ListingWarning] = []
|
||||
|
||||
title = soup.select_one(".container-details-offer h1")
|
||||
if title is None:
|
||||
title = soup.find("h1")
|
||||
if title is not None:
|
||||
warnings.append(_warning("title", "Recovered title from generic h1 fallback"))
|
||||
else:
|
||||
warnings.append(_warning("title", "Title missing from Apec listing"))
|
||||
title_text = _clean_text(title.get_text(" ", strip=True)) if title is not None else None
|
||||
if title is not None and not _has_useful_text(title_text):
|
||||
warnings.append(_warning("title", "Title is empty or placeholder text"))
|
||||
|
||||
details_offer_list = soup.select_one(".details-offer-list")
|
||||
|
||||
location = None
|
||||
contract_type = None
|
||||
if details_offer_list is not None:
|
||||
location_item = details_offer_list.select_one("li:nth-of-type(3)")
|
||||
if location_item is not None:
|
||||
location = _clean_text(location_item.get_text(" ", strip=True))
|
||||
if not _has_useful_text(location):
|
||||
warnings.append(_warning("location", "Location is empty or placeholder text"))
|
||||
else:
|
||||
warnings.append(_warning("location", "Location missing from details-offer list"))
|
||||
|
||||
contract_item = details_offer_list.select_one("li:nth-of-type(2)")
|
||||
if contract_item is None:
|
||||
warnings.append(_warning("contract_type", "Contract type missing from details-offer list"))
|
||||
else:
|
||||
span = contract_item.find("span")
|
||||
if span is not None:
|
||||
contract_type = _clean_text(span.get_text(" ", strip=True))
|
||||
if not _has_useful_text(contract_type):
|
||||
warnings.append(_warning("contract_type", "Contract type is empty or placeholder text"))
|
||||
else:
|
||||
match = _CONTRACT_PATTERN.search(contract_item.get_text(" ", strip=True))
|
||||
if match is not None:
|
||||
contract_type = match.group(1)
|
||||
warnings.append(_warning("contract_type", "Recovered contract type from text fallback"))
|
||||
else:
|
||||
warnings.append(_warning("contract_type", "Contract type missing from details-offer list"))
|
||||
else:
|
||||
warnings.append(_warning("location", "Location missing from Apec listing"))
|
||||
warnings.append(_warning("contract_type", "Contract type missing from Apec listing"))
|
||||
|
||||
description_text = _detail_block_text(soup, "Descriptif du poste")
|
||||
if description_text is None:
|
||||
warnings.append(_warning("description_text", "Description missing from Apec listing"))
|
||||
elif not _has_useful_text(description_text):
|
||||
warnings.append(_warning("description_text", "Description is empty or placeholder text"))
|
||||
description_text = None
|
||||
|
||||
requested_source_job_id = _extract_source_job_id_from_url(url)
|
||||
ref_source_job_id = _extract_source_job_id(soup)
|
||||
|
||||
if source_job_id is not None:
|
||||
if (
|
||||
requested_source_job_id is not None
|
||||
and ref_source_job_id is not None
|
||||
and requested_source_job_id != ref_source_job_id
|
||||
):
|
||||
warnings.append(_warning("source_job_id", "Recovered source job id from ref-offre fallback"))
|
||||
normalized_source_job_id = ref_source_job_id
|
||||
else:
|
||||
normalized_source_job_id = source_job_id
|
||||
else:
|
||||
if ref_source_job_id is None:
|
||||
if requested_source_job_id is None:
|
||||
warnings.append(_warning("source_job_id", "Source job id missing from Apec listing"))
|
||||
normalized_source_job_id = None
|
||||
else:
|
||||
warnings.append(_warning("source_job_id", "Recovered source job id from detail URL fallback"))
|
||||
normalized_source_job_id = requested_source_job_id
|
||||
else:
|
||||
warnings.append(_warning("source_job_id", "Recovered source job id from ref-offre fallback"))
|
||||
normalized_source_job_id = ref_source_job_id
|
||||
|
||||
company = soup.select_one(".card-ents .ents-name")
|
||||
if company is None:
|
||||
for selector, warning_message in (
|
||||
(".card-ents-quote", "Recovered company from .card-ents-quote fallback"),
|
||||
(".details-offer-list li:first-of-type", "Recovered company from details-offer-list fallback"),
|
||||
):
|
||||
company = soup.select_one(selector)
|
||||
if company is not None:
|
||||
warnings.append(_warning("company", warning_message))
|
||||
break
|
||||
|
||||
company_text = _clean_text(company.get_text(" ", strip=True)) if company is not None else None
|
||||
if company_text is None:
|
||||
warnings.append(_warning("company", "Company missing from Apec listing"))
|
||||
elif not _has_useful_text(company_text):
|
||||
warnings.append(_warning("company", "Company is empty or placeholder text"))
|
||||
company_text = None
|
||||
|
||||
published_at_value = published_at or _extract_listing_date(
|
||||
soup,
|
||||
_PUBLISHED_AT_PATTERN,
|
||||
field="published_at",
|
||||
missing_message="Published date missing from Apec listing",
|
||||
invalid_message="Published date is invalid",
|
||||
warnings=warnings,
|
||||
warn_on_missing=True,
|
||||
)
|
||||
|
||||
refreshed_at_value = refreshed_at or _extract_listing_date(
|
||||
soup,
|
||||
_REFRESHED_AT_PATTERN,
|
||||
field="refreshed_at",
|
||||
invalid_message="Refreshed date is invalid",
|
||||
warnings=warnings,
|
||||
warn_on_missing=False,
|
||||
)
|
||||
|
||||
return ApecListing(
|
||||
source="apec",
|
||||
source_job_id=normalized_source_job_id,
|
||||
url=url,
|
||||
title=title_text if _has_useful_text(title_text) else None,
|
||||
company=company_text,
|
||||
location=location if _has_useful_text(location) else None,
|
||||
contract_type=contract_type if _has_useful_text(contract_type) else None,
|
||||
description_text=description_text,
|
||||
published_at=published_at_value,
|
||||
refreshed_at=refreshed_at_value,
|
||||
fetched_at=fetched_at,
|
||||
warnings=warnings,
|
||||
)
|
||||
63
src/job_research/apec/query_derivation.py
Normal file
63
src/job_research/apec/query_derivation.py
Normal file
@ -0,0 +1,63 @@
|
||||
from job_research.apec.adapter import ApecSearchFilters
|
||||
from job_research.models import CandidateProfileOutput
|
||||
|
||||
|
||||
def _normalize_term(raw_term: str) -> str:
|
||||
return " ".join(raw_term.split())
|
||||
|
||||
|
||||
def _normalize_constraint(raw_term: str) -> str:
|
||||
return _normalize_term(raw_term).casefold()
|
||||
|
||||
|
||||
def derive_apec_search_filters(profile: CandidateProfileOutput) -> ApecSearchFilters:
|
||||
normalized_constraints = {_normalize_constraint(constraint) for constraint in profile.constraints}
|
||||
|
||||
return ApecSearchFilters(
|
||||
location="France" if "france only" in normalized_constraints else None,
|
||||
contract_type="CDI" if "cdi only" in normalized_constraints else None,
|
||||
)
|
||||
|
||||
|
||||
def derive_apec_queries(profile: CandidateProfileOutput) -> list[str]:
|
||||
queries: list[str] = []
|
||||
seen: set[str] = set()
|
||||
|
||||
def add_query(raw_query: str) -> None:
|
||||
query = _normalize_term(raw_query)
|
||||
if not query or query in seen or len(queries) == 5:
|
||||
return
|
||||
|
||||
seen.add(query)
|
||||
queries.append(query)
|
||||
|
||||
unique_roles: list[str] = []
|
||||
for target_role in profile.target_roles:
|
||||
query = " ".join(target_role.split())
|
||||
if not query or query in unique_roles:
|
||||
continue
|
||||
|
||||
unique_roles.append(query)
|
||||
|
||||
support_terms = [_normalize_term(term) for term in profile.strengths]
|
||||
support_terms.extend(_normalize_term(term) for term in profile.skills_to_emphasize)
|
||||
support_terms = [term for term in support_terms if term]
|
||||
|
||||
for target_role in unique_roles:
|
||||
add_query(target_role)
|
||||
if len(queries) == 5:
|
||||
return queries
|
||||
|
||||
if unique_roles:
|
||||
primary_role = unique_roles[0]
|
||||
for term in support_terms:
|
||||
add_query(f"{primary_role} {term}")
|
||||
if len(queries) == 5:
|
||||
break
|
||||
else:
|
||||
for term in support_terms:
|
||||
add_query(term)
|
||||
if len(queries) == 5:
|
||||
break
|
||||
|
||||
return queries
|
||||
278
src/job_research/cli.py
Normal file
278
src/job_research/cli.py
Normal file
@ -0,0 +1,278 @@
|
||||
from contextlib import nullcontext
|
||||
from datetime import datetime, timezone
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from urllib.parse import unquote, urlparse
|
||||
|
||||
import typer
|
||||
import yaml
|
||||
from pydantic import ValidationError
|
||||
|
||||
from job_research.apec.adapter import ApecAdapter, ApecSearchFilters
|
||||
from job_research.apec.dedupe import dedupe_apec_listings
|
||||
from job_research.apec.normalize import normalize_apec_listing
|
||||
from job_research.apec.query_derivation import derive_apec_queries, derive_apec_search_filters
|
||||
from job_research.models import ApecRunMeta, ApecSnapshotMeta, CandidateProfileOutput, ListingError
|
||||
from job_research.profile.cv_extractor import extract_cv_signals, extract_pdf_text
|
||||
from job_research.profile.merge import build_candidate_profile_output
|
||||
from job_research.profile.profile_parser import AuthoredProfile, parse_profile_markdown
|
||||
from job_research.storage import apec_run_paths, load_yaml, save_candidate_profile_yaml
|
||||
|
||||
app = typer.Typer(help="Build one canonical candidate profile YAML")
|
||||
|
||||
|
||||
def _utc_now() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def _snapshot_stem(url: str, source_job_id: str | None) -> str:
|
||||
if source_job_id:
|
||||
return source_job_id
|
||||
|
||||
parsed_url = urlparse(url)
|
||||
fallback = parsed_url.path.rstrip("/").rsplit("/", 1)[-1] or parsed_url.netloc or "listing"
|
||||
if parsed_url.query:
|
||||
fallback = f"{fallback}-{parsed_url.query}"
|
||||
|
||||
stem = re.sub(r"[^A-Za-z0-9]+", "-", unquote(fallback)).strip("-")
|
||||
return stem or "listing"
|
||||
|
||||
|
||||
def _write_yaml(path: Path, payload: Any) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(yaml.safe_dump(payload, sort_keys=False, allow_unicode=True), encoding="utf-8")
|
||||
|
||||
|
||||
def _load_candidate_profile(profile_path: Path) -> CandidateProfileOutput:
|
||||
try:
|
||||
return CandidateProfileOutput.model_validate(load_yaml(profile_path))
|
||||
except FileNotFoundError as exc:
|
||||
raise ValueError(f"candidate-profile.yaml not found at {profile_path}") from exc
|
||||
except (OSError, UnicodeDecodeError) as exc:
|
||||
raise ValueError(f"candidate-profile.yaml not readable at {profile_path}: {exc}") from exc
|
||||
except (yaml.YAMLError, ValidationError, ValueError) as exc:
|
||||
raise ValueError(f"invalid candidate-profile.yaml at {profile_path}: {exc}") from exc
|
||||
|
||||
|
||||
def _load_cv_text(cv: Path) -> str:
|
||||
try:
|
||||
cv_text = extract_pdf_text(cv) if cv.suffix.lower() == ".pdf" else cv.read_text(encoding="utf-8")
|
||||
except Exception as exc: # pragma: no cover - defensive boundary
|
||||
raise ValueError(f"CV input not readable at {cv}: {exc}") from exc
|
||||
|
||||
if not cv_text.strip():
|
||||
raise ValueError("No readable text found in CV input")
|
||||
|
||||
return cv_text
|
||||
|
||||
|
||||
def _load_authored_profile(profile: Path) -> AuthoredProfile:
|
||||
try:
|
||||
return parse_profile_markdown(profile.read_text(encoding="utf-8"))
|
||||
except Exception as exc: # pragma: no cover - defensive boundary
|
||||
raise ValueError(f"profile markdown invalid at {profile}: {exc}") from exc
|
||||
|
||||
|
||||
@app.callback()
|
||||
def main_command() -> None:
|
||||
pass
|
||||
|
||||
@app.command("build-profile")
|
||||
def build_profile(
|
||||
cv: Path = typer.Option(
|
||||
...,
|
||||
"--cv",
|
||||
exists=True,
|
||||
dir_okay=False,
|
||||
readable=True,
|
||||
help="Path to the CV PDF or UTF-8 text file.",
|
||||
),
|
||||
profile: Path = typer.Option(
|
||||
...,
|
||||
"--profile",
|
||||
exists=True,
|
||||
dir_okay=False,
|
||||
readable=True,
|
||||
help="Path to the light-template markdown profile.",
|
||||
),
|
||||
out: Path = typer.Option(
|
||||
Path("data/candidate-profile.yaml"),
|
||||
"--out",
|
||||
dir_okay=False,
|
||||
help="Path to write the canonical YAML profile.",
|
||||
),
|
||||
) -> None:
|
||||
"""Build candidate-profile.yaml from CV and markdown profile."""
|
||||
|
||||
try:
|
||||
cv_text = _load_cv_text(cv)
|
||||
authored_profile = _load_authored_profile(profile)
|
||||
except ValueError as exc:
|
||||
typer.echo(str(exc), err=True)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
cv_signals = extract_cv_signals(cv_text)
|
||||
candidate_profile = build_candidate_profile_output(cv_signals, authored_profile)
|
||||
|
||||
try:
|
||||
save_candidate_profile_yaml(out, candidate_profile)
|
||||
except OSError as exc:
|
||||
typer.echo(f"Unable to write candidate profile to {out}: {exc}", err=True)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
typer.echo(f"candidate profile written to {out}")
|
||||
warning_count = len(candidate_profile.warnings)
|
||||
if warning_count:
|
||||
typer.echo(f"Warnings included: {warning_count}")
|
||||
else:
|
||||
typer.echo("No warnings included.")
|
||||
|
||||
|
||||
@app.command("fetch-apec")
|
||||
def fetch_apec(
|
||||
data_root: Path = typer.Option(
|
||||
Path("data"),
|
||||
"--data-root",
|
||||
file_okay=False,
|
||||
dir_okay=True,
|
||||
help="Directory containing candidate-profile.yaml and Apec run artifacts.",
|
||||
),
|
||||
) -> None:
|
||||
"""Fetch, normalize, dedupe, and persist Apec listings."""
|
||||
|
||||
profile_path = data_root / "candidate-profile.yaml"
|
||||
try:
|
||||
profile = _load_candidate_profile(profile_path)
|
||||
except ValueError as exc:
|
||||
typer.echo(str(exc), err=True)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
derived_queries = derive_apec_queries(profile)
|
||||
|
||||
if not derived_queries:
|
||||
typer.echo("No usable Apec queries derived from candidate profile", err=True)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
derived_search_filters = derive_apec_search_filters(profile)
|
||||
search_filters = ApecSearchFilters(
|
||||
location=derived_search_filters.location or "France",
|
||||
contract_type=derived_search_filters.contract_type or "CDI",
|
||||
)
|
||||
|
||||
current = _utc_now().astimezone(timezone.utc)
|
||||
run_id = current.strftime("%Y-%m-%dT%H-%M-%S-%fZ")
|
||||
run_started_at = current.replace(microsecond=0).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
adapter = ApecAdapter(max_listings=50)
|
||||
try:
|
||||
search_results = adapter.search(derived_queries, search_filters=search_filters)[:50]
|
||||
except Exception as exc: # pragma: no cover - defensive boundary
|
||||
typer.echo(f"Unable to fetch Apec search results: {exc}", err=True)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
paths = apec_run_paths(data_root, run_id)
|
||||
try:
|
||||
paths["snapshots"].mkdir(parents=True, exist_ok=True)
|
||||
except OSError as exc: # pragma: no cover - defensive boundary
|
||||
typer.echo(f"Unable to create Apec snapshot directory: {exc}", err=True)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
normalized_listings = []
|
||||
listing_errors: list[ListingError] = list(getattr(adapter, "search_errors", []))
|
||||
snapshot_metadata: list[ApecSnapshotMeta] = []
|
||||
fetched_count = 0
|
||||
successful_fetch_count = 0
|
||||
|
||||
browser_session = getattr(adapter, "browser_session", None)
|
||||
session_context = browser_session() if callable(browser_session) else nullcontext()
|
||||
|
||||
with session_context:
|
||||
for result in search_results:
|
||||
fetched_count += 1
|
||||
fetched_at = _utc_now().astimezone(timezone.utc).replace(microsecond=0).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
try:
|
||||
html = adapter.fetch_listing_html(result.url)
|
||||
except Exception as exc: # pragma: no cover - defensive boundary
|
||||
listing_errors.append(ListingError(url=result.url, stage="fetch_html", message=str(exc)))
|
||||
continue
|
||||
|
||||
successful_fetch_count += 1
|
||||
|
||||
snapshot_path = paths["snapshots"] / f"{_snapshot_stem(result.url, result.source_job_id)}.html"
|
||||
snapshot_meta = ApecSnapshotMeta(
|
||||
url=result.url,
|
||||
source_job_id=result.source_job_id,
|
||||
snapshot_file=None,
|
||||
fetched_at=fetched_at,
|
||||
)
|
||||
|
||||
try:
|
||||
snapshot_path.write_text(html, encoding="utf-8")
|
||||
except Exception as exc: # pragma: no cover - defensive boundary
|
||||
listing_errors.append(ListingError(url=result.url, stage="snapshot_write", message=str(exc)))
|
||||
else:
|
||||
snapshot_meta.snapshot_file = snapshot_path.name
|
||||
|
||||
snapshot_metadata.append(snapshot_meta)
|
||||
|
||||
try:
|
||||
listing = normalize_apec_listing(
|
||||
url=result.url,
|
||||
html=html,
|
||||
fetched_at=fetched_at,
|
||||
source_job_id=result.source_job_id,
|
||||
)
|
||||
except Exception as exc: # pragma: no cover - defensive boundary
|
||||
listing_errors.append(ListingError(url=result.url, stage="normalize", message=str(exc)))
|
||||
continue
|
||||
|
||||
normalized_listings.append(listing)
|
||||
|
||||
if search_results and successful_fetch_count == 0:
|
||||
typer.echo("No listings could be fetched or normalized from Apec", err=True)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
deduplicated_listings = dedupe_apec_listings(normalized_listings)
|
||||
failed_count = len({error.url for error in listing_errors})
|
||||
run_meta = ApecRunMeta(
|
||||
run_id=run_id,
|
||||
run_started_at=run_started_at,
|
||||
derived_queries=derived_queries,
|
||||
snapshots=snapshot_metadata,
|
||||
fetched_count=fetched_count,
|
||||
normalized_count=len(normalized_listings),
|
||||
deduplicated_count=len(deduplicated_listings),
|
||||
failed_count=failed_count,
|
||||
listing_errors=listing_errors,
|
||||
)
|
||||
|
||||
artifact_write_errors: list[str] = []
|
||||
|
||||
try:
|
||||
_write_yaml(paths["listings"], [listing.model_dump(mode="json") for listing in deduplicated_listings])
|
||||
except OSError as exc: # pragma: no cover - defensive boundary
|
||||
artifact_write_errors.append(f"listings.yaml: {exc}")
|
||||
|
||||
try:
|
||||
_write_yaml(paths["run_meta"], run_meta.model_dump(mode="json"))
|
||||
except OSError as exc: # pragma: no cover - defensive boundary
|
||||
artifact_write_errors.append(f"run-meta.yaml: {exc}")
|
||||
|
||||
if artifact_write_errors:
|
||||
typer.echo(f"Unable to write Apec run artifacts: {'; '.join(artifact_write_errors)}", err=True)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
typer.echo(
|
||||
f"query={len(derived_queries)} fetched={fetched_count} normalized={len(normalized_listings)} "
|
||||
f"deduplicated={len(deduplicated_listings)} failed={failed_count}"
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
app()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
82
src/job_research/models.py
Normal file
82
src/job_research/models.py
Normal file
@ -0,0 +1,82 @@
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class ExperienceEntry(BaseModel):
|
||||
company: str
|
||||
title: str
|
||||
start: str | None = None
|
||||
end: str | None = None
|
||||
highlights: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class EducationEntry(BaseModel):
|
||||
institution: str
|
||||
credential: str
|
||||
start: str | None = None
|
||||
end: str | None = None
|
||||
|
||||
|
||||
class WarningItem(BaseModel):
|
||||
field: str
|
||||
message: str
|
||||
|
||||
|
||||
class ListingWarning(BaseModel):
|
||||
field: str
|
||||
message: str
|
||||
|
||||
|
||||
class ListingError(BaseModel):
|
||||
url: str
|
||||
stage: str
|
||||
message: str
|
||||
|
||||
|
||||
class ApecSnapshotMeta(BaseModel):
|
||||
url: str
|
||||
source_job_id: str | None = None
|
||||
snapshot_file: str | None = None
|
||||
fetched_at: str
|
||||
|
||||
|
||||
class ApecListing(BaseModel):
|
||||
source: str
|
||||
source_job_id: str | None = None
|
||||
url: str
|
||||
title: str | None = None
|
||||
company: str | None = None
|
||||
location: str | None = None
|
||||
contract_type: str | None = None
|
||||
description_text: str | None = None
|
||||
published_at: str | None = None
|
||||
refreshed_at: str | None = None
|
||||
fetched_at: str
|
||||
warnings: list[ListingWarning] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ApecRunMeta(BaseModel):
|
||||
run_id: str
|
||||
run_started_at: str
|
||||
derived_queries: list[str] = Field(default_factory=list)
|
||||
snapshots: list[ApecSnapshotMeta] = Field(default_factory=list)
|
||||
fetched_count: int = 0
|
||||
normalized_count: int = 0
|
||||
deduplicated_count: int = 0
|
||||
failed_count: int = 0
|
||||
listing_errors: list[ListingError] = Field(default_factory=list)
|
||||
|
||||
|
||||
class CandidateProfileOutput(BaseModel):
|
||||
name: str | None = None
|
||||
summary: str | None = None
|
||||
target_roles: list[str] = Field(default_factory=list)
|
||||
strengths: list[str] = Field(default_factory=list)
|
||||
skills_to_emphasize: list[str] = Field(default_factory=list)
|
||||
constraints: list[str] = Field(default_factory=list)
|
||||
notes: list[str] = Field(default_factory=list)
|
||||
location: str | None = None
|
||||
languages: list[str] = Field(default_factory=list)
|
||||
skills: list[str] = Field(default_factory=list)
|
||||
experience_entries: list[ExperienceEntry] = Field(default_factory=list)
|
||||
education_entries: list[EducationEntry] = Field(default_factory=list)
|
||||
warnings: list[WarningItem] = Field(default_factory=list)
|
||||
1
src/job_research/profile/__init__.py
Normal file
1
src/job_research/profile/__init__.py
Normal file
@ -0,0 +1 @@
|
||||
__all__ = ["profile_parser"]
|
||||
403
src/job_research/profile/cv_extractor.py
Normal file
403
src/job_research/profile/cv_extractor.py
Normal file
@ -0,0 +1,403 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from pypdf import PdfReader
|
||||
|
||||
from job_research.models import WarningItem
|
||||
|
||||
|
||||
EXPERIENCE_LINE_CONNECTORS = (" at ", " chez ", " au ", " à ")
|
||||
|
||||
EXPERIENCE_TITLE_STOPWORDS = {
|
||||
"a",
|
||||
"an",
|
||||
"and",
|
||||
"as",
|
||||
"at",
|
||||
"after",
|
||||
"before",
|
||||
"by",
|
||||
"for",
|
||||
"from",
|
||||
"in",
|
||||
"into",
|
||||
"of",
|
||||
"on",
|
||||
"or",
|
||||
"the",
|
||||
"to",
|
||||
"with",
|
||||
"within",
|
||||
"without",
|
||||
"while",
|
||||
"during",
|
||||
"since",
|
||||
"because",
|
||||
}
|
||||
|
||||
EXPERIENCE_TITLE_ACTION_STARTS = {
|
||||
"built",
|
||||
"created",
|
||||
"delivered",
|
||||
"designed",
|
||||
"developed",
|
||||
"deployed",
|
||||
"implemented",
|
||||
"improved",
|
||||
"managed",
|
||||
"migrated",
|
||||
"maintained",
|
||||
"worked",
|
||||
}
|
||||
|
||||
EXPERIENCE_TITLE_LABELS = {
|
||||
"education",
|
||||
"experience",
|
||||
"formation",
|
||||
"languages",
|
||||
"langues",
|
||||
"location",
|
||||
"profile",
|
||||
"skills",
|
||||
"compétences",
|
||||
"competences",
|
||||
"summary",
|
||||
}
|
||||
|
||||
EXPERIENCE_PROSE_MARKERS = {
|
||||
"after",
|
||||
"before",
|
||||
"because",
|
||||
"during",
|
||||
"from",
|
||||
"joined",
|
||||
"joining",
|
||||
"left",
|
||||
"leaving",
|
||||
"in",
|
||||
"moved",
|
||||
"moving",
|
||||
"relocated",
|
||||
"relocating",
|
||||
"since",
|
||||
"studying",
|
||||
"then",
|
||||
"to",
|
||||
"toward",
|
||||
"towards",
|
||||
"transferring",
|
||||
"transitioning",
|
||||
"until",
|
||||
"while",
|
||||
"working",
|
||||
"work",
|
||||
"worked",
|
||||
}
|
||||
|
||||
EXPERIENCE_PROSE_COMPANY_PATTERNS = (
|
||||
re.compile(r"\bau\s+sein\s+de\b", re.IGNORECASE),
|
||||
re.compile(r"\b(before|after|during|while|since|because)\b", re.IGNORECASE),
|
||||
re.compile(r"\b(joined|joining|moved|moving|relocated|relocating|worked|working)\b", re.IGNORECASE),
|
||||
re.compile(
|
||||
r"\bin\s+(?:paris|london|lyon|france|berlin|amsterdam|madrid|rome|marseille|bordeaux|toulouse|nantes|lille|grenoble|strasbourg|nice|rennes|montpellier|remote)\b",
|
||||
re.IGNORECASE,
|
||||
),
|
||||
)
|
||||
|
||||
EDUCATION_TITLE_KEYWORDS = {
|
||||
"bachelor",
|
||||
"degree",
|
||||
"diploma",
|
||||
"diplome d'ingenieur",
|
||||
"diplome d'ingénieur",
|
||||
"diplôme d'ingénieur",
|
||||
"diplôme d'ingenieur",
|
||||
"engineering school",
|
||||
"bac+5",
|
||||
"licence",
|
||||
"bsc",
|
||||
"master",
|
||||
"msc",
|
||||
}
|
||||
|
||||
EDUCATION_INSTITUTION_MARKERS = {
|
||||
"academy",
|
||||
"college",
|
||||
"centrale",
|
||||
"ecole",
|
||||
"école",
|
||||
"ens",
|
||||
"epita",
|
||||
"epitech",
|
||||
"hec",
|
||||
"imt",
|
||||
"institute",
|
||||
"institut",
|
||||
"insa",
|
||||
"polytech",
|
||||
"school",
|
||||
"university",
|
||||
"universite",
|
||||
"université",
|
||||
}
|
||||
|
||||
EDUCATION_ENTRY_SEPARATORS = (" at ", ", ", " - ", " – ", " — ")
|
||||
|
||||
LEADING_BULLET_MARKERS = {"-", "*", "+", "•", "‣", "∙", "●", "◦", "–", "—"}
|
||||
|
||||
YEARS_OF_EXPERIENCE_PATTERNS = (
|
||||
re.compile(r"^years of experience\s*:\s*(\d+)\s*$", re.IGNORECASE),
|
||||
re.compile(r"^ann[ée]es d['’]exp[ée]rience\s*:\s*(\d+)\s*$", re.IGNORECASE),
|
||||
)
|
||||
|
||||
LOW_CONFIDENCE_NAME_PATTERNS = (
|
||||
re.compile(r"\b(cv|resume|curriculum vitae|profile)\b", re.IGNORECASE),
|
||||
re.compile(r"[|/@]"),
|
||||
re.compile(
|
||||
r"\b(data engineer|software engineer|developer|analyst|scientist|consultant|architect|manager|product owner|backend|frontend|full stack)\b",
|
||||
re.IGNORECASE,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def extract_pdf_text(path: Path) -> str:
|
||||
reader = PdfReader(str(path))
|
||||
page_texts = [
|
||||
text.strip()
|
||||
for text in (page.extract_text() or "" for page in reader.pages)
|
||||
if text.strip()
|
||||
]
|
||||
|
||||
if not page_texts:
|
||||
raise ValueError("No extractable text found in PDF CV")
|
||||
|
||||
return "\n".join(page_texts)
|
||||
|
||||
|
||||
def extract_cv_signals(text: str) -> dict[str, object]:
|
||||
lines = [_normalize_line(line) for line in text.splitlines()]
|
||||
non_empty_lines = [line for line in lines if line]
|
||||
warnings: list[WarningItem] = []
|
||||
|
||||
name = non_empty_lines[0] if non_empty_lines else None
|
||||
if name is not None and _looks_like_low_confidence_name(name):
|
||||
warnings.append(
|
||||
WarningItem(
|
||||
field="name",
|
||||
message="First CV line looks like a header or tagline; review manually.",
|
||||
)
|
||||
)
|
||||
|
||||
location = None
|
||||
languages: list[str] = []
|
||||
skills: list[str] = []
|
||||
experience_entries: list[dict[str, str]] = []
|
||||
education_entries: list[dict[str, str]] = []
|
||||
years_of_experience: int | None = None
|
||||
in_education_section = False
|
||||
pending_education_credential: str | None = None
|
||||
|
||||
for line in non_empty_lines[1:]:
|
||||
lowered = line.lower()
|
||||
|
||||
years_of_experience_line = _parse_years_of_experience(line)
|
||||
if years_of_experience_line is not None:
|
||||
years_of_experience = years_of_experience_line
|
||||
continue
|
||||
|
||||
if lowered.startswith(("education:", "formation:")):
|
||||
in_education_section = True
|
||||
pending_education_credential = None
|
||||
remainder = line.split(":", 1)[1].strip()
|
||||
if remainder:
|
||||
education_entry = _parse_education_entry(remainder)
|
||||
if education_entry:
|
||||
education_entries.append(education_entry)
|
||||
elif _looks_like_education_credential(remainder):
|
||||
pending_education_credential = remainder
|
||||
continue
|
||||
|
||||
if in_education_section:
|
||||
if pending_education_credential and _looks_like_institution(line):
|
||||
education_entries.append(
|
||||
{
|
||||
"credential": pending_education_credential,
|
||||
"institution": line,
|
||||
}
|
||||
)
|
||||
pending_education_credential = None
|
||||
continue
|
||||
|
||||
education_entry = _parse_education_entry(line)
|
||||
if education_entry:
|
||||
education_entries.append(education_entry)
|
||||
pending_education_credential = None
|
||||
continue
|
||||
|
||||
if _looks_like_education_credential(line):
|
||||
pending_education_credential = line
|
||||
continue
|
||||
|
||||
pending_education_credential = None
|
||||
in_education_section = False
|
||||
|
||||
if lowered.startswith("location:"):
|
||||
location = line.split(":", 1)[1].strip() or None
|
||||
continue
|
||||
if lowered.startswith(("languages:", "langues:")):
|
||||
languages = _parse_csv_field(line)
|
||||
continue
|
||||
if lowered.startswith(("skills:", "compétences:")):
|
||||
skills = _parse_csv_field(line)
|
||||
continue
|
||||
education_entry = _parse_education_entry(line)
|
||||
if education_entry:
|
||||
education_entries.append(education_entry)
|
||||
continue
|
||||
experience_entry = _parse_experience_entry(line)
|
||||
if experience_entry:
|
||||
experience_entries.append(experience_entry)
|
||||
|
||||
payload = {
|
||||
"name": name,
|
||||
"location": location,
|
||||
"languages": languages,
|
||||
"skills": skills,
|
||||
"experience_entries": experience_entries,
|
||||
"education_entries": education_entries,
|
||||
"warnings": warnings,
|
||||
}
|
||||
|
||||
if years_of_experience is not None:
|
||||
payload["years_of_experience"] = years_of_experience
|
||||
|
||||
return payload
|
||||
|
||||
|
||||
def _parse_csv_field(line: str) -> list[str]:
|
||||
_, value = line.split(":", 1)
|
||||
return [item.strip() for item in value.split(",") if item.strip()]
|
||||
|
||||
|
||||
def _normalize_line(line: str) -> str:
|
||||
stripped = line.strip()
|
||||
if not stripped:
|
||||
return ""
|
||||
|
||||
if stripped[0] in LEADING_BULLET_MARKERS:
|
||||
return stripped[1:].lstrip()
|
||||
|
||||
return stripped
|
||||
|
||||
|
||||
def _parse_years_of_experience(line: str) -> int | None:
|
||||
normalized = line.replace("’", "'")
|
||||
|
||||
for pattern in YEARS_OF_EXPERIENCE_PATTERNS:
|
||||
match = pattern.match(normalized)
|
||||
if match:
|
||||
return int(match.group(1))
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _parse_education_entry(line: str) -> dict[str, str] | None:
|
||||
for separator in EDUCATION_ENTRY_SEPARATORS:
|
||||
if separator not in line:
|
||||
continue
|
||||
|
||||
left, right = (part.strip() for part in line.split(separator, 1))
|
||||
if not left or not right:
|
||||
continue
|
||||
|
||||
if _looks_like_education_credential(left) and _looks_like_institution(right):
|
||||
return {"credential": left, "institution": right}
|
||||
|
||||
if _looks_like_education_credential(right) and _looks_like_institution(left):
|
||||
return {"credential": right, "institution": left}
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _looks_like_education_credential(text: str) -> bool:
|
||||
lowered = text.lower().replace(".", "")
|
||||
return any(
|
||||
re.search(rf"\b{re.escape(keyword)}\b", lowered)
|
||||
for keyword in EDUCATION_TITLE_KEYWORDS
|
||||
)
|
||||
|
||||
|
||||
def _looks_like_institution(text: str) -> bool:
|
||||
lowered = text.lower()
|
||||
return any(marker in lowered for marker in EDUCATION_INSTITUTION_MARKERS)
|
||||
|
||||
|
||||
def _parse_experience_entry(line: str) -> dict[str, str] | None:
|
||||
if " au sein " in line.lower():
|
||||
return None
|
||||
|
||||
for connector in EXPERIENCE_LINE_CONNECTORS:
|
||||
if line.count(connector) != 1:
|
||||
continue
|
||||
|
||||
title, company = (part.strip() for part in line.split(connector, 1))
|
||||
if not title or not company:
|
||||
continue
|
||||
|
||||
if not _looks_like_experience_title(title):
|
||||
continue
|
||||
|
||||
if _looks_like_prose_company(company):
|
||||
continue
|
||||
|
||||
return {
|
||||
"title": title,
|
||||
"company": company,
|
||||
}
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _looks_like_experience_line(line: str) -> bool:
|
||||
return _parse_experience_entry(line) is not None
|
||||
|
||||
|
||||
def _looks_like_experience_title(title: str) -> bool:
|
||||
title_words = [word.strip(".,;:!?()[]{}") for word in title.split()]
|
||||
if not title_words:
|
||||
return False
|
||||
|
||||
first_word = title_words[0]
|
||||
normalized_title = " ".join(title_words).casefold()
|
||||
if normalized_title in EXPERIENCE_TITLE_LABELS:
|
||||
return False
|
||||
|
||||
if first_word.casefold() in EXPERIENCE_TITLE_STOPWORDS:
|
||||
return False
|
||||
|
||||
if first_word.casefold() in EXPERIENCE_TITLE_ACTION_STARTS:
|
||||
return False
|
||||
|
||||
if len(title_words) == 1 and len(first_word) < 2:
|
||||
return False
|
||||
|
||||
return not any(
|
||||
word.casefold() in EXPERIENCE_PROSE_MARKERS for word in title_words if word
|
||||
)
|
||||
|
||||
|
||||
def _looks_like_prose_company(company: str) -> bool:
|
||||
return any(pattern.search(company) for pattern in EXPERIENCE_PROSE_COMPANY_PATTERNS)
|
||||
|
||||
|
||||
def _looks_like_low_confidence_name(name: str) -> bool:
|
||||
normalized = " ".join(name.split())
|
||||
if not normalized:
|
||||
return True
|
||||
|
||||
if len(normalized.split()) > 4:
|
||||
return True
|
||||
|
||||
return any(pattern.search(normalized) for pattern in LOW_CONFIDENCE_NAME_PATTERNS)
|
||||
121
src/job_research/profile/merge.py
Normal file
121
src/job_research/profile/merge.py
Normal file
@ -0,0 +1,121 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
from job_research.models import (
|
||||
CandidateProfileOutput,
|
||||
EducationEntry,
|
||||
ExperienceEntry,
|
||||
WarningItem,
|
||||
)
|
||||
from job_research.profile.profile_parser import AuthoredProfile
|
||||
|
||||
|
||||
EXPERIENCE_NOTE_MARKERS = (
|
||||
"experience",
|
||||
"expérience",
|
||||
"années d'expérience",
|
||||
"annees d'experience",
|
||||
)
|
||||
|
||||
|
||||
MISSING_CV_FACT_WARNINGS = (
|
||||
("name", "No candidate name was extracted from the CV."),
|
||||
("experience_entries", "No experience entries were extracted from the CV."),
|
||||
("skills", "No skills were extracted from the CV."),
|
||||
("education_entries", "No education entries were extracted from the CV."),
|
||||
)
|
||||
|
||||
|
||||
def build_candidate_profile_output(
|
||||
cv_signals: dict[str, object], authored: AuthoredProfile
|
||||
) -> CandidateProfileOutput:
|
||||
warnings: list[WarningItem] = []
|
||||
|
||||
_append_years_of_experience_warning(cv_signals, authored.notes, warnings)
|
||||
_append_cv_extraction_warnings(cv_signals, warnings)
|
||||
_append_missing_cv_fact_warnings(cv_signals, warnings)
|
||||
|
||||
merged_skills: list[str] = []
|
||||
for skill in [
|
||||
*(cv_signals.get("skills") or []),
|
||||
*authored.strengths,
|
||||
*authored.skills_to_emphasize,
|
||||
]:
|
||||
if skill not in merged_skills:
|
||||
merged_skills.append(skill)
|
||||
|
||||
experience_entries = cv_signals.get("experience_entries") or []
|
||||
education_entries = cv_signals.get("education_entries") or []
|
||||
|
||||
return CandidateProfileOutput(
|
||||
name=cv_signals.get("name"),
|
||||
summary=authored.summary,
|
||||
target_roles=authored.target_roles,
|
||||
strengths=authored.strengths,
|
||||
skills_to_emphasize=authored.skills_to_emphasize,
|
||||
constraints=authored.constraints,
|
||||
notes=authored.notes,
|
||||
location=cv_signals.get("location"),
|
||||
languages=cv_signals.get("languages") or [],
|
||||
skills=merged_skills,
|
||||
experience_entries=[ExperienceEntry.model_validate(item) for item in experience_entries],
|
||||
education_entries=[EducationEntry.model_validate(item) for item in education_entries],
|
||||
warnings=warnings,
|
||||
)
|
||||
|
||||
|
||||
def _append_years_of_experience_warning(
|
||||
cv_signals: dict[str, object], notes: list[str], warnings: list[WarningItem]
|
||||
) -> None:
|
||||
years_of_experience = cv_signals.get("years_of_experience")
|
||||
if years_of_experience is None:
|
||||
return
|
||||
|
||||
for note in notes:
|
||||
note_years_of_experience = _note_years_of_experience(note)
|
||||
if note_years_of_experience is None or note_years_of_experience == years_of_experience:
|
||||
continue
|
||||
|
||||
warnings.append(
|
||||
WarningItem(
|
||||
field="years_of_experience",
|
||||
message=(
|
||||
"CV-derived years of experience does not match a markdown note. "
|
||||
"Review manually."
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
return
|
||||
|
||||
|
||||
def _append_missing_cv_fact_warnings(
|
||||
cv_signals: dict[str, object], warnings: list[WarningItem]
|
||||
) -> None:
|
||||
for field, message in MISSING_CV_FACT_WARNINGS:
|
||||
if cv_signals.get(field):
|
||||
continue
|
||||
|
||||
warnings.append(WarningItem(field=field, message=message))
|
||||
|
||||
|
||||
def _append_cv_extraction_warnings(
|
||||
cv_signals: dict[str, object], warnings: list[WarningItem]
|
||||
) -> None:
|
||||
for warning in cv_signals.get("warnings") or []:
|
||||
warnings.append(
|
||||
warning if isinstance(warning, WarningItem) else WarningItem.model_validate(warning)
|
||||
)
|
||||
|
||||
|
||||
def _note_years_of_experience(note: str) -> int | None:
|
||||
normalized = note.casefold().replace("’", "'")
|
||||
if not any(marker in normalized for marker in EXPERIENCE_NOTE_MARKERS):
|
||||
return None
|
||||
|
||||
matches = re.findall(r"\b(\d{1,2})\b", normalized)
|
||||
if len(matches) != 1:
|
||||
return None
|
||||
|
||||
return int(matches[0])
|
||||
102
src/job_research/profile/profile_parser.py
Normal file
102
src/job_research/profile/profile_parser.py
Normal file
@ -0,0 +1,102 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import defaultdict
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
|
||||
LIST_MARKERS = ("- ", "* ", "+ ")
|
||||
|
||||
|
||||
@dataclass
|
||||
class AuthoredProfile:
|
||||
summary: str | None = None
|
||||
target_roles: list[str] = field(default_factory=list)
|
||||
strengths: list[str] = field(default_factory=list)
|
||||
skills_to_emphasize: list[str] = field(default_factory=list)
|
||||
constraints: list[str] = field(default_factory=list)
|
||||
notes: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
REQUIRED_SECTION_NAMES = (
|
||||
"summary",
|
||||
"target roles",
|
||||
"strengths",
|
||||
"skills to emphasize",
|
||||
"constraints",
|
||||
"notes",
|
||||
)
|
||||
REQUIRED_SECTIONS = set(REQUIRED_SECTION_NAMES)
|
||||
|
||||
|
||||
def parse_profile_markdown(markdown: str) -> AuthoredProfile:
|
||||
sections: dict[str, list[str]] = defaultdict(list)
|
||||
current_section: str | None = None
|
||||
|
||||
for raw_line in markdown.splitlines():
|
||||
line = raw_line.strip()
|
||||
if line.startswith("## "):
|
||||
current_section = line[3:].strip().lower()
|
||||
continue
|
||||
if not line or current_section is None:
|
||||
continue
|
||||
sections[current_section].append(line)
|
||||
|
||||
missing = REQUIRED_SECTIONS - set(sections)
|
||||
if missing:
|
||||
missing_text = ", ".join(sorted(missing))
|
||||
raise ValueError(f"Missing required markdown sections: {missing_text}")
|
||||
|
||||
for section_name in REQUIRED_SECTION_NAMES:
|
||||
if not _has_usable_section_content(sections[section_name]):
|
||||
raise ValueError(f"Missing usable content in section '{section_name}'")
|
||||
|
||||
return AuthoredProfile(
|
||||
summary=" ".join(sections["summary"]),
|
||||
target_roles=_parse_list_section("target roles", sections["target roles"]),
|
||||
strengths=_parse_list_section("strengths", sections["strengths"]),
|
||||
skills_to_emphasize=_parse_list_section(
|
||||
"skills to emphasize", sections["skills to emphasize"]
|
||||
),
|
||||
constraints=_parse_list_section("constraints", sections["constraints"]),
|
||||
notes=_parse_notes_section(sections["notes"]),
|
||||
)
|
||||
|
||||
|
||||
def _parse_list_section(section_name: str, lines: list[str]) -> list[str]:
|
||||
items: list[str] = []
|
||||
|
||||
for line in lines:
|
||||
item = _strip_list_marker(line)
|
||||
if item is None:
|
||||
raise ValueError(f"Unsupported content in section '{section_name}': {line}")
|
||||
if not item:
|
||||
raise ValueError(f"Missing usable content in section '{section_name}'")
|
||||
items.append(item)
|
||||
|
||||
return items
|
||||
|
||||
|
||||
def _parse_notes_section(lines: list[str]) -> list[str]:
|
||||
notes: list[str] = []
|
||||
|
||||
for line in lines:
|
||||
item = _strip_list_marker(line)
|
||||
if item == "":
|
||||
raise ValueError("Missing usable content in section 'notes'")
|
||||
notes.append(item if item is not None else line)
|
||||
|
||||
return notes
|
||||
|
||||
|
||||
def _strip_list_marker(line: str) -> str | None:
|
||||
for marker in LIST_MARKERS:
|
||||
if line == marker.strip():
|
||||
return ""
|
||||
if line.startswith(marker):
|
||||
return line[len(marker):].strip()
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _has_usable_section_content(lines: list[str]) -> bool:
|
||||
return any(line not in {"-", "*", "+"} for line in lines)
|
||||
33
src/job_research/storage.py
Normal file
33
src/job_research/storage.py
Normal file
@ -0,0 +1,33 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
from job_research.models import CandidateProfileOutput
|
||||
|
||||
|
||||
def save_candidate_profile_yaml(path: Path, profile: CandidateProfileOutput) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
payload = profile.model_dump(mode="json")
|
||||
path.write_text(yaml.safe_dump(payload, sort_keys=False, allow_unicode=True), encoding="utf-8")
|
||||
|
||||
|
||||
def load_yaml(path: Path) -> dict[str, Any]:
|
||||
payload = yaml.safe_load(path.read_text(encoding="utf-8"))
|
||||
if not isinstance(payload, Mapping):
|
||||
raise ValueError("candidate-profile YAML root must be a mapping")
|
||||
|
||||
return dict(payload)
|
||||
|
||||
|
||||
def apec_run_paths(data_root: Path, run_id: str) -> dict[str, Path]:
|
||||
run_dir = data_root / "apec" / "runs" / run_id
|
||||
return {
|
||||
"run_dir": run_dir,
|
||||
"listings": run_dir / "listings.yaml",
|
||||
"run_meta": run_dir / "run-meta.yaml",
|
||||
"snapshots": run_dir / "snapshots",
|
||||
}
|
||||
701
tests/apec/test_adapter.py
Normal file
701
tests/apec/test_adapter.py
Normal file
@ -0,0 +1,701 @@
|
||||
from contextlib import contextmanager
|
||||
from urllib.parse import parse_qs, quote_plus, urlparse
|
||||
|
||||
import pytest
|
||||
from playwright.sync_api import Error as PlaywrightError
|
||||
from playwright.sync_api import TimeoutError as PlaywrightTimeoutError
|
||||
|
||||
from job_research.apec import adapter as adapter_module
|
||||
from job_research.apec.adapter import ApecAdapter, ApecSearchFilters
|
||||
|
||||
|
||||
_RESULT_LINK_SELECTOR = "a[href*='/candidat/recherche-emploi.html/emploi/detail-offre/']"
|
||||
_SEARCH_INPUT_SELECTOR = 'input[name="keywords"]'
|
||||
|
||||
|
||||
class _FakeResultButton:
|
||||
def __init__(self, page, name: str) -> None:
|
||||
self.page = page
|
||||
self.name = name
|
||||
|
||||
def click(self, timeout: int | None = None) -> None:
|
||||
if self.name == "Rechercher":
|
||||
self.page.url = (
|
||||
"https://www.apec.fr/candidat/recherche-emploi.html/emploi"
|
||||
f"?motsCles={quote_plus(self.page.current_query)}&page=0"
|
||||
)
|
||||
self.page.current_page = 0
|
||||
elif self.name in {"ACCEPTER", "Accepter tous les cookies"}:
|
||||
self.page.consent_button_clicks.append(self.name)
|
||||
self.page.consent_accepted = True
|
||||
|
||||
|
||||
class _FakeLocator:
|
||||
def __init__(self, page, selector: str) -> None:
|
||||
self.page = page
|
||||
self.selector = selector
|
||||
|
||||
def fill(self, value: str) -> None:
|
||||
self.page.current_query = value
|
||||
|
||||
def check(self, timeout: int | None = None) -> None:
|
||||
if self.selector == 'input[name="cguAcceptees"]':
|
||||
self.page.cgu_checkbox_checked = True
|
||||
return None
|
||||
|
||||
raise PlaywrightTimeoutError(f"selector not found: {self.selector}")
|
||||
|
||||
def evaluate_all(self, function: str):
|
||||
if self.selector == _RESULT_LINK_SELECTOR:
|
||||
return list(self.page.current_results())
|
||||
|
||||
return []
|
||||
|
||||
|
||||
class _FakeDetailPage:
|
||||
def __init__(
|
||||
self,
|
||||
result_pages: dict[str, dict[int, list[str]]],
|
||||
*,
|
||||
rendered_html: str = "<html>rendered</html>",
|
||||
search_ready: bool = True,
|
||||
zero_result_queries: set[str] | None = None,
|
||||
consent_required: bool = False,
|
||||
) -> None:
|
||||
self.result_pages = result_pages
|
||||
self.rendered_html = rendered_html
|
||||
self.shell_html = "<html>shell</html>"
|
||||
self.waited_functions: list[tuple[str, int | None]] = []
|
||||
self.search_ready = search_ready
|
||||
self.zero_result_queries = zero_result_queries or set()
|
||||
self.consent_required = consent_required
|
||||
self.cgu_checkbox_checked = False
|
||||
self.consent_button_clicks: list[str] = []
|
||||
self.consent_accepted = not consent_required
|
||||
self.goto_urls: list[str] = []
|
||||
self.current_query = ""
|
||||
self.current_page = 0
|
||||
self.url = ""
|
||||
self.rendered = False
|
||||
self.default_timeout: int | None = None
|
||||
self.closed = False
|
||||
|
||||
def goto(self, url: str, wait_until: str | None = None) -> None:
|
||||
self.goto_urls.append(url)
|
||||
self.url = url
|
||||
|
||||
parsed_url = urlparse(url)
|
||||
params = parse_qs(parsed_url.query)
|
||||
if "motsCles" in params:
|
||||
self.current_query = params["motsCles"][0]
|
||||
if "page" in params:
|
||||
self.current_page = int(params["page"][0])
|
||||
|
||||
if self.current_query in self.zero_result_queries and "/detail-offre/" not in parsed_url.path:
|
||||
self.url = (
|
||||
f"{parsed_url.scheme}://{parsed_url.netloc}"
|
||||
f"{parsed_url.path}/recherche-avancee?{parsed_url.query}&error=true"
|
||||
)
|
||||
|
||||
if "/detail-offre/" in parsed_url.path:
|
||||
self.rendered = False
|
||||
|
||||
def wait_for_load_state(self, state: str) -> None:
|
||||
return None
|
||||
|
||||
def set_default_timeout(self, timeout: int) -> None:
|
||||
self.default_timeout = timeout
|
||||
|
||||
def wait_for_function(self, function: str, polling: int | None = None, timeout: int | None = None) -> None:
|
||||
if self.consent_required and not self.consent_accepted:
|
||||
raise PlaywrightTimeoutError("consent not accepted")
|
||||
|
||||
self.waited_functions.append((function, polling))
|
||||
self.rendered = True
|
||||
return None
|
||||
|
||||
def wait_for_selector(self, selector: str, timeout: int | None = None) -> None:
|
||||
if selector == _SEARCH_INPUT_SELECTOR:
|
||||
if self.search_ready and self.consent_accepted:
|
||||
return None
|
||||
|
||||
raise PlaywrightTimeoutError(f"selector not found: {selector}")
|
||||
|
||||
if selector == _RESULT_LINK_SELECTOR and self.current_results():
|
||||
return None
|
||||
|
||||
raise PlaywrightTimeoutError(f"selector not found: {selector}")
|
||||
|
||||
def get_by_role(self, role: str, name: str):
|
||||
return _FakeResultButton(self, name)
|
||||
|
||||
def locator(self, selector: str):
|
||||
return _FakeLocator(self, selector)
|
||||
|
||||
def content(self) -> str:
|
||||
return self.rendered_html if self.rendered else self.shell_html
|
||||
|
||||
def current_results(self) -> list[str]:
|
||||
return self.result_pages.get(self.current_query, {}).get(self.current_page, [])
|
||||
|
||||
def close(self) -> None:
|
||||
self.closed = True
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _fake_open_public_page(page: _FakeDetailPage):
|
||||
yield page
|
||||
|
||||
|
||||
def test_search_continues_past_duplicate_only_pages(monkeypatch) -> None:
|
||||
first_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=0&selectedIndex=0"
|
||||
second_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/222?motsCles=beta&page=1&selectedIndex=0"
|
||||
page = _FakeDetailPage(
|
||||
{
|
||||
"alpha": {0: [first_result], 1: []},
|
||||
"beta": {0: [first_result], 1: [second_result], 2: []},
|
||||
}
|
||||
)
|
||||
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
results = ApecAdapter(max_listings=10).search(
|
||||
["alpha", "beta"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.url for result in results] == [first_result, second_result]
|
||||
assert [result.source_job_id for result in results] == ["111", "222"]
|
||||
assert "motsCles=alpha" in page.goto_urls[0]
|
||||
assert "lieux=799" in page.goto_urls[0]
|
||||
assert "typesContrat=101888" in page.goto_urls[0]
|
||||
assert any(
|
||||
"motsCles=beta" in url and "lieux=799" in url and "typesContrat=101888" in url and "page=1" in url
|
||||
for url in page.goto_urls
|
||||
)
|
||||
assert any("page=1" in url for url in page.goto_urls)
|
||||
|
||||
|
||||
def test_search_continues_past_duplicate_only_pages_until_a_later_hit(monkeypatch) -> None:
|
||||
first_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=0&selectedIndex=0"
|
||||
second_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/222?motsCles=beta&page=3&selectedIndex=0"
|
||||
page = _FakeDetailPage(
|
||||
{
|
||||
"alpha": {0: [first_result], 1: [first_result], 2: [first_result], 3: [second_result], 4: []},
|
||||
"beta": {0: [first_result], 1: []},
|
||||
}
|
||||
)
|
||||
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
results = ApecAdapter(max_listings=10).search(
|
||||
["alpha", "beta"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.url for result in results] == [first_result, second_result]
|
||||
assert any("page=1" in url for url in page.goto_urls)
|
||||
assert any("page=3" in url for url in page.goto_urls)
|
||||
|
||||
|
||||
def test_search_continues_after_query_and_pagination_navigation_failures(monkeypatch) -> None:
|
||||
first_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=beta&page=0&selectedIndex=0"
|
||||
second_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/222?motsCles=beta&page=1&selectedIndex=0"
|
||||
page = _FakeDetailPage(
|
||||
{
|
||||
"beta": {0: [first_result], 1: [second_result]},
|
||||
}
|
||||
)
|
||||
|
||||
original_goto = page.goto
|
||||
goto_calls = 0
|
||||
|
||||
def flaky_goto(url: str, wait_until: str | None = None) -> None:
|
||||
nonlocal goto_calls
|
||||
|
||||
goto_calls += 1
|
||||
if goto_calls == 1:
|
||||
raise RuntimeError("navigation boom")
|
||||
|
||||
original_goto(url, wait_until=wait_until)
|
||||
|
||||
original_wait_for_load_state = page.wait_for_load_state
|
||||
|
||||
def flaky_wait_for_load_state(state: str) -> None:
|
||||
if page.current_page == 1:
|
||||
raise RuntimeError("load boom")
|
||||
|
||||
original_wait_for_load_state(state)
|
||||
|
||||
monkeypatch.setattr(page, "goto", flaky_goto)
|
||||
monkeypatch.setattr(page, "wait_for_load_state", flaky_wait_for_load_state)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
results = ApecAdapter(max_listings=10).search(
|
||||
["alpha", "beta"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.url for result in results] == [first_result]
|
||||
assert [result.source_job_id for result in results] == ["111"]
|
||||
|
||||
|
||||
def test_search_stops_after_max_page_count(monkeypatch) -> None:
|
||||
page = _FakeDetailPage(
|
||||
{
|
||||
"alpha": {
|
||||
0: ["https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=0&selectedIndex=0"],
|
||||
1: ["https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/222?motsCles=alpha&page=1&selectedIndex=0"],
|
||||
2: ["https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/333?motsCles=alpha&page=2&selectedIndex=0"],
|
||||
3: ["https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/444?motsCles=alpha&page=3&selectedIndex=0"],
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
original_goto = page.goto
|
||||
|
||||
def bounded_goto(url: str, wait_until: str | None = None) -> None:
|
||||
original_goto(url, wait_until=wait_until)
|
||||
if page.current_page >= 3:
|
||||
raise AssertionError("pagination should stop before page 3")
|
||||
|
||||
monkeypatch.setattr(page, "goto", bounded_goto)
|
||||
monkeypatch.setattr(adapter_module, "_MAX_PAGES_PER_QUERY", 3)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
results = ApecAdapter(max_listings=10).search(
|
||||
["alpha"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.source_job_id for result in results] == ["111", "222", "333"]
|
||||
assert not any("page=3" in url for url in page.goto_urls)
|
||||
|
||||
|
||||
def test_search_stops_after_consecutive_no_progress_pages(monkeypatch) -> None:
|
||||
page = _FakeDetailPage(
|
||||
{
|
||||
"alpha": {
|
||||
0: ["https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=0&selectedIndex=0"],
|
||||
1: ["https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=1&selectedIndex=0"],
|
||||
2: ["https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=2&selectedIndex=0"],
|
||||
3: ["https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/222?motsCles=alpha&page=3&selectedIndex=0"],
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
original_goto = page.goto
|
||||
|
||||
def bounded_goto(url: str, wait_until: str | None = None) -> None:
|
||||
original_goto(url, wait_until=wait_until)
|
||||
if page.current_page >= 3:
|
||||
raise AssertionError("pagination should stop before page 3")
|
||||
|
||||
monkeypatch.setattr(page, "goto", bounded_goto)
|
||||
monkeypatch.setattr(adapter_module, "_MAX_CONSECUTIVE_NO_PROGRESS_PAGES", 2)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
results = ApecAdapter(max_listings=10).search(
|
||||
["alpha"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.source_job_id for result in results] == ["111"]
|
||||
assert not any("page=3" in url for url in page.goto_urls)
|
||||
|
||||
|
||||
def test_search_stops_when_result_page_url_repeats(monkeypatch) -> None:
|
||||
first_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=0&selectedIndex=0"
|
||||
page = _FakeDetailPage(
|
||||
{
|
||||
"alpha": {0: [first_result], 1: [first_result], 2: [first_result]},
|
||||
}
|
||||
)
|
||||
|
||||
original_goto = page.goto
|
||||
initial_result_page_url: str | None = None
|
||||
|
||||
def looping_goto(url: str, wait_until: str | None = None) -> None:
|
||||
nonlocal initial_result_page_url
|
||||
|
||||
original_goto(url, wait_until=wait_until)
|
||||
|
||||
if initial_result_page_url is None and page.current_page == 0:
|
||||
initial_result_page_url = page.url
|
||||
elif initial_result_page_url is not None and page.current_page > 0:
|
||||
page.url = initial_result_page_url
|
||||
|
||||
monkeypatch.setattr(page, "goto", looping_goto)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
results = ApecAdapter(max_listings=10).search(
|
||||
["alpha"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.url for result in results] == [first_result]
|
||||
assert any("page=1" in url for url in page.goto_urls)
|
||||
assert not any("page=2" in url for url in page.goto_urls)
|
||||
|
||||
|
||||
def test_search_raises_when_every_query_fails_to_load_a_search_page(monkeypatch) -> None:
|
||||
page = _FakeDetailPage({"alpha": {0: []}}, search_ready=False)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
with pytest.raises(adapter_module.ApecSearchError):
|
||||
ApecAdapter(max_listings=10).search(
|
||||
["alpha", "beta"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
|
||||
def test_search_treats_zero_results_redirect_as_usable_and_records_other_failures(monkeypatch) -> None:
|
||||
page = _FakeDetailPage(
|
||||
{"alpha": {0: []}, "beta": {0: []}},
|
||||
zero_result_queries={"alpha"},
|
||||
)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
adapter = ApecAdapter(max_listings=10)
|
||||
results = adapter.search(
|
||||
["alpha", "beta"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert results == []
|
||||
assert [error.stage for error in adapter.search_errors] == ["search"]
|
||||
assert "beta" in adapter.search_errors[0].url
|
||||
|
||||
|
||||
def test_search_raises_when_every_query_renders_broken_search_shell(monkeypatch) -> None:
|
||||
page = _FakeDetailPage({"alpha": {0: []}, "beta": {0: []}}, search_ready=True)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
adapter = ApecAdapter(max_listings=10)
|
||||
|
||||
with pytest.raises(adapter_module.ApecSearchError):
|
||||
adapter.search(
|
||||
["alpha", "beta"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [error.stage for error in adapter.search_errors] == ["search", "search"]
|
||||
|
||||
|
||||
def test_search_accepts_current_cgu_popin_before_waiting_for_results(monkeypatch) -> None:
|
||||
first_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=0&selectedIndex=0"
|
||||
page = _FakeDetailPage({"alpha": {0: [first_result]}}, consent_required=True)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
results = ApecAdapter(max_listings=10).search(
|
||||
["alpha"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.url for result in results] == [first_result]
|
||||
assert page.cgu_checkbox_checked is True
|
||||
assert page.consent_button_clicks == ["ACCEPTER"]
|
||||
assert page.consent_accepted is True
|
||||
|
||||
|
||||
def test_search_ignores_unexpected_consent_widget_playwright_errors(monkeypatch) -> None:
|
||||
first_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=0&selectedIndex=0"
|
||||
page = _FakeDetailPage({"alpha": {0: [first_result]}})
|
||||
|
||||
class _ExplodingConsentButton:
|
||||
def click(self, timeout: int | None = None) -> None:
|
||||
raise PlaywrightError("consent widget boom")
|
||||
|
||||
def exploding_get_by_role(role: str, name: str):
|
||||
return _ExplodingConsentButton()
|
||||
|
||||
monkeypatch.setattr(page, "get_by_role", exploding_get_by_role)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
results = ApecAdapter(max_listings=10).search(
|
||||
["alpha"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.url for result in results] == [first_result]
|
||||
assert page.cgu_checkbox_checked is True
|
||||
|
||||
|
||||
def test_search_records_pagination_navigation_failures(monkeypatch) -> None:
|
||||
first_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=0&selectedIndex=0"
|
||||
page = _FakeDetailPage(
|
||||
{
|
||||
"alpha": {
|
||||
0: [first_result],
|
||||
1: ["https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/222?motsCles=alpha&page=1&selectedIndex=0"],
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
original_goto = page.goto
|
||||
|
||||
def flaky_goto(url: str, wait_until: str | None = None) -> None:
|
||||
if "page=1" in url:
|
||||
raise RuntimeError("navigation boom")
|
||||
|
||||
original_goto(url, wait_until=wait_until)
|
||||
|
||||
monkeypatch.setattr(page, "goto", flaky_goto)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
adapter = ApecAdapter(max_listings=10)
|
||||
results = adapter.search(
|
||||
["alpha"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.url for result in results] == [first_result]
|
||||
assert [error.stage for error in adapter.search_errors] == ["search"]
|
||||
assert "page=1" in adapter.search_errors[0].url
|
||||
assert adapter.search_errors[0].message == "page 1 navigation failed"
|
||||
|
||||
|
||||
def test_search_records_pagination_render_failures(monkeypatch) -> None:
|
||||
first_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=0&selectedIndex=0"
|
||||
second_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/222?motsCles=alpha&page=1&selectedIndex=0"
|
||||
page = _FakeDetailPage(
|
||||
{
|
||||
"alpha": {
|
||||
0: [first_result],
|
||||
1: [second_result],
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
original_wait_for_selector = page.wait_for_selector
|
||||
|
||||
def flaky_wait_for_selector(selector: str, timeout: int | None = None) -> None:
|
||||
if selector == _RESULT_LINK_SELECTOR and page.current_page == 1:
|
||||
raise PlaywrightTimeoutError(f"selector not found: {selector}")
|
||||
|
||||
original_wait_for_selector(selector, timeout=timeout)
|
||||
|
||||
monkeypatch.setattr(page, "wait_for_selector", flaky_wait_for_selector)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
adapter = ApecAdapter(max_listings=10)
|
||||
results = adapter.search(
|
||||
["alpha"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.url for result in results] == [first_result]
|
||||
assert [error.stage for error in adapter.search_errors] == ["search"]
|
||||
assert "page=1" in adapter.search_errors[0].url
|
||||
assert adapter.search_errors[0].message == "page 1 results did not render"
|
||||
|
||||
|
||||
def test_search_records_evaluate_all_failures_and_continues_to_next_query(monkeypatch) -> None:
|
||||
first_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111?motsCles=alpha&page=0&selectedIndex=0"
|
||||
second_result = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/222?motsCles=beta&page=0&selectedIndex=0"
|
||||
page = _FakeDetailPage(
|
||||
{
|
||||
"alpha": {
|
||||
0: [first_result],
|
||||
1: [first_result],
|
||||
},
|
||||
"beta": {
|
||||
0: [second_result],
|
||||
1: [second_result],
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
original_locator = page.locator
|
||||
|
||||
class _FlakyLocator:
|
||||
def __init__(self, locator) -> None:
|
||||
self._locator = locator
|
||||
|
||||
def evaluate_all(self, function: str):
|
||||
if (
|
||||
self._locator.selector == _RESULT_LINK_SELECTOR
|
||||
and page.current_query == "alpha"
|
||||
and page.current_page == 1
|
||||
):
|
||||
raise RuntimeError("evaluate boom")
|
||||
|
||||
return self._locator.evaluate_all(function)
|
||||
|
||||
def __getattr__(self, name: str):
|
||||
return getattr(self._locator, name)
|
||||
|
||||
def flaky_locator(selector: str):
|
||||
return _FlakyLocator(original_locator(selector))
|
||||
|
||||
monkeypatch.setattr(page, "locator", flaky_locator)
|
||||
monkeypatch.setattr(adapter_module, "_MAX_CONSECUTIVE_NO_PROGRESS_PAGES", 1)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
adapter = ApecAdapter(max_listings=10)
|
||||
results = adapter.search(
|
||||
["alpha", "beta"],
|
||||
search_filters=ApecSearchFilters(location="France", contract_type="CDI"),
|
||||
)
|
||||
|
||||
assert [result.url for result in results] == [first_result, second_result]
|
||||
assert [error.stage for error in adapter.search_errors] == ["search"]
|
||||
assert "page=1" in adapter.search_errors[0].url
|
||||
|
||||
|
||||
def test_fetch_listing_html_waits_for_rendered_offer_content(monkeypatch) -> None:
|
||||
page = _FakeDetailPage({}, rendered_html="<html>rendered offer</html>")
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
html = ApecAdapter().fetch_listing_html(
|
||||
"https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111"
|
||||
)
|
||||
|
||||
assert html == "<html>rendered offer</html>"
|
||||
assert len(page.waited_functions) == 1
|
||||
assert ".container-details-offer h1" in page.waited_functions[0][0]
|
||||
assert ".ref-offre" in page.waited_functions[0][0]
|
||||
assert ".details-offer-list" in page.waited_functions[0][0]
|
||||
assert "Descriptif du poste" not in page.waited_functions[0][0]
|
||||
assert page.waited_functions[0][1] == 1000
|
||||
|
||||
|
||||
def test_fetch_listing_html_accepts_current_cgu_popin_before_waiting_for_detail_content(monkeypatch) -> None:
|
||||
page = _FakeDetailPage({}, rendered_html="<html>rendered offer</html>", consent_required=True)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
html = ApecAdapter().fetch_listing_html(
|
||||
"https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111"
|
||||
)
|
||||
|
||||
assert html == "<html>rendered offer</html>"
|
||||
assert page.cgu_checkbox_checked is True
|
||||
assert page.consent_button_clicks == ["ACCEPTER"]
|
||||
assert page.consent_accepted is True
|
||||
|
||||
|
||||
def test_fetch_listing_html_uses_explicit_company_fallback_chain(monkeypatch) -> None:
|
||||
page = _FakeDetailPage({}, rendered_html="<html>rendered offer</html>")
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
ApecAdapter().fetch_listing_html(
|
||||
"https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111"
|
||||
)
|
||||
|
||||
wait_script = page.waited_functions[0][0]
|
||||
assert "companySelectors" not in wait_script
|
||||
assert ".container-details-offer h1" in wait_script
|
||||
assert ".ref-offre" in wait_script
|
||||
assert ".details-offer-list" in wait_script
|
||||
|
||||
|
||||
def test_fetch_listing_html_rejects_redirected_non_apec_urls(monkeypatch) -> None:
|
||||
page = _FakeDetailPage({}, rendered_html="<html>redirected</html>")
|
||||
|
||||
original_goto = page.goto
|
||||
|
||||
def redirecting_goto(url: str, wait_until: str | None = None) -> None:
|
||||
original_goto(url, wait_until=wait_until)
|
||||
page.url = "https://www.apec.fr/candidat/recherche-emploi.html/emploi/recherche-avancee?error=true"
|
||||
|
||||
monkeypatch.setattr(page, "goto", redirecting_goto)
|
||||
monkeypatch.setattr(adapter_module, "_open_public_page", lambda: _fake_open_public_page(page))
|
||||
|
||||
with pytest.raises(ValueError, match="unexpected URL after redirects"):
|
||||
ApecAdapter().fetch_listing_html(
|
||||
"https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111"
|
||||
)
|
||||
|
||||
assert len(page.waited_functions) == 1
|
||||
assert page.goto_urls == ["https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111"]
|
||||
|
||||
|
||||
def test_fetch_listing_html_rejects_non_apec_hosts() -> None:
|
||||
adapter = ApecAdapter()
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
adapter.fetch_listing_html("https://evilapec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111")
|
||||
|
||||
|
||||
def test_fetch_listing_html_reuses_browser_context_across_calls(monkeypatch) -> None:
|
||||
class FakePage:
|
||||
def __init__(self) -> None:
|
||||
self.goto_urls: list[str] = []
|
||||
self.default_timeout: int | None = None
|
||||
self.url = ""
|
||||
|
||||
def set_default_timeout(self, timeout: int) -> None:
|
||||
self.default_timeout = timeout
|
||||
|
||||
def goto(self, url: str, wait_until: str | None = None) -> None:
|
||||
self.goto_urls.append(url)
|
||||
self.url = url
|
||||
|
||||
def wait_for_function(self, function: str, polling: int | None = None, timeout: int | None = None) -> None:
|
||||
return None
|
||||
|
||||
def content(self) -> str:
|
||||
return "<html>shared</html>"
|
||||
|
||||
def close(self) -> None:
|
||||
return None
|
||||
|
||||
class FakeBrowserContext:
|
||||
def __init__(self) -> None:
|
||||
self.new_page_calls = 0
|
||||
|
||||
def new_page(self) -> FakePage:
|
||||
self.new_page_calls += 1
|
||||
return FakePage()
|
||||
|
||||
def close(self) -> None:
|
||||
return None
|
||||
|
||||
class FakeBrowser:
|
||||
def __init__(self, browser_context: FakeBrowserContext) -> None:
|
||||
self.browser_context = browser_context
|
||||
|
||||
def new_context(self) -> FakeBrowserContext:
|
||||
return self.browser_context
|
||||
|
||||
def close(self) -> None:
|
||||
return None
|
||||
|
||||
class FakeChromium:
|
||||
def __init__(self, browser: FakeBrowser) -> None:
|
||||
self.browser = browser
|
||||
self.launch_calls = 0
|
||||
|
||||
def launch(self, headless: bool = True) -> FakeBrowser:
|
||||
self.launch_calls += 1
|
||||
return self.browser
|
||||
|
||||
class FakePlaywright:
|
||||
def __init__(self, chromium: FakeChromium) -> None:
|
||||
self.chromium = chromium
|
||||
|
||||
class FakePlaywrightManager:
|
||||
def __init__(self, chromium: FakeChromium) -> None:
|
||||
self.playwright = FakePlaywright(chromium)
|
||||
|
||||
def __enter__(self) -> FakePlaywright:
|
||||
return self.playwright
|
||||
|
||||
def __exit__(self, exc_type, exc, tb) -> None:
|
||||
return None
|
||||
|
||||
browser_context = FakeBrowserContext()
|
||||
browser = FakeBrowser(browser_context)
|
||||
chromium = FakeChromium(browser)
|
||||
|
||||
monkeypatch.setattr(adapter_module, "sync_playwright", lambda: FakePlaywrightManager(chromium))
|
||||
|
||||
adapter = ApecAdapter()
|
||||
with adapter.browser_session():
|
||||
html_one = adapter.fetch_listing_html("https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/111")
|
||||
html_two = adapter.fetch_listing_html("https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/222")
|
||||
|
||||
assert html_one == "<html>shared</html>"
|
||||
assert html_two == "<html>shared</html>"
|
||||
assert chromium.launch_calls == 1
|
||||
assert browser_context.new_page_calls == 2
|
||||
212
tests/apec/test_dedupe.py
Normal file
212
tests/apec/test_dedupe.py
Normal file
@ -0,0 +1,212 @@
|
||||
from job_research.apec.dedupe import dedupe_apec_listings
|
||||
from job_research.models import ApecListing
|
||||
|
||||
|
||||
def test_dedupe_apec_listings_by_url_preserves_first_listing() -> None:
|
||||
first = ApecListing(
|
||||
source="apec",
|
||||
url="https://example.test/job/1",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
)
|
||||
second = ApecListing(
|
||||
source="apec",
|
||||
url="https://example.test/job/1",
|
||||
fetched_at="2026-06-01T10:01:00Z",
|
||||
)
|
||||
|
||||
deduped = dedupe_apec_listings([first, second])
|
||||
|
||||
assert deduped == [first]
|
||||
|
||||
|
||||
def test_dedupe_apec_listings_by_source_job_id_ignores_url_changes() -> None:
|
||||
first = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="job-123",
|
||||
url="https://example.test/job/1",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
)
|
||||
second = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="job-123",
|
||||
url="https://example.test/job/2",
|
||||
fetched_at="2026-06-01T10:01:00Z",
|
||||
)
|
||||
|
||||
deduped = dedupe_apec_listings([first, second])
|
||||
|
||||
assert deduped == [first]
|
||||
|
||||
|
||||
def test_dedupe_apec_listings_collapses_mixed_key_duplicates() -> None:
|
||||
first = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="job-123",
|
||||
url="https://example.test/job/1",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
)
|
||||
second = ApecListing(
|
||||
source="apec",
|
||||
source_job_id=None,
|
||||
url="https://example.test/job/1",
|
||||
fetched_at="2026-06-01T10:01:00Z",
|
||||
)
|
||||
|
||||
deduped = dedupe_apec_listings([first, second])
|
||||
|
||||
assert deduped == [first]
|
||||
|
||||
|
||||
def test_dedupe_apec_listings_keeps_secondary_ids_from_skipped_rows() -> None:
|
||||
first = ApecListing(
|
||||
source="apec",
|
||||
source_job_id=None,
|
||||
url="url1",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
)
|
||||
second = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="job-123",
|
||||
url="url1",
|
||||
fetched_at="2026-06-01T10:01:00Z",
|
||||
)
|
||||
third = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="job-123",
|
||||
url="url2",
|
||||
fetched_at="2026-06-01T10:02:00Z",
|
||||
)
|
||||
|
||||
deduped = dedupe_apec_listings([first, second, third])
|
||||
|
||||
assert deduped == [first]
|
||||
|
||||
|
||||
def test_dedupe_apec_listings_merges_metadata_from_duplicate_rows() -> None:
|
||||
first = ApecListing(
|
||||
source="apec",
|
||||
source_job_id=None,
|
||||
published_at=None,
|
||||
refreshed_at=None,
|
||||
url="url1",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
)
|
||||
second = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="job-123",
|
||||
published_at="2026-06-01",
|
||||
refreshed_at="2026-06-02",
|
||||
url="url1",
|
||||
fetched_at="2026-06-01T10:01:00Z",
|
||||
)
|
||||
|
||||
deduped = dedupe_apec_listings([first, second])
|
||||
|
||||
assert len(deduped) == 1
|
||||
assert deduped[0].url == "url1"
|
||||
assert deduped[0].source_job_id == "job-123"
|
||||
assert deduped[0].published_at == "2026-06-01"
|
||||
assert deduped[0].refreshed_at == "2026-06-02"
|
||||
|
||||
|
||||
def test_dedupe_apec_listings_merges_metadata_through_alias_chain() -> None:
|
||||
first = ApecListing(
|
||||
source="apec",
|
||||
source_job_id=None,
|
||||
url="u1",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
)
|
||||
second = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="i2",
|
||||
url="u2",
|
||||
fetched_at="2026-06-01T10:01:00Z",
|
||||
)
|
||||
third = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="i4",
|
||||
url="u2",
|
||||
fetched_at="2026-06-01T10:02:00Z",
|
||||
)
|
||||
fourth = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="i2",
|
||||
url="u1",
|
||||
fetched_at="2026-06-01T10:03:00Z",
|
||||
)
|
||||
fifth = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="i4",
|
||||
url="u6",
|
||||
company="NewestCo",
|
||||
fetched_at="2026-06-01T10:04:00Z",
|
||||
)
|
||||
|
||||
deduped = dedupe_apec_listings([first, second, third, fourth, fifth])
|
||||
|
||||
assert len(deduped) == 1
|
||||
assert deduped[0].url == "u1"
|
||||
assert deduped[0].source_job_id == "i2"
|
||||
assert deduped[0].company == "NewestCo"
|
||||
|
||||
|
||||
def test_dedupe_apec_listings_keeps_one_survivor_for_cluster_alias_chain() -> None:
|
||||
first = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="id2",
|
||||
url="u2",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
)
|
||||
second = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="id3",
|
||||
url="u2",
|
||||
fetched_at="2026-06-01T10:01:00Z",
|
||||
)
|
||||
third = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="id2",
|
||||
url="u1",
|
||||
fetched_at="2026-06-01T10:02:00Z",
|
||||
)
|
||||
fourth = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="id3",
|
||||
url="u3",
|
||||
fetched_at="2026-06-01T10:03:00Z",
|
||||
)
|
||||
|
||||
deduped = dedupe_apec_listings([first, second, third, fourth])
|
||||
|
||||
assert len(deduped) == 1
|
||||
assert deduped[0].url == "u2"
|
||||
assert deduped[0].source_job_id == "id2"
|
||||
|
||||
|
||||
def test_dedupe_apec_listings_keeps_first_listing_as_bridge_survivor() -> None:
|
||||
first = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="id1",
|
||||
url="u1",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
)
|
||||
second = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="id2",
|
||||
url="u2",
|
||||
fetched_at="2026-06-01T10:01:00Z",
|
||||
)
|
||||
third = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="id1",
|
||||
url="u2",
|
||||
company="NewestCo",
|
||||
fetched_at="2026-06-01T10:02:00Z",
|
||||
)
|
||||
|
||||
deduped = dedupe_apec_listings([first, second, third])
|
||||
|
||||
assert len(deduped) == 1
|
||||
assert deduped[0].url == "u1"
|
||||
assert deduped[0].source_job_id == "id1"
|
||||
assert deduped[0].company == "NewestCo"
|
||||
372
tests/apec/test_normalize.py
Normal file
372
tests/apec/test_normalize.py
Normal file
@ -0,0 +1,372 @@
|
||||
from job_research.apec.normalize import normalize_apec_listing
|
||||
|
||||
|
||||
def test_normalize_apec_listing_extracts_minimal_shape() -> None:
|
||||
html = """
|
||||
<html>
|
||||
<body>
|
||||
<main class="container-details-offer">
|
||||
<h1>Data Engineer F/H</h1>
|
||||
<div class="card-offer">
|
||||
<div class="ref-offre">Ref. Apec : 178554452W</div>
|
||||
<ul class="details-offer-list mb-20">
|
||||
<li>CLOUD TEMPLE</li>
|
||||
<li>1 <span> CDI </span></li>
|
||||
<li>Puteaux - 92</li>
|
||||
</ul>
|
||||
<p>Publiée le 20/04/2026 Actualisée le 02/06/2026</p>
|
||||
</div>
|
||||
<article class="card card-ents mb-20">
|
||||
<div class="list-hzt mb-20">
|
||||
<span class="ents-name">CLOUD TEMPLE</span>
|
||||
</div>
|
||||
</article>
|
||||
<div class="details-post">
|
||||
<h4>Salaire</h4>
|
||||
<span>A partir de 70 k€ brut annuel</span>
|
||||
</div>
|
||||
<div class="details-post">
|
||||
<h4>Prise de poste</h4>
|
||||
<span>Dès que possible</span>
|
||||
</div>
|
||||
<div class="details-post">
|
||||
<h4>Expérience</h4>
|
||||
<span>Minimum 7 ans</span>
|
||||
</div>
|
||||
<div class="details-post">
|
||||
<h4>Descriptif du poste</h4>
|
||||
<p>Build pipelines</p>
|
||||
<div class="nested-late-sections">
|
||||
<h4>Profil recherché</h4>
|
||||
<p>Python / SQL</p>
|
||||
<h4>Compétences attendues</h4>
|
||||
<p>Ignored</p>
|
||||
<h4>Entreprise</h4>
|
||||
<p>Ignored</p>
|
||||
<div class="recruiter">Ignored recruiter info</div>
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
listing = normalize_apec_listing(
|
||||
url="https://example.test/job/123",
|
||||
html=html,
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
source_job_id="job-123",
|
||||
)
|
||||
|
||||
assert listing.source == "apec"
|
||||
assert listing.source_job_id == "job-123"
|
||||
assert listing.url == "https://example.test/job/123"
|
||||
assert listing.title == "Data Engineer F/H"
|
||||
assert listing.company == "CLOUD TEMPLE"
|
||||
assert listing.location == "Puteaux - 92"
|
||||
assert listing.contract_type == "CDI"
|
||||
assert listing.description_text == "Build pipelines"
|
||||
assert listing.published_at == "2026-04-20"
|
||||
assert listing.refreshed_at == "2026-06-02"
|
||||
assert listing.fetched_at == "2026-06-01T10:00:00Z"
|
||||
|
||||
|
||||
def test_normalize_apec_listing_prefers_final_source_job_id_from_detail_page() -> None:
|
||||
html = """
|
||||
<html>
|
||||
<body>
|
||||
<main class="container-details-offer">
|
||||
<h1>Data Engineer F/H</h1>
|
||||
<div class="card-offer">
|
||||
<div class="ref-offre">Ref. Apec : FINAL456</div>
|
||||
<ul class="details-offer-list mb-20">
|
||||
<li>CLOUD TEMPLE</li>
|
||||
<li>1 <span> CDI </span></li>
|
||||
<li>Puteaux - 92</li>
|
||||
</ul>
|
||||
<p>Publiée le 20/04/2026 Actualisée le 02/06/2026</p>
|
||||
</div>
|
||||
<article class="card card-ents mb-20">
|
||||
<div class="list-hzt mb-20">
|
||||
<span class="ents-name">CLOUD TEMPLE</span>
|
||||
</div>
|
||||
</article>
|
||||
<div class="details-post">
|
||||
<h4>Descriptif du poste</h4>
|
||||
<p>Build pipelines</p>
|
||||
</div>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
listing = normalize_apec_listing(
|
||||
url="https://www.apec.fr/candidat/recherche-emploi.html/emploi/detail-offre/REQUESTED123",
|
||||
html=html,
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
source_job_id="REQUESTED123",
|
||||
)
|
||||
|
||||
assert listing.source_job_id == "FINAL456"
|
||||
|
||||
|
||||
def test_normalize_apec_listing_warns_and_returns_none_for_invalid_dates() -> None:
|
||||
html = """
|
||||
<html>
|
||||
<body>
|
||||
<main class="container-details-offer">
|
||||
<h1>Data Engineer F/H</h1>
|
||||
<div class="card-offer">
|
||||
<div class="ref-offre">Ref. Apec : 178554452W</div>
|
||||
<ul class="details-offer-list mb-20">
|
||||
<li>CLOUD TEMPLE</li>
|
||||
<li>1 <span> CDI </span></li>
|
||||
<li>Puteaux - 92</li>
|
||||
</ul>
|
||||
<p>Publiée le 32/13/2026 Actualisée le 31/02/2026</p>
|
||||
</div>
|
||||
<article class="card card-ents mb-20">
|
||||
<div class="list-hzt mb-20">
|
||||
<span class="ents-name">CLOUD TEMPLE</span>
|
||||
</div>
|
||||
</article>
|
||||
<div class="details-post">
|
||||
<h4>Descriptif du poste</h4>
|
||||
<p>Build pipelines</p>
|
||||
</div>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
listing = normalize_apec_listing(
|
||||
url="https://example.test/job/123",
|
||||
html=html,
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
source_job_id="job-123",
|
||||
)
|
||||
|
||||
assert listing.published_at is None
|
||||
assert listing.refreshed_at is None
|
||||
assert [warning.field for warning in listing.warnings] == ["published_at", "refreshed_at"]
|
||||
|
||||
|
||||
def test_normalize_apec_listing_uses_details_offer_list_company_fallback() -> None:
|
||||
html = """
|
||||
<html>
|
||||
<body>
|
||||
<main class="container-details-offer">
|
||||
<h1>Data Engineer F/H</h1>
|
||||
<div class="card-offer">
|
||||
<div class="ref-offre">Ref. Apec : 178554452W</div>
|
||||
<ul class="details-offer-list mb-20">
|
||||
<li>Fallback Company</li>
|
||||
<li>1 <span> CDI </span></li>
|
||||
<li>Puteaux - 92</li>
|
||||
</ul>
|
||||
<p>Publiée le 20/04/2026 Actualisée le 02/06/2026</p>
|
||||
</div>
|
||||
<div class="details-post">
|
||||
<h4>Salaire</h4>
|
||||
<span>A partir de 70 k€ brut annuel</span>
|
||||
</div>
|
||||
<div class="details-post">
|
||||
<h4>Prise de poste</h4>
|
||||
<span>Dès que possible</span>
|
||||
</div>
|
||||
<div class="details-post">
|
||||
<h4>Expérience</h4>
|
||||
<span>Minimum 7 ans</span>
|
||||
</div>
|
||||
<div class="details-post">
|
||||
<h4>Descriptif du poste</h4>
|
||||
<p>Build pipelines</p>
|
||||
<div class="nested-late-sections">
|
||||
<h4>Profil recherché</h4>
|
||||
<p>Python / SQL</p>
|
||||
<h4>Compétences attendues</h4>
|
||||
<p>Ignored</p>
|
||||
<h4>Entreprise</h4>
|
||||
<p>Ignored</p>
|
||||
<div class="recruiter">Ignored recruiter info</div>
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
listing = normalize_apec_listing(
|
||||
url="https://example.test/job/123",
|
||||
html=html,
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
source_job_id=None,
|
||||
)
|
||||
|
||||
assert listing.company == "Fallback Company"
|
||||
assert listing.description_text == "Build pipelines"
|
||||
assert listing.refreshed_at == "2026-06-02"
|
||||
|
||||
|
||||
def test_normalize_apec_listing_records_warnings_for_fallback_and_missing_fields() -> None:
|
||||
html = """
|
||||
<html>
|
||||
<body>
|
||||
<main class="container-details-offer">
|
||||
<div class="card-offer">
|
||||
<div class="ref-offre">Ref. Apec : 178554452W</div>
|
||||
<ul class="details-offer-list mb-20">
|
||||
<li>Fallback Company</li>
|
||||
<li>1 CDI</li>
|
||||
</ul>
|
||||
</div>
|
||||
<article class="card card-ents mb-20">
|
||||
<div class="list-hzt mb-20">
|
||||
<span class="card-ents-quote">Fallback Company</span>
|
||||
</div>
|
||||
</article>
|
||||
</main>
|
||||
<h1>Fallback Title</h1>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
listing = normalize_apec_listing(
|
||||
url="https://example.test/job/123",
|
||||
html=html,
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
source_job_id=None,
|
||||
)
|
||||
|
||||
assert [warning.field for warning in listing.warnings] == [
|
||||
"title",
|
||||
"location",
|
||||
"contract_type",
|
||||
"description_text",
|
||||
"source_job_id",
|
||||
"company",
|
||||
"published_at",
|
||||
]
|
||||
|
||||
|
||||
def test_normalize_apec_listing_records_warnings_for_placeholder_text_values() -> None:
|
||||
html = """
|
||||
<html>
|
||||
<body>
|
||||
<main class="container-details-offer">
|
||||
<h1> N/A </h1>
|
||||
<div class="card-offer">
|
||||
<div class="ref-offre">Ref. Apec : 178554452W</div>
|
||||
<ul class="details-offer-list mb-20">
|
||||
<li>Example Corp</li>
|
||||
<li>1 <span> N/A </span></li>
|
||||
<li> - </li>
|
||||
</ul>
|
||||
<p>Publiée le 20/04/2026 Actualisée le 02/06/2026</p>
|
||||
</div>
|
||||
<article class="card card-ents mb-20">
|
||||
<div class="list-hzt mb-20">
|
||||
<span class="ents-name">Example Corp</span>
|
||||
</div>
|
||||
</article>
|
||||
<div class="details-post">
|
||||
<h4>Descriptif du poste</h4>
|
||||
<p>Build pipelines</p>
|
||||
</div>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
listing = normalize_apec_listing(
|
||||
url="https://example.test/job/123",
|
||||
html=html,
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
source_job_id="178554452W",
|
||||
)
|
||||
|
||||
assert listing.title is None
|
||||
assert listing.location is None
|
||||
assert listing.contract_type is None
|
||||
assert [warning.field for warning in listing.warnings] == ["title", "location", "contract_type"]
|
||||
|
||||
|
||||
def test_normalize_apec_listing_records_warning_for_placeholder_company() -> None:
|
||||
html = """
|
||||
<html>
|
||||
<body>
|
||||
<main class="container-details-offer">
|
||||
<h1>Data Engineer F/H</h1>
|
||||
<div class="card-offer">
|
||||
<div class="ref-offre">Ref. Apec : 178554452W</div>
|
||||
<ul class="details-offer-list mb-20">
|
||||
<li>Example Corp</li>
|
||||
<li>1 <span> CDI </span></li>
|
||||
<li>Puteaux - 92</li>
|
||||
</ul>
|
||||
<p>Publiée le 20/04/2026 Actualisée le 02/06/2026</p>
|
||||
</div>
|
||||
<article class="card card-ents mb-20">
|
||||
<div class="list-hzt mb-20">
|
||||
<span class="ents-name">N/A</span>
|
||||
</div>
|
||||
</article>
|
||||
<div class="details-post">
|
||||
<h4>Descriptif du poste</h4>
|
||||
<p>Build pipelines</p>
|
||||
</div>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
listing = normalize_apec_listing(
|
||||
url="https://example.test/job/123",
|
||||
html=html,
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
source_job_id="178554452W",
|
||||
)
|
||||
|
||||
assert listing.company is None
|
||||
assert listing.refreshed_at == "2026-06-02"
|
||||
assert [warning.field for warning in listing.warnings] == ["company"]
|
||||
|
||||
|
||||
def test_normalize_apec_listing_records_warning_for_placeholder_description_text() -> None:
|
||||
html = """
|
||||
<html>
|
||||
<body>
|
||||
<main class="container-details-offer">
|
||||
<h1>Data Engineer F/H</h1>
|
||||
<div class="card-offer">
|
||||
<div class="ref-offre">Ref. Apec : 178554452W</div>
|
||||
<ul class="details-offer-list mb-20">
|
||||
<li>CLOUD TEMPLE</li>
|
||||
<li>1 <span> CDI </span></li>
|
||||
<li>Puteaux - 92</li>
|
||||
</ul>
|
||||
<p>Publiée le 20/04/2026 Actualisée le 02/06/2026</p>
|
||||
</div>
|
||||
<article class="card card-ents mb-20">
|
||||
<div class="list-hzt mb-20">
|
||||
<span class="ents-name">CLOUD TEMPLE</span>
|
||||
</div>
|
||||
</article>
|
||||
<div class="details-post">
|
||||
<h4>Descriptif du poste</h4>
|
||||
<p>N/A</p>
|
||||
</div>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
listing = normalize_apec_listing(
|
||||
url="https://example.test/job/123",
|
||||
html=html,
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
source_job_id="job-123",
|
||||
)
|
||||
|
||||
assert listing.description_text is None
|
||||
assert [warning.field for warning in listing.warnings] == ["description_text"]
|
||||
75
tests/apec/test_query_derivation.py
Normal file
75
tests/apec/test_query_derivation.py
Normal file
@ -0,0 +1,75 @@
|
||||
from job_research.apec.adapter import ApecSearchFilters
|
||||
from job_research.apec.query_derivation import derive_apec_queries, derive_apec_search_filters
|
||||
from job_research.models import CandidateProfileOutput
|
||||
|
||||
|
||||
def test_derive_apec_queries_preserves_order_dedupes_and_caps_at_five() -> None:
|
||||
profile = CandidateProfileOutput(
|
||||
target_roles=[
|
||||
"Data Engineer",
|
||||
"Analytics Engineer",
|
||||
"Data Engineer",
|
||||
"BI Engineer",
|
||||
"Junior Data Platform Engineer",
|
||||
"ML Engineer",
|
||||
"Backend Engineer",
|
||||
],
|
||||
strengths=["Python", "SQL"],
|
||||
skills_to_emphasize=["BigQuery", "Terraform"],
|
||||
)
|
||||
|
||||
queries = derive_apec_queries(profile)
|
||||
|
||||
assert queries == [
|
||||
"Data Engineer",
|
||||
"Analytics Engineer",
|
||||
"BI Engineer",
|
||||
"Junior Data Platform Engineer",
|
||||
"ML Engineer",
|
||||
]
|
||||
|
||||
|
||||
def test_derive_apec_queries_uses_up_to_five_target_roles_when_no_support_terms_exist() -> None:
|
||||
profile = CandidateProfileOutput(
|
||||
target_roles=[
|
||||
"Data Engineer",
|
||||
"Analytics Engineer",
|
||||
"BI Engineer",
|
||||
"Junior Data Platform Engineer",
|
||||
"ML Engineer",
|
||||
"Backend Engineer",
|
||||
]
|
||||
)
|
||||
|
||||
queries = derive_apec_queries(profile)
|
||||
|
||||
assert queries == [
|
||||
"Data Engineer",
|
||||
"Analytics Engineer",
|
||||
"BI Engineer",
|
||||
"Junior Data Platform Engineer",
|
||||
"ML Engineer",
|
||||
]
|
||||
|
||||
|
||||
def test_derive_apec_queries_uses_support_terms_without_constraints() -> None:
|
||||
profile = CandidateProfileOutput(
|
||||
target_roles=["Data Engineer"],
|
||||
strengths=["Python"],
|
||||
skills_to_emphasize=["BigQuery"],
|
||||
constraints=["CDI only", "France only"],
|
||||
)
|
||||
|
||||
queries = derive_apec_queries(profile)
|
||||
|
||||
assert queries == [
|
||||
"Data Engineer",
|
||||
"Data Engineer Python",
|
||||
"Data Engineer BigQuery",
|
||||
]
|
||||
|
||||
|
||||
def test_derive_apec_search_filters_from_constraints() -> None:
|
||||
profile = CandidateProfileOutput(constraints=["CDI only", "France only"])
|
||||
|
||||
assert derive_apec_search_filters(profile) == ApecSearchFilters(location="France", contract_type="CDI")
|
||||
431
tests/profile/test_cv_extractor.py
Normal file
431
tests/profile/test_cv_extractor.py
Normal file
@ -0,0 +1,431 @@
|
||||
from textwrap import dedent
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from job_research.profile.cv_extractor import extract_cv_signals
|
||||
from job_research.profile.cv_extractor import extract_pdf_text
|
||||
|
||||
|
||||
def test_extract_cv_signals_reads_basic_fields_from_text() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Languages: French, English
|
||||
Skills: Python, SQL, Terraform, GCP, BigQuery
|
||||
Data Engineer at Company A
|
||||
Analytics Engineer at Company B
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["name"] == "Tonio"
|
||||
assert extracted["location"] == "France"
|
||||
assert extracted["languages"] == ["French", "English"]
|
||||
assert extracted["skills"] == ["Python", "SQL", "Terraform", "GCP", "BigQuery"]
|
||||
assert extracted["experience_entries"][0]["title"] == "Data Engineer"
|
||||
assert len(extracted["experience_entries"]) == 2
|
||||
|
||||
|
||||
def test_extract_cv_signals_flags_low_confidence_first_line_as_name() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Data Engineer | Python | GCP
|
||||
Location: France
|
||||
Languages: French, English
|
||||
Skills: Python, SQL
|
||||
Data Engineer at Company A
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["name"] == "Data Engineer | Python | GCP"
|
||||
assert [warning.field for warning in extracted["warnings"]] == ["name"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("line", "expected"),
|
||||
[
|
||||
("Years of experience: 2", 2),
|
||||
("Années d'expérience : 3", 3),
|
||||
],
|
||||
)
|
||||
def test_extract_cv_signals_extracts_years_of_experience_from_explicit_line(
|
||||
line: str, expected: int
|
||||
) -> None:
|
||||
text = dedent(
|
||||
f"""
|
||||
Tonio
|
||||
Location: France
|
||||
{line}
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["years_of_experience"] == expected
|
||||
|
||||
|
||||
def test_extract_cv_signals_allows_single_word_titles() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Consultant at Accenture
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == [
|
||||
{"title": "Consultant", "company": "Accenture"}
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_allows_lowercase_company_names() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Data Engineer at leboncoin
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == [
|
||||
{"title": "Data Engineer", "company": "leboncoin"}
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_ignores_prose_after_company_name() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Senior engineer at Microsoft before moving to Paris.
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == []
|
||||
|
||||
|
||||
def test_extract_cv_signals_normalizes_bullet_prefixed_fields_and_experience() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
- Location: France
|
||||
- Data Engineer at Company A
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["location"] == "France"
|
||||
assert extracted["experience_entries"] == [
|
||||
{"title": "Data Engineer", "company": "Company A"}
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_normalizes_en_dash_bullet_prefixed_fields_and_experience() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
– Location: France
|
||||
— Data Engineer at Company A
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["location"] == "France"
|
||||
assert extracted["experience_entries"] == [
|
||||
{"title": "Data Engineer", "company": "Company A"}
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_recognizes_french_field_labels() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Formation:
|
||||
M.Sc. in Data Engineering at EPITA
|
||||
Langues: French, English
|
||||
Compétences: Python, SQL
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["languages"] == ["French", "English"]
|
||||
assert extracted["skills"] == ["Python", "SQL"]
|
||||
assert extracted["education_entries"] == [
|
||||
{"credential": "M.Sc. in Data Engineering", "institution": "EPITA"}
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_parses_french_experience_connectors() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Ingénieur chez Dassault Systèmes
|
||||
Développeur au CNRS
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == [
|
||||
{"title": "Ingénieur", "company": "Dassault Systèmes"},
|
||||
{"title": "Développeur", "company": "CNRS"},
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_parses_clear_titles_with_french_and_english_connectors() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Ingénieur à Thales
|
||||
Ingénieur Data chez BNP Paribas
|
||||
Consultant BI chez Accenture
|
||||
Head of Data at Qonto
|
||||
Product Owner at Qonto
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == [
|
||||
{"title": "Ingénieur", "company": "Thales"},
|
||||
{"title": "Ingénieur Data", "company": "BNP Paribas"},
|
||||
{"title": "Consultant BI", "company": "Accenture"},
|
||||
{"title": "Head of Data", "company": "Qonto"},
|
||||
{"title": "Product Owner", "company": "Qonto"},
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_accepts_lowercase_short_and_real_company_titles() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
data engineer at Company A
|
||||
iOS Engineer at Company A
|
||||
3D Artist at Studio
|
||||
R Developer at Company A
|
||||
Data Engineer at Made in Design
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == [
|
||||
{"title": "data engineer", "company": "Company A"},
|
||||
{"title": "iOS Engineer", "company": "Company A"},
|
||||
{"title": "3D Artist", "company": "Studio"},
|
||||
{"title": "R Developer", "company": "Company A"},
|
||||
{"title": "Data Engineer", "company": "Made in Design"},
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_ignores_french_prose_continuations() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Ingénieur au sein de BNP Paribas
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == []
|
||||
|
||||
|
||||
def test_extract_cv_signals_rejects_label_like_lines_without_colons() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location at Paris
|
||||
Summary at a glance
|
||||
Profile at LinkedIn
|
||||
Education at EPITA
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == []
|
||||
|
||||
|
||||
def test_extract_cv_signals_rejects_narrative_bullet_experience_lines() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Implemented data pipelines at Airbnb
|
||||
Designed dashboards at Company A
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == []
|
||||
|
||||
|
||||
def test_extract_cv_signals_ignores_in_paris_prose_tail() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Data Engineer at Microsoft in Paris
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == []
|
||||
|
||||
|
||||
def test_extract_cv_signals_extracts_education_entries_after_heading() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Education:
|
||||
- Master of Science in Data Science at University of Paris
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["education_entries"] == [
|
||||
{
|
||||
"credential": "Master of Science in Data Science",
|
||||
"institution": "University of Paris",
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_extracts_common_french_education_entries() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Education:
|
||||
Diplôme d'ingénieur
|
||||
CentraleSupélec
|
||||
MSc
|
||||
University of Paris
|
||||
BSc
|
||||
University of Oxford
|
||||
Bac+5
|
||||
École Polytechnique
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["education_entries"] == [
|
||||
{
|
||||
"credential": "Diplôme d'ingénieur",
|
||||
"institution": "CentraleSupélec",
|
||||
},
|
||||
{"credential": "MSc", "institution": "University of Paris"},
|
||||
{"credential": "BSc", "institution": "University of Oxford"},
|
||||
{"credential": "Bac+5", "institution": "École Polytechnique"},
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_extracts_dotted_degree_variants() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Education:
|
||||
M.Sc. in Data Engineering at EPITA
|
||||
B.Sc. in Computer Science at University X
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["education_entries"] == [
|
||||
{"credential": "M.Sc. in Data Engineering", "institution": "EPITA"},
|
||||
{
|
||||
"credential": "B.Sc. in Computer Science",
|
||||
"institution": "University X",
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def test_extract_cv_signals_ignores_before_moving_prose_tail() -> None:
|
||||
text = dedent(
|
||||
"""
|
||||
Tonio
|
||||
Location: France
|
||||
Senior Engineer at Microsoft Before moving to Paris.
|
||||
"""
|
||||
).strip()
|
||||
|
||||
extracted = extract_cv_signals(text)
|
||||
|
||||
assert extracted["experience_entries"] == []
|
||||
|
||||
|
||||
def test_extract_pdf_text_skips_blank_pages(monkeypatch) -> None:
|
||||
class FakePage:
|
||||
def __init__(self, text: str | None) -> None:
|
||||
self._text = text
|
||||
|
||||
def extract_text(self) -> str | None:
|
||||
return self._text
|
||||
|
||||
class FakePdfReader:
|
||||
def __init__(self, path: str) -> None:
|
||||
self.path = path
|
||||
self.pages = [
|
||||
FakePage(None),
|
||||
FakePage("Tonio"),
|
||||
FakePage(""),
|
||||
FakePage("Data Engineer"),
|
||||
]
|
||||
|
||||
seen_paths: list[str] = []
|
||||
|
||||
def fake_pdf_reader(path: str) -> FakePdfReader:
|
||||
seen_paths.append(path)
|
||||
return FakePdfReader(path)
|
||||
|
||||
monkeypatch.setattr("job_research.profile.cv_extractor.PdfReader", fake_pdf_reader)
|
||||
|
||||
extracted = extract_pdf_text(Path("cv.pdf"))
|
||||
|
||||
assert extracted == "Tonio\nData Engineer"
|
||||
assert seen_paths == ["cv.pdf"]
|
||||
|
||||
|
||||
def test_extract_pdf_text_rejects_textless_pdfs(monkeypatch) -> None:
|
||||
class FakePage:
|
||||
def extract_text(self) -> str | None:
|
||||
return " "
|
||||
|
||||
class FakePdfReader:
|
||||
def __init__(self, path: str) -> None:
|
||||
self.path = path
|
||||
self.pages = [FakePage(), FakePage()]
|
||||
|
||||
def fake_pdf_reader(path: str) -> FakePdfReader:
|
||||
return FakePdfReader(path)
|
||||
|
||||
monkeypatch.setattr("job_research.profile.cv_extractor.PdfReader", fake_pdf_reader)
|
||||
|
||||
with pytest.raises(ValueError, match="No extractable text"):
|
||||
extract_pdf_text(Path("cv.pdf"))
|
||||
106
tests/profile/test_merge.py
Normal file
106
tests/profile/test_merge.py
Normal file
@ -0,0 +1,106 @@
|
||||
from job_research.profile.merge import build_candidate_profile_output
|
||||
from job_research.profile.profile_parser import AuthoredProfile
|
||||
from job_research.models import WarningItem
|
||||
|
||||
|
||||
def test_build_candidate_profile_output_writes_warning_when_facts_conflict() -> None:
|
||||
cv_signals = {
|
||||
"name": "Tonio",
|
||||
"location": "France",
|
||||
"languages": ["French", "English"],
|
||||
"skills": ["Python", "SQL"],
|
||||
"experience_entries": [{"title": "Data Engineer", "company": "A"}],
|
||||
"education_entries": [],
|
||||
"years_of_experience": 2,
|
||||
}
|
||||
authored = AuthoredProfile(
|
||||
summary="Junior data engineer focused on GCP.",
|
||||
target_roles=["Data Engineer"],
|
||||
strengths=["Python"],
|
||||
skills_to_emphasize=["BigQuery", "GCP"],
|
||||
constraints=["CDI only"],
|
||||
notes=["Years of experience feels closer to 3."],
|
||||
)
|
||||
|
||||
output = build_candidate_profile_output(cv_signals, authored)
|
||||
|
||||
assert output.summary == "Junior data engineer focused on GCP."
|
||||
assert output.constraints == ["CDI only"]
|
||||
assert any(item.field == "years_of_experience" for item in output.warnings)
|
||||
|
||||
|
||||
def test_build_candidate_profile_output_writes_warning_for_french_experience_note() -> None:
|
||||
cv_signals = {
|
||||
"name": "Tonio",
|
||||
"location": "France",
|
||||
"languages": ["French", "English"],
|
||||
"skills": ["Python", "SQL"],
|
||||
"experience_entries": [{"title": "Data Engineer", "company": "A"}],
|
||||
"education_entries": [],
|
||||
"years_of_experience": 2,
|
||||
}
|
||||
authored = AuthoredProfile(
|
||||
summary="Junior data engineer focused on GCP.",
|
||||
target_roles=["Data Engineer"],
|
||||
strengths=["Python"],
|
||||
skills_to_emphasize=["BigQuery", "GCP"],
|
||||
constraints=["CDI only"],
|
||||
notes=["Années d'expérience semble plus proche de 3."],
|
||||
)
|
||||
|
||||
output = build_candidate_profile_output(cv_signals, authored)
|
||||
|
||||
assert any(item.field == "years_of_experience" for item in output.warnings)
|
||||
|
||||
|
||||
def test_build_candidate_profile_output_warns_on_missing_core_cv_facts() -> None:
|
||||
cv_signals = {
|
||||
"location": "France",
|
||||
"languages": ["French", "English"],
|
||||
"skills": [],
|
||||
"experience_entries": [],
|
||||
"education_entries": [],
|
||||
}
|
||||
authored = AuthoredProfile(
|
||||
summary="Junior data engineer focused on GCP."
|
||||
)
|
||||
|
||||
output = build_candidate_profile_output(cv_signals, authored)
|
||||
|
||||
assert [item.field for item in output.warnings] == [
|
||||
"name",
|
||||
"experience_entries",
|
||||
"skills",
|
||||
"education_entries",
|
||||
]
|
||||
|
||||
|
||||
def test_build_candidate_profile_output_propagates_cv_extraction_warnings() -> None:
|
||||
cv_signals = {
|
||||
"name": "Data Engineer | Python | GCP",
|
||||
"location": "France",
|
||||
"languages": ["French", "English"],
|
||||
"skills": ["Python", "SQL"],
|
||||
"experience_entries": [{"title": "Data Engineer", "company": "A"}],
|
||||
"education_entries": [{"credential": "MSc", "institution": "Example University"}],
|
||||
"warnings": [
|
||||
WarningItem(
|
||||
field="name",
|
||||
message="First CV line looks like a header or tagline; review manually.",
|
||||
)
|
||||
],
|
||||
}
|
||||
authored = AuthoredProfile(
|
||||
summary="Junior data engineer focused on GCP.",
|
||||
target_roles=["Data Engineer"],
|
||||
strengths=["Python"],
|
||||
skills_to_emphasize=["BigQuery", "GCP"],
|
||||
constraints=["CDI only"],
|
||||
notes=[],
|
||||
)
|
||||
|
||||
output = build_candidate_profile_output(cv_signals, authored)
|
||||
|
||||
assert output.warnings == [
|
||||
WarningItem(field="name", message="First CV line looks like a header or tagline; review manually.")
|
||||
]
|
||||
109
tests/profile/test_profile_parser.py
Normal file
109
tests/profile/test_profile_parser.py
Normal file
@ -0,0 +1,109 @@
|
||||
import pytest
|
||||
|
||||
from textwrap import dedent
|
||||
|
||||
from job_research.profile.profile_parser import parse_profile_markdown
|
||||
|
||||
|
||||
def test_parse_profile_markdown_reads_light_required_template() -> None:
|
||||
markdown = dedent(
|
||||
"""
|
||||
# Candidate Profile
|
||||
|
||||
## Summary
|
||||
Junior data engineer focused on Python and GCP.
|
||||
|
||||
## Target Roles
|
||||
* Data Engineer
|
||||
+ Analytics Engineer
|
||||
|
||||
## Strengths
|
||||
- Python
|
||||
- SQL
|
||||
|
||||
## Skills To Emphasize
|
||||
- BigQuery
|
||||
- Terraform
|
||||
|
||||
## Constraints
|
||||
- CDI only
|
||||
- France only
|
||||
|
||||
## Notes
|
||||
Slight preference for French listings.
|
||||
"""
|
||||
).strip()
|
||||
|
||||
profile = parse_profile_markdown(markdown)
|
||||
|
||||
assert profile.summary == "Junior data engineer focused on Python and GCP."
|
||||
assert profile.target_roles == ["Data Engineer", "Analytics Engineer"]
|
||||
assert profile.strengths == ["Python", "SQL"]
|
||||
assert profile.skills_to_emphasize == ["BigQuery", "Terraform"]
|
||||
assert profile.constraints == ["CDI only", "France only"]
|
||||
assert profile.notes == ["Slight preference for French listings."]
|
||||
|
||||
|
||||
def test_parse_profile_markdown_rejects_unsupported_list_content() -> None:
|
||||
markdown = dedent(
|
||||
"""
|
||||
# Candidate Profile
|
||||
|
||||
## Summary
|
||||
Junior data engineer focused on Python and GCP.
|
||||
|
||||
## Target Roles
|
||||
- Data Engineer
|
||||
Analytics Engineer
|
||||
|
||||
## Strengths
|
||||
- Python
|
||||
|
||||
## Skills To Emphasize
|
||||
- BigQuery
|
||||
|
||||
## Constraints
|
||||
- CDI only
|
||||
|
||||
## Notes
|
||||
- Slight preference for French listings.
|
||||
"""
|
||||
).strip()
|
||||
|
||||
with pytest.raises(ValueError, match="Unsupported content in section 'target roles'"):
|
||||
parse_profile_markdown(markdown)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("section_name", ["Target Roles", "Notes"])
|
||||
def test_parse_profile_markdown_rejects_blank_bullet_only_required_sections(
|
||||
section_name: str,
|
||||
) -> None:
|
||||
target_roles = "- " if section_name == "Target Roles" else "- Data Engineer"
|
||||
notes = "- " if section_name == "Notes" else "Slight preference for French listings."
|
||||
|
||||
markdown = dedent(
|
||||
f"""
|
||||
# Candidate Profile
|
||||
|
||||
## Summary
|
||||
Junior data engineer focused on Python and GCP.
|
||||
|
||||
## Target Roles
|
||||
{target_roles}
|
||||
|
||||
## Strengths
|
||||
- Python
|
||||
|
||||
## Skills To Emphasize
|
||||
- BigQuery
|
||||
|
||||
## Constraints
|
||||
- CDI only
|
||||
|
||||
## Notes
|
||||
{notes}
|
||||
"""
|
||||
).strip()
|
||||
|
||||
with pytest.raises(ValueError, match=f"Missing usable content in section '{section_name.lower()}'"):
|
||||
parse_profile_markdown(markdown)
|
||||
1467
tests/test_apec_cli.py
Normal file
1467
tests/test_apec_cli.py
Normal file
File diff suppressed because it is too large
Load Diff
82
tests/test_apec_storage.py
Normal file
82
tests/test_apec_storage.py
Normal file
@ -0,0 +1,82 @@
|
||||
from pathlib import Path
|
||||
|
||||
from job_research.models import ApecListing, ApecRunMeta, ApecSnapshotMeta, ListingWarning
|
||||
from job_research.storage import apec_run_paths
|
||||
|
||||
|
||||
FIXED_RUN_ID = "2026-06-01T10-00-00-123456Z"
|
||||
|
||||
|
||||
def test_apec_models_serialize_expected_listing_shape() -> None:
|
||||
listing = ApecListing(
|
||||
source="apec",
|
||||
source_job_id="123",
|
||||
url="https://example.test/job/123",
|
||||
title="Data Engineer",
|
||||
company="Example",
|
||||
location="Paris",
|
||||
contract_type="CDI",
|
||||
description_text="Build pipelines",
|
||||
published_at="2026-06-01",
|
||||
refreshed_at="2026-06-02",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
warnings=[
|
||||
ListingWarning(
|
||||
field="location",
|
||||
message="Location inferred from page text",
|
||||
)
|
||||
],
|
||||
)
|
||||
run_meta = ApecRunMeta(
|
||||
run_id=FIXED_RUN_ID,
|
||||
run_started_at="2026-06-01T10:00:00Z",
|
||||
derived_queries=["Data Engineer"],
|
||||
snapshots=[
|
||||
ApecSnapshotMeta(
|
||||
url="https://example.test/job/123",
|
||||
source_job_id="123",
|
||||
snapshot_file="job-123.html",
|
||||
fetched_at="2026-06-01T10:00:00Z",
|
||||
)
|
||||
],
|
||||
fetched_count=1,
|
||||
normalized_count=1,
|
||||
deduplicated_count=1,
|
||||
failed_count=0,
|
||||
listing_errors=[],
|
||||
)
|
||||
|
||||
assert listing.model_dump()["source"] == "apec"
|
||||
assert listing.model_dump()["warnings"][0]["field"] == "location"
|
||||
assert listing.model_dump()["refreshed_at"] == "2026-06-02"
|
||||
assert run_meta.model_dump()["run_id"] == FIXED_RUN_ID
|
||||
assert run_meta.model_dump()["run_started_at"] == "2026-06-01T10:00:00Z"
|
||||
assert run_meta.model_dump()["derived_queries"] == ["Data Engineer"]
|
||||
assert run_meta.model_dump(mode="json")["snapshots"] == [
|
||||
{
|
||||
"url": "https://example.test/job/123",
|
||||
"source_job_id": "123",
|
||||
"snapshot_file": "job-123.html",
|
||||
"fetched_at": "2026-06-01T10:00:00Z",
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_apec_run_paths_builds_expected_layout(tmp_path: Path) -> None:
|
||||
paths = apec_run_paths(tmp_path, run_id=FIXED_RUN_ID)
|
||||
run_dir = tmp_path / "apec" / "runs" / FIXED_RUN_ID
|
||||
|
||||
assert paths["run_dir"] == run_dir
|
||||
assert paths["listings"] == run_dir / "listings.yaml"
|
||||
assert paths["run_meta"] == run_dir / "run-meta.yaml"
|
||||
assert paths["snapshots"] == run_dir / "snapshots"
|
||||
|
||||
|
||||
def test_apec_run_artifacts_include_snapshot_and_meta(tmp_path: Path) -> None:
|
||||
paths = apec_run_paths(tmp_path, run_id=FIXED_RUN_ID)
|
||||
paths["snapshots"].mkdir(parents=True, exist_ok=True)
|
||||
|
||||
snapshot = paths["snapshots"] / "job-123.html"
|
||||
snapshot.write_text("<html>snapshot</html>", encoding="utf-8")
|
||||
|
||||
assert snapshot.read_text(encoding="utf-8") == "<html>snapshot</html>"
|
||||
364
tests/test_cli.py
Normal file
364
tests/test_cli.py
Normal file
@ -0,0 +1,364 @@
|
||||
from subprocess import run
|
||||
from textwrap import dedent
|
||||
from pathlib import Path
|
||||
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from job_research.cli import app
|
||||
from job_research.storage import load_yaml
|
||||
|
||||
|
||||
def test_installed_cli_help_exposes_build_profile_subcommand() -> None:
|
||||
result = run(["uv", "run", "job-research", "--help"], capture_output=True, text=True, check=False)
|
||||
|
||||
assert result.returncode == 0
|
||||
assert "build-profile" in result.stdout
|
||||
assert "Build candidate-profile.yaml from CV and markdown profile." in result.stdout
|
||||
|
||||
|
||||
def test_installed_cli_subcommand_help_works() -> None:
|
||||
result = run(["uv", "run", "job-research", "build-profile", "--help"], capture_output=True, text=True, check=False)
|
||||
|
||||
assert result.returncode == 0
|
||||
assert "Usage: job-research build-profile" in result.stdout
|
||||
|
||||
|
||||
def test_build_profile_writes_yaml_from_cv_and_profile(tmp_path) -> None:
|
||||
cv = tmp_path / "cv.txt"
|
||||
cv.write_text(
|
||||
dedent(
|
||||
"""
|
||||
Tonio Example
|
||||
Location: France
|
||||
Languages: French, English
|
||||
Skills: Python, SQL, Terraform
|
||||
Years of experience: 3
|
||||
Data Engineer at Acme
|
||||
Education: Master of Science at Example University
|
||||
"""
|
||||
).strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
profile = tmp_path / "profile.md"
|
||||
profile.write_text(
|
||||
dedent(
|
||||
"""
|
||||
# Candidate Profile
|
||||
|
||||
## Summary
|
||||
Junior data engineer focused on Python and GCP.
|
||||
|
||||
## Target Roles
|
||||
- Data Engineer
|
||||
|
||||
## Strengths
|
||||
- Python
|
||||
- SQL
|
||||
|
||||
## Skills To Emphasize
|
||||
- GCP
|
||||
- BigQuery
|
||||
|
||||
## Constraints
|
||||
- CDI only
|
||||
- France only
|
||||
|
||||
## Notes
|
||||
- Slight preference for French listings.
|
||||
- Years of experience feels closer to 4.
|
||||
"""
|
||||
).strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
out = tmp_path / "candidate-profile.yaml"
|
||||
|
||||
result = run(
|
||||
["uv", "run", "job-research", "build-profile", "--cv", str(cv), "--profile", str(profile), "--out", str(out)],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
assert result.returncode == 0
|
||||
assert f"candidate profile written to {out}" in result.stdout
|
||||
assert "Warnings included: 1" in result.stdout
|
||||
|
||||
payload = load_yaml(out)
|
||||
|
||||
assert payload["name"] == "Tonio Example"
|
||||
assert payload["summary"] == "Junior data engineer focused on Python and GCP."
|
||||
assert payload["target_roles"] == ["Data Engineer"]
|
||||
assert payload["skills"] == ["Python", "SQL", "Terraform", "GCP", "BigQuery"]
|
||||
assert any(item["field"] == "years_of_experience" for item in payload["warnings"])
|
||||
|
||||
|
||||
def test_build_profile_reports_when_no_warnings_are_included(tmp_path) -> None:
|
||||
cv = tmp_path / "cv.txt"
|
||||
cv.write_text(
|
||||
dedent(
|
||||
"""
|
||||
Tonio Example
|
||||
Location: France
|
||||
Languages: French, English
|
||||
Skills: Python, SQL
|
||||
Data Engineer at Acme
|
||||
Education: Master of Science at Example University
|
||||
"""
|
||||
).strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
profile = tmp_path / "profile.md"
|
||||
profile.write_text(
|
||||
dedent(
|
||||
"""
|
||||
# Candidate Profile
|
||||
|
||||
## Summary
|
||||
Junior data engineer focused on Python and GCP.
|
||||
|
||||
## Target Roles
|
||||
- Data Engineer
|
||||
|
||||
## Strengths
|
||||
- Python
|
||||
- SQL
|
||||
|
||||
## Skills To Emphasize
|
||||
- GCP
|
||||
- BigQuery
|
||||
|
||||
## Constraints
|
||||
- CDI only
|
||||
- France only
|
||||
|
||||
## Notes
|
||||
- Slight preference for French listings.
|
||||
"""
|
||||
).strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
out = tmp_path / "candidate-profile.yaml"
|
||||
|
||||
result = run(
|
||||
["uv", "run", "job-research", "build-profile", "--cv", str(cv), "--profile", str(profile), "--out", str(out)],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
assert result.returncode == 0
|
||||
assert f"candidate profile written to {out}" in result.stdout
|
||||
assert "No warnings included." in result.stdout
|
||||
|
||||
|
||||
def test_build_profile_reports_output_write_failures_cleanly(tmp_path, monkeypatch) -> None:
|
||||
cv = tmp_path / "cv.txt"
|
||||
cv.write_text(
|
||||
dedent(
|
||||
"""
|
||||
Tonio Example
|
||||
Location: France
|
||||
Languages: French, English
|
||||
Skills: Python, SQL
|
||||
Data Engineer at Acme
|
||||
Education: Master of Science at Example University
|
||||
"""
|
||||
).strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
profile = tmp_path / "profile.md"
|
||||
profile.write_text(
|
||||
dedent(
|
||||
"""
|
||||
# Candidate Profile
|
||||
|
||||
## Summary
|
||||
Junior data engineer focused on Python and GCP.
|
||||
|
||||
## Target Roles
|
||||
- Data Engineer
|
||||
|
||||
## Strengths
|
||||
- Python
|
||||
- SQL
|
||||
|
||||
## Skills To Emphasize
|
||||
- GCP
|
||||
- BigQuery
|
||||
|
||||
## Constraints
|
||||
- CDI only
|
||||
- France only
|
||||
|
||||
## Notes
|
||||
- Slight preference for French listings.
|
||||
"""
|
||||
).strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
out = tmp_path / "candidate-profile.yaml"
|
||||
|
||||
original_write_text = Path.write_text
|
||||
|
||||
def flaky_write_text(
|
||||
self: Path,
|
||||
data: str,
|
||||
encoding: str | None = None,
|
||||
errors: str | None = None,
|
||||
newline: str | None = None,
|
||||
) -> int:
|
||||
if self == out:
|
||||
raise OSError("disk full")
|
||||
|
||||
return original_write_text(self, data, encoding=encoding, errors=errors, newline=newline)
|
||||
|
||||
monkeypatch.setattr(Path, "write_text", flaky_write_text)
|
||||
|
||||
result = CliRunner().invoke(app, ["build-profile", "--cv", str(cv), "--profile", str(profile), "--out", str(out)])
|
||||
|
||||
assert result.exit_code == 1
|
||||
assert not out.exists()
|
||||
assert "Unable to write candidate profile to" in result.stderr
|
||||
assert "disk full" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_build_profile_rejects_empty_cv_text_before_writing(tmp_path) -> None:
|
||||
cv = tmp_path / "cv.txt"
|
||||
cv.write_text(" \n", encoding="utf-8")
|
||||
profile = tmp_path / "profile.md"
|
||||
profile.write_text(
|
||||
dedent(
|
||||
"""
|
||||
# Candidate Profile
|
||||
|
||||
## Summary
|
||||
Junior data engineer.
|
||||
|
||||
## Target Roles
|
||||
- Data Engineer
|
||||
|
||||
## Strengths
|
||||
- Python
|
||||
|
||||
## Skills To Emphasize
|
||||
- BigQuery
|
||||
|
||||
## Constraints
|
||||
- CDI only
|
||||
|
||||
## Notes
|
||||
- Slight preference for French listings.
|
||||
"""
|
||||
).strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
out = tmp_path / "candidate-profile.yaml"
|
||||
|
||||
result = run(
|
||||
["uv", "run", "job-research", "build-profile", "--cv", str(cv), "--profile", str(profile), "--out", str(out)],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
assert result.returncode != 0
|
||||
assert not out.exists()
|
||||
assert "No readable text found in CV input" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_build_profile_reports_unreadable_pdf_input_cleanly(tmp_path, monkeypatch) -> None:
|
||||
cv = tmp_path / "cv.pdf"
|
||||
cv.write_bytes(b"%PDF-1.4\n")
|
||||
profile = tmp_path / "profile.md"
|
||||
profile.write_text(
|
||||
dedent(
|
||||
"""
|
||||
# Candidate Profile
|
||||
|
||||
## Summary
|
||||
Junior data engineer.
|
||||
|
||||
## Target Roles
|
||||
- Data Engineer
|
||||
|
||||
## Strengths
|
||||
- Python
|
||||
|
||||
## Skills To Emphasize
|
||||
- BigQuery
|
||||
|
||||
## Constraints
|
||||
- CDI only
|
||||
|
||||
## Notes
|
||||
- Slight preference for French listings.
|
||||
"""
|
||||
).strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
out = tmp_path / "candidate-profile.yaml"
|
||||
|
||||
def broken_extract_pdf_text(path):
|
||||
raise ValueError("broken pdf")
|
||||
|
||||
monkeypatch.setattr("job_research.cli.extract_pdf_text", broken_extract_pdf_text)
|
||||
|
||||
result = CliRunner().invoke(app, ["build-profile", "--cv", str(cv), "--profile", str(profile), "--out", str(out)])
|
||||
|
||||
assert result.exit_code == 1
|
||||
assert not out.exists()
|
||||
assert "CV input not readable" in result.stderr
|
||||
assert "broken pdf" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_build_profile_reports_malformed_profile_markdown_cleanly(tmp_path) -> None:
|
||||
cv = tmp_path / "cv.txt"
|
||||
cv.write_text(
|
||||
dedent(
|
||||
"""
|
||||
Tonio Example
|
||||
Location: France
|
||||
Languages: French, English
|
||||
Skills: Python
|
||||
"""
|
||||
).strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
profile = tmp_path / "profile.md"
|
||||
profile.write_text(
|
||||
dedent(
|
||||
"""
|
||||
# Candidate Profile
|
||||
|
||||
## Summary
|
||||
Junior data engineer.
|
||||
|
||||
## Target Roles
|
||||
Data Engineer
|
||||
|
||||
## Strengths
|
||||
- Python
|
||||
|
||||
## Skills To Emphasize
|
||||
- BigQuery
|
||||
|
||||
## Constraints
|
||||
- CDI only
|
||||
|
||||
## Notes
|
||||
- Slight preference for French listings.
|
||||
"""
|
||||
).strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
out = tmp_path / "candidate-profile.yaml"
|
||||
|
||||
result = CliRunner().invoke(app, ["build-profile", "--cv", str(cv), "--profile", str(profile), "--out", str(out)])
|
||||
|
||||
assert result.exit_code == 1
|
||||
assert not out.exists()
|
||||
assert "profile markdown invalid" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
39
tests/test_storage.py
Normal file
39
tests/test_storage.py
Normal file
@ -0,0 +1,39 @@
|
||||
import pytest
|
||||
|
||||
from job_research.models import CandidateProfileOutput, ExperienceEntry, WarningItem
|
||||
from job_research.storage import save_candidate_profile_yaml, load_yaml
|
||||
|
||||
|
||||
def test_save_candidate_profile_yaml_round_trips_readable_output(tmp_path) -> None:
|
||||
profile = CandidateProfileOutput(
|
||||
name="Tonio",
|
||||
summary="Junior data engineer focused on Python and GCP.",
|
||||
target_roles=["Data Engineer"],
|
||||
strengths=["Python", "SQL"],
|
||||
skills_to_emphasize=["GCP", "BigQuery"],
|
||||
constraints=["CDI only", "France only"],
|
||||
notes=["Slight preference for French listings."],
|
||||
location="France",
|
||||
languages=["French", "English"],
|
||||
skills=["Python", "SQL", "Terraform", "GCP", "BigQuery"],
|
||||
experience_entries=[ExperienceEntry(company="A", title="Data Engineer")],
|
||||
education_entries=[],
|
||||
warnings=[WarningItem(field="years_of_experience", message="CV and profile disagree.")],
|
||||
)
|
||||
|
||||
out = tmp_path / "candidate-profile.yaml"
|
||||
save_candidate_profile_yaml(out, profile)
|
||||
|
||||
payload = load_yaml(out)
|
||||
|
||||
assert payload["name"] == "Tonio"
|
||||
assert payload["constraints"] == ["CDI only", "France only"]
|
||||
assert payload["warnings"][0]["field"] == "years_of_experience"
|
||||
|
||||
|
||||
def test_load_yaml_rejects_non_mapping_root(tmp_path) -> None:
|
||||
path = tmp_path / "candidate-profile.yaml"
|
||||
path.write_text("[]", encoding="utf-8")
|
||||
|
||||
with pytest.raises(ValueError, match="mapping"):
|
||||
load_yaml(path)
|
||||
412
uv.lock
generated
412
uv.lock
generated
@ -2,7 +2,417 @@ version = 1
|
||||
revision = 3
|
||||
requires-python = ">=3.13"
|
||||
|
||||
[[package]]
|
||||
name = "annotated-doc"
|
||||
version = "0.0.4"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/57/ba/046ceea27344560984e26a590f90bc7f4a75b06701f653222458922b558c/annotated_doc-0.0.4.tar.gz", hash = "sha256:fbcda96e87e9c92ad167c2e53839e57503ecfda18804ea28102353485033faa4", size = 7288, upload-time = "2025-11-10T22:07:42.062Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/1e/d3/26bf1008eb3d2daa8ef4cacc7f3bfdc11818d111f7e2d0201bc6e3b49d45/annotated_doc-0.0.4-py3-none-any.whl", hash = "sha256:571ac1dc6991c450b25a9c2d84a3705e2ae7a53467b5d111c24fa8baabbed320", size = 5303, upload-time = "2025-11-10T22:07:40.673Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "annotated-types"
|
||||
version = "0.7.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/ee/67/531ea369ba64dcff5ec9c3402f9f51bf748cec26dde048a2f973a4eea7f5/annotated_types-0.7.0.tar.gz", hash = "sha256:aff07c09a53a08bc8cfccb9c85b05f1aa9a2a6f23728d790723543408344ce89", size = 16081, upload-time = "2024-05-20T21:33:25.928Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/78/b6/6307fbef88d9b5ee7421e68d78a9f162e0da4900bc5f5793f6d3d0e34fb8/annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53", size = 13643, upload-time = "2024-05-20T21:33:24.1Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "beautifulsoup4"
|
||||
version = "4.14.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "soupsieve" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c3/b0/1c6a16426d389813b48d95e26898aff79abbde42ad353958ad95cc8c9b21/beautifulsoup4-4.14.3.tar.gz", hash = "sha256:6292b1c5186d356bba669ef9f7f051757099565ad9ada5dd630bd9de5fa7fb86", size = 627737, upload-time = "2025-11-30T15:08:26.084Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/1a/39/47f9197bdd44df24d67ac8893641e16f386c984a0619ef2ee4c51fbbc019/beautifulsoup4-4.14.3-py3-none-any.whl", hash = "sha256:0918bfe44902e6ad8d57732ba310582e98da931428d231a5ecb9e7c703a735bb", size = 107721, upload-time = "2025-11-30T15:08:24.087Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "colorama"
|
||||
version = "0.4.6"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "greenlet"
|
||||
version = "3.5.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/6d/6e/802acd792aebb2256fbbee8cacf2727faaeb6f240ac11008f09eae4414bc/greenlet-3.5.1.tar.gz", hash = "sha256:5a56aeb7d5d9cc4b3a735efb5095bd4b4f6f0e4f93e5ca876d0e2315137b7829", size = 197356, upload-time = "2026-05-20T15:05:03.917Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/27/69/7f7e5372d998b81001899b1c0823c957aa413ba0f2662e65821611cc31e4/greenlet-3.5.1-cp313-cp313-macosx_11_0_universal2.whl", hash = "sha256:51518ff74664078fc51bffcc6fc529b0df5ae58da192691cee765d45ce944a2b", size = 285060, upload-time = "2026-05-20T13:08:51.899Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b1/bf/387f9b6b865fd2ae0d0be09e0004827295a01b71be76ed350dd1e28a91a4/greenlet-3.5.1-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1ffdb3c0bb002c99cd8f298957e046c3dbf6006b5b7cdf11a4e19194624a0a0a", size = 604370, upload-time = "2026-05-20T14:00:07.492Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/32/f5/169ce3d4e4c67291bd18f8cbe0299c9f3e45102c7f1fb3c14780c93e4532/greenlet-3.5.1-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7715a5a2c3378ba602c3a440558261e13a820bb53a82693aacd7b7f6d964e283", size = 616987, upload-time = "2026-05-20T14:05:44.237Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/19/ba/c24110c55dffa55aa6e1d98b45310da33801aeba7686ff0190fe5d46fd32/greenlet-3.5.1-cp313-cp313-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d40a890035c0058cadbdc4af7569800fd28a0e527a0fdbb7b5f9418f176846ce", size = 622911, upload-time = "2026-05-20T14:09:10.598Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ee/e5/7f2e41d5273be07e77560d61ea4e56485b4d6c316d2a84518c62d1364061/greenlet-3.5.1-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:dc71ff466927a201b08305acac451ebe1aedfcea002f62f1f2f2ac2ac1e6a135", size = 613911, upload-time = "2026-05-20T13:14:27.539Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ec/7b/d20db2e8a5ad6c038702f3179b136f93f0a3d1a21a0c0777f3e470cdf4b2/greenlet-3.5.1-cp313-cp313-manylinux_2_39_riscv64.whl", hash = "sha256:67821bb03e4e98664490edb787ff6af501194c29bbee0f5c1dfdcf1dc3d9d436", size = 425228, upload-time = "2026-05-20T14:01:40.837Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c5/a4/fbdc67579b73615a1f91615e814303cc71e06128f7baaba87be79b8fb90c/greenlet-3.5.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:cd443683db272ebaaca03af98c0b063ab30db70ea8a31a1559f35e3f7b744ccd", size = 1570689, upload-time = "2026-05-20T14:02:27.225Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e6/b4/77abbe35078be39718a46cd49caf16bceb35662f97a34101dca28aa98e47/greenlet-3.5.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:089fff7a6ce8d9316d1f65ebc00273a56be258c1725b32b94de90a3a979557e1", size = 1635602, upload-time = "2026-05-20T13:14:36.344Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/37/f7/129f27ca700845b8ee8ca88ce7f43435a1239c2eddb7677fc938822762cf/greenlet-3.5.1-cp313-cp313-win_amd64.whl", hash = "sha256:110a1ca7b49b014b097f6078272c3f4ed31af45b254de5228b79adba879f6af9", size = 238683, upload-time = "2026-05-20T13:11:50.57Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6d/5c/a485a36e87df8d8fd0632ee01511244f5156a20ed3746cc6599340326395/greenlet-3.5.1-cp313-cp313-win_arm64.whl", hash = "sha256:f16ba1efc0715b680a18b8123d90dad887c6112ae3555b4b5c32c149540c6b4e", size = 235499, upload-time = "2026-05-20T13:12:42.028Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8a/cb/c62454606daf5640369c94d8a9dd540599b1bfc090e2d2180cb77f4038d2/greenlet-3.5.1-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:d8ab31c9de8651a2facdd5c5bb0011f2380dd1a7af78ce2adf4b56095294fc07", size = 285579, upload-time = "2026-05-20T13:08:56.396Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ec/71/c4270398c2eba968a6071af1dfbdcaeee6ec1c24bc8b435b8cc452700da6/greenlet-3.5.1-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5e300185139abc337ade480c327183adf42a875ac7181bfe66d7d4efea31fbea", size = 651106, upload-time = "2026-05-20T14:00:09.448Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1a/ab/71e34b78a44ec271fb5f550c17bc46d301ddc5953890d935f270b0dcdb5a/greenlet-3.5.1-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7ffdb990dcaa0234cf9845aead5df2e3c3a8b6507d409274dd87e0d5ab05ffc2", size = 663478, upload-time = "2026-05-20T14:05:45.88Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c6/2d/2d80842910da44f78c286532d084b8a5c3717c844ae80ceb3858738ae89a/greenlet-3.5.1-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:6c09df69dc1712d131332054a858a3e5cca400967fa3a672e2324fbb0971448c", size = 667767, upload-time = "2026-05-20T14:09:12.15Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/77/96/4efd6fa5c62c85426a0c19077a586258ebc3a2a146ff2493e4312a697a22/greenlet-3.5.1-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2f82b3597e9d83b63408affed0b48fd0f54935edac4302237b9a837be0dae33c", size = 660800, upload-time = "2026-05-20T13:14:29.129Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e9/d3/dad2eecedfbb1ed7050a20dcfae40c1442b74bc7423608be2c7e03ee7133/greenlet-3.5.1-cp314-cp314-manylinux_2_39_riscv64.whl", hash = "sha256:a4764e0bfc6a4d114c865b32520805c16a990ef5f286a514413b05d5ecd6a23d", size = 470786, upload-time = "2026-05-20T14:01:42.064Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7a/e0/6c71401a25cac7000261304e866a2f2cc04dc74810d40e2f118aa4799495/greenlet-3.5.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c0141e37414c10164e702b8fb1473304221ad98f71600850c6ef7ff4880feba0", size = 1617518, upload-time = "2026-05-20T14:02:28.662Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/41/26/c5c06643e8c0af9e7bf18e16cb51d0ab7625155f0392e1c9015d66d556cd/greenlet-3.5.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:50ae25a67bea74ea41fb14b960bc532df73eb713417b2d61892dced82fe8d3bc", size = 1681593, upload-time = "2026-05-20T13:14:39.417Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8a/bd/e11a108317485075e68af9d23039619b86b28130c3b50d227d42edece64b/greenlet-3.5.1-cp314-cp314-win_amd64.whl", hash = "sha256:8a17c42330e261299766b75ac1ea32caa437a9453c8f65d16a13140db378ecd3", size = 239800, upload-time = "2026-05-20T13:09:30.128Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/47/f8/8e8e8417b7bf28639a5a56356ef934d0375e1d0c70a57e04d7701e870ffe/greenlet-3.5.1-cp314-cp314-win_arm64.whl", hash = "sha256:7b5f5fae05b8ac6d176a61b60c394a8cbdc2b5b91b81793066e68745cf165e54", size = 236862, upload-time = "2026-05-20T13:09:10.498Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/90/12/41bf27fde4d3605d3773ae57751eda182b8be2f5398011c041173b1d9534/greenlet-3.5.1-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:ea8da1e900d758d078810d4255d8c6aa572181896a31ec79d779eb79c3adc9ad", size = 293637, upload-time = "2026-05-20T13:12:35.529Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/44/44/ba14b23e9757707050c2f397d305bbcae62e5d7cad122f8b6baec5ae4a1f/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a19570c52a21420dcbc94e661994bc325c0b5b11304540fed514586da5dc8f2e", size = 650840, upload-time = "2026-05-20T14:00:11.079Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a8/37/5ddc2b686a6844f91abecef43411842426da2e1573f60b49ecf2547f4ae1/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3d955c89b75eeca4723d7cc14135f393cd47c32e2a6cb4a8e4c6e760a26b0986", size = 656416, upload-time = "2026-05-20T14:05:47.118Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8c/46/5987dcd1a2570ba84f3b187536b2ca3ae97613387e57f5cfa99df068fe5e/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:ea37d5a157eb9493820d3792ac4ece28619a394391d2b9f2f78057d396ff0f0f", size = 656607, upload-time = "2026-05-20T14:09:13.949Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e1/f0/d17510297c35a2992712f0bf84de3779749999f7d3d63aa1f09db7c62dbe/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:de2daaaebd1a5aa88c49045b6baf9310b3263796bd88db713edf37cf53e7bb4e", size = 654397, upload-time = "2026-05-20T13:14:30.696Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2c/c1/6da0a9ddcc29d7e51ef14883fa3dc1e53b3f4ffba00582106c7bf55da1d8/greenlet-3.5.1-cp314-cp314t-manylinux_2_39_riscv64.whl", hash = "sha256:8d8a23250ea3ec7b36de8fa4b541e9e2db3ee82915cc060ab0631609ad8b28de", size = 488287, upload-time = "2026-05-20T14:01:43.143Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/37/eb/147387705bb89092645b012586e7273cb5ed3c90ef7eaf3a69173eaf0209/greenlet-3.5.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:3bfbd69cc349e43bf3a8ae1c85548ff0718efc887615c2db16c3833d7b0b072d", size = 1614469, upload-time = "2026-05-20T14:02:30.192Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a6/4e/37ee0da7732b7aa9896f17e15579a9df34b9fcb9dd494f0adfa749af6623/greenlet-3.5.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:4378720dd888136c27215a0214d32a4d37c3852765d45bc37aad0623423cfd78", size = 1675115, upload-time = "2026-05-20T13:14:40.972Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/57/f3/97dfcf4a6eb5077f8a672234216fb5923eb89f2cab7081cb10b2cf75b605/greenlet-3.5.1-cp314-cp314t-win_amd64.whl", hash = "sha256:45718441607f9325d948db98cbc691276059316d0358c188c246da4e1d4d23d2", size = 245246, upload-time = "2026-05-20T13:12:22.646Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5d/73/d7f72e34b582f694f4a9b248162db7b09cc458a259ba8f0c0bfa1a34ea7d/greenlet-3.5.1-cp315-cp315-macosx_11_0_universal2.whl", hash = "sha256:2baee5ca02031757ffe8cc3d69f0cc0aec7065ce362622da74f32d3bcab1c541", size = 285575, upload-time = "2026-05-20T13:12:07.043Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/df/59/fa9c6e87dc8ad27a95dabe2f29f372b733d05a8a67470f6c901ed9975655/greenlet-3.5.1-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9b1ec3274918a81d3ea778b9e75b56b72b33f300edb6cf7f3a7fe1dae56683de", size = 656428, upload-time = "2026-05-20T14:00:12.556Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f6/f9/e753408871eaa61dfe35e619cfc67512b036fde99893685d50eea9e07146/greenlet-3.5.1-cp315-cp315-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:111e2390ffffc47d5840b01711dd7fac07d4c09283d0283e7f3264b14e284c64", size = 667064, upload-time = "2026-05-20T14:05:48.662Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/dc/74/807a047255bf1e09303627c46dc043dca596b6958a354d904f32ab382005/greenlet-3.5.1-cp315-cp315-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:10a9a1c0bfbc93d41156ffcb90c75fbc05544054faf15dcc1fdf9765f8b607f0", size = 672962, upload-time = "2026-05-20T14:09:15.532Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/96/27/5565b5b40389f1c7753003a07e21892fda8660926787036d5bc0308b8113/greenlet-3.5.1-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e630136e905fe5ff43e86945ae41220b6d1470956a39220e708110ac48d01ea5", size = 665697, upload-time = "2026-05-20T13:14:32.943Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/76/32/19d4e13225193c29b13e308015223f7d75fd3d8623d49dd19040d2ce8ec1/greenlet-3.5.1-cp315-cp315-manylinux_2_39_riscv64.whl", hash = "sha256:ef08c1567c78074b22d1a200183d52d04a14df447bf70bcbb6a3507a48e776fc", size = 476047, upload-time = "2026-05-20T14:01:44.39Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cf/82/e7de4178c0c2d1c9a5a3be3cc0b33e46a85b3ee4a77c071bf7ad8600e079/greenlet-3.5.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:975eac34b44a7077ca4d421348455b94f0f518246a7f14bc6d2fdcfe5b584368", size = 1621256, upload-time = "2026-05-20T14:02:31.91Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/00/10/f2dddcf7dacac17dfc68691809589adad06135eb28930429cf58a6467a2f/greenlet-3.5.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:9ab3c3a0b2ae6198e67c898dad5215a49f9ae0d0081b3c3ec59f333e39eeca26", size = 1685956, upload-time = "2026-05-20T13:14:42.55Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/22/17/4a232b32133230ada52f70e9d7f5b65b0caef8772f01849bd8d149e7e4ca/greenlet-3.5.1-cp315-cp315-win_amd64.whl", hash = "sha256:cbfc69be86e10dcfef5b1e6269d1d6926552aa89ee39e1de3353360c1b6989ab", size = 239802, upload-time = "2026-05-20T13:13:15.481Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c2/ae/4e623a7e6d4d2a5f4cb8e4c82de4169fc637942caae68d6e676b8a128ac5/greenlet-3.5.1-cp315-cp315-win_arm64.whl", hash = "sha256:92fd6d44ac5e5a887c8a5dc4a8ba0ba908527c31c12f78c6bc7dcfe8aab279f6", size = 236853, upload-time = "2026-05-20T13:15:37.301Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7a/57/816d9cff29119da3505b3d6a5e14a8af89006ac36f47f891ff293ee05af1/greenlet-3.5.1-cp315-cp315t-macosx_11_0_universal2.whl", hash = "sha256:a6fdf2433a5441ef9a95464f7c3e674775da1c8c1177fff311cee1acad4626ed", size = 293877, upload-time = "2026-05-20T13:10:19.078Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/23/a1/59b0a7c7d140ff1a75626680b9a9899b79a9176cab298b394968fb023295/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7546556f0d649f99f6a361098a55f761181bb2ea12ff150bb16d26092ad88244", size = 655333, upload-time = "2026-05-20T14:00:14.758Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/72/1b/5efe127597625042218939d01855109f352779050768b670b52edcc16a6c/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d5ee3ea898009fa898f85f9982255d35278c477bebe185beca249cab42d4526c", size = 659443, upload-time = "2026-05-20T14:05:50.159Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c9/9d/1dcdf7b95ab3cf8c7b6d7277c18a5e167312f2b362ddfcc5d5e6d8d84b43/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a57b0d05a0448eed231d59c0ceb287dde984551e54cbc51ac2d4865712838e9c", size = 659998, upload-time = "2026-05-20T14:09:16.912Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6c/6d/c404246ea4d22d097a7426d0efb5b781bd7eb67715f09e79001bd552ab18/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a5c81f74d204d3edd136ebfd50dce53acbb776995d721a0fe801626cfc93b8cd", size = 658356, upload-time = "2026-05-20T13:14:35.091Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/05/7e/c4959664fc231d587d66d8e81f2095e98056ba1954beafdcbe635e251052/greenlet-3.5.1-cp315-cp315t-manylinux_2_39_riscv64.whl", hash = "sha256:b0703c2cef53e01baec47f7a3868009913ad71ec678bbecb42a6f40895e4ce62", size = 494470, upload-time = "2026-05-20T14:01:45.611Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/51/02/f8ee37fb6d2219329f350af241c27fcf12df57e723d11f6fc6d3bacdadaa/greenlet-3.5.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:2c18ef16bf6d4dd410e4dd52996888ea1497be26892fe5bbc73580aba4287b8e", size = 1619216, upload-time = "2026-05-20T14:02:33.403Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/93/c5/3dc9475ace2c7a3680da12372cddd7f1ac874eb410a1ac48d3e9dab83782/greenlet-3.5.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:17d86354f0ae6b61bf9be5148d0dd34e06c3cb7c602c671f79f29ac3b150e659", size = 1678427, upload-time = "2026-05-20T13:14:43.71Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/df/4e/750c15c317a41ffb36f0bf40b933e3d744a7dede61889f74443ea69690cf/greenlet-3.5.1-cp315-cp315t-win_amd64.whl", hash = "sha256:e7516cf6ae6b8a582c2770a0caed47b8a48373ed732c33d69a72913ae6ac923e", size = 245225, upload-time = "2026-05-20T13:13:59.366Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4f/fd/d3baea2eeb7b617efd47e87ca06e2ec2c6118d303aa9e918e0ce16eadc10/greenlet-3.5.1-cp315-cp315t-win_arm64.whl", hash = "sha256:5028648bf2253ec4745add746129d3904121fa7fe871a76bed23c5720573ce0a", size = 239590, upload-time = "2026-05-20T13:13:37.382Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "iniconfig"
|
||||
version = "2.3.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "job-research"
|
||||
version = "0.1.0"
|
||||
source = { virtual = "." }
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "beautifulsoup4" },
|
||||
{ name = "playwright" },
|
||||
{ name = "pydantic" },
|
||||
{ name = "pypdf" },
|
||||
{ name = "pyyaml" },
|
||||
{ name = "typer" },
|
||||
]
|
||||
|
||||
[package.dev-dependencies]
|
||||
dev = [
|
||||
{ name = "pytest" },
|
||||
]
|
||||
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "beautifulsoup4", specifier = ">=4.12,<5" },
|
||||
{ name = "playwright", specifier = ">=1.52,<2" },
|
||||
{ name = "pydantic", specifier = ">=2.7,<3" },
|
||||
{ name = "pypdf", specifier = ">=5.0,<6" },
|
||||
{ name = "pyyaml", specifier = ">=6.0,<7" },
|
||||
{ name = "typer", specifier = ">=0.12,<1" },
|
||||
]
|
||||
|
||||
[package.metadata.requires-dev]
|
||||
dev = [{ name = "pytest", specifier = ">=8.2,<9" }]
|
||||
|
||||
[[package]]
|
||||
name = "markdown-it-py"
|
||||
version = "4.2.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "mdurl" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mdurl"
|
||||
version = "0.1.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "packaging"
|
||||
version = "26.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "playwright"
|
||||
version = "1.60.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "greenlet" },
|
||||
{ name = "pyee" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/21/f0/832bd9677194908da118064eef20082f2791e3d18215cc6d9391ee2c5a67/playwright-1.60.0-py3-none-macosx_10_13_x86_64.whl", hash = "sha256:6a8cd0fec171fb3089e95e898c8bc8a6f35dea0b78b399e12fcc19427e91b1d7", size = 43474635, upload-time = "2026-05-18T12:00:31.969Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/59/7b/e1d32ae8a3ed937ec2be3721c5f728b13d731a0b7c6442e0b3bec5094ac0/playwright-1.60.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:39b5420ba6145045b69ced4c5c47d4d9fe5bddfc8ff816c518913afcb25ec7a5", size = 42261327, upload-time = "2026-05-18T12:00:35.638Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d7/bc/23de499ded6411c188a20c5a0dea6f0cd4ed5d2b3cc6042a5dbd3ed609aa/playwright-1.60.0-py3-none-macosx_11_0_universal2.whl", hash = "sha256:2581d0e6a3392c71f91b27460c7fd093356818dc430f48153896c8aeeaef7705", size = 43474636, upload-time = "2026-05-18T12:00:39.294Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/22/7b/1d679f4fced4ea94efadd17103856d8c565384f68382a1681264e46f5925/playwright-1.60.0-py3-none-manylinux1_x86_64.whl", hash = "sha256:1c2bfae7884fb3fb05b853290eab8f343d524e5016f2f1def702acbbdf14c93e", size = 47467220, upload-time = "2026-05-18T12:00:43.179Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/84/c2/1528d267d4442bd2c6b8eaeab819dd52c2030bf80e89293f0ba1f687473b/playwright-1.60.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:43e66564125ee31b07a58cefb21e256d62d67d8d1713e6858df7a3019d8ed353", size = 47154856, upload-time = "2026-05-18T12:00:46.715Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/bb/4e/b008b6440a7a1624378041da94829956d4b8f7ab9ef5aad22d0dc3f2e26d/playwright-1.60.0-py3-none-win32.whl", hash = "sha256:ec94e416ea320711e0ad4bf185dcbf41833672961e90773e1885255d7db7b7e7", size = 37902157, upload-time = "2026-05-18T12:00:50.374Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/55/f0/0541524133104f9cc20bf900870ff4a736b76a23483f3a55295ddfa58409/playwright-1.60.0-py3-none-win_amd64.whl", hash = "sha256:9566821ce6030a1f9e7146a24e19355ab0d98805fd0f9be50bb3d8fef1750c02", size = 37902159, upload-time = "2026-05-18T12:00:53.728Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/80/c8/210f282d278e4709cdd71b12a31af45a30a22ab3207b387e29b37e478713/playwright-1.60.0-py3-none-win_arm64.whl", hash = "sha256:6e4f6700a4c2250efff8e690a81d66e3855754fb587b6b87cf5c784014f91537", size = 34037981, upload-time = "2026-05-18T12:00:57.584Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pluggy"
|
||||
version = "1.6.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pydantic"
|
||||
version = "2.13.4"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "annotated-types" },
|
||||
{ name = "pydantic-core" },
|
||||
{ name = "typing-extensions" },
|
||||
{ name = "typing-inspection" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/18/a5/b60d21ac674192f8ab0ba4e9fd860690f9b4a6e51ca5df118733b487d8d6/pydantic-2.13.4.tar.gz", hash = "sha256:c40756b57adaa8b1efeeced5c196f3f3b7c435f90e84ea7f443901bec8099ef6", size = 844775, upload-time = "2026-05-06T13:43:05.343Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/fd/7b/122376b1fd3c62c1ed9dc80c931ace4844b3c55407b6fb2d199377c9736f/pydantic-2.13.4-py3-none-any.whl", hash = "sha256:45a282cde31d808236fd7ea9d919b128653c8b38b393d1c4ab335c62924d9aba", size = 472262, upload-time = "2026-05-06T13:43:02.641Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pydantic-core"
|
||||
version = "2.46.4"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/9d/56/921726b776ace8d8f5db44c4ef961006580d91dc52b803c489fafd1aa249/pydantic_core-2.46.4.tar.gz", hash = "sha256:62f875393d7f270851f20523dd2e29f082bcc82292d66db2b64ea71f64b6e1c1", size = 471464, upload-time = "2026-05-06T13:37:06.98Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/51/a2/5d30b469c5267a17b39dec53208222f76a8d351dfac4af661888c5aee77d/pydantic_core-2.46.4-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:5d5902252db0d3cedf8d4a1bc68f70eeb430f7e4c7104c8c476753519b423008", size = 2106306, upload-time = "2026-05-06T13:37:48.029Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c1/81/4fa520eaffa8bd7d1525e644cd6d39e7d60b1592bc5b516693c7340b50f1/pydantic_core-2.46.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c94f0688e7b8d0a67abf40e57a7eaaecd17cc9586706a31b76c031f63df052b4", size = 1951906, upload-time = "2026-05-06T13:37:17.012Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/03/d5/fd02da45b659668b05923b17ba3a0100a0a3d5541e3bd8fcc4ecb711309e/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f027324c56cd5406ca49c124b0db10e56c69064fec039acc571c29020cc87c76", size = 1976802, upload-time = "2026-05-06T13:37:35.113Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/21/f2/95727e1368be3d3ed485eaab7adbd7dda408f33f7a36e8b48e0144002b91/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e739fee756ba1010f8bcccb534252e85a35fe45ae92c295a06059ce58b74ccd3", size = 2052446, upload-time = "2026-05-06T13:37:12.313Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9c/86/5d99feea3f77c7234b8718075b23db11532773c1a0dbd9b9490215dc2eeb/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9d56801be94b86a9da183e5f3766e6310752b99ff647e38b09a9500d88e46e76", size = 2232757, upload-time = "2026-05-06T13:39:01.149Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d2/3a/508ac615935ef7588cf6d9e9b91309fdc2da751af865e02a9098de88258c/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2412e734dcb48da14d4e4006b82b46b74f2518b8a26ee7e58c6844a6cd6d03c4", size = 2309275, upload-time = "2026-05-06T13:37:41.406Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/07/f8/41db9de19d7987d6b04715a02b3b40aea467000275d9d758ffaa31af7d50/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9551187363ffc0de2a00b2e47c25aeaeb1020b69b668762966df15fc5659dd5a", size = 2094467, upload-time = "2026-05-06T13:39:18.847Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2c/e2/f35033184cb11d0052daf4416e8e10a502ea2ac006fc4f459aee872727d1/pydantic_core-2.46.4-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:0186750b482eefa11d7f435892b09c5c606193ef3375bcf94aa00ae6bfb66262", size = 2134417, upload-time = "2026-05-06T13:40:17.944Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/7b/6ceeb1cc90e193862f444ebe373d8fdf613f0a82572dde03fb10734c6c71/pydantic_core-2.46.4-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:5855698a4856556d86e8e6cd8434bc3ac0314ee8e12089ae0e143f64c6256e4e", size = 2179782, upload-time = "2026-05-06T13:40:32.618Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5a/f2/c8d7773ede6af08036423a00ae0ceffce266c3c52a096c435d68c896083f/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:cbaf13819775b7f769bf4a1f066cb6df7a28d4480081a589828ef190226881cd", size = 2188782, upload-time = "2026-05-06T13:36:51.018Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/59/31/0c864784e31f09f05cdd87606f08923b9c9e7f6e51dd27f20f62f975ce9f/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_armv7l.whl", hash = "sha256:633147d34cf4550417f12e2b1a0383973bdf5cdfde212cb09e9a581cf10820be", size = 2328334, upload-time = "2026-05-06T13:40:37.764Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c2/eb/4f6c8a41efa30baa755590f4141abf3a8c370fab610915733e74134a7270/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:82cf5301172168103724d49a1444d3378cb20cdee30b116a1bd6031236298a5d", size = 2372986, upload-time = "2026-05-06T13:39:34.152Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5b/24/b375a480d53113860c299764bfe9f349a3dc9108b3adc0d7f0d786492ebf/pydantic_core-2.46.4-cp313-cp313-win32.whl", hash = "sha256:9fa8ae11da9e2b3126c6426f147e0fba88d96d65921799bb30c6abd1cb2c97fb", size = 1973693, upload-time = "2026-05-06T13:37:55.072Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/e8/cff247591966f2d22ec8c003cd7587e27b7ba7b81ab2fb888e3ab75dc285/pydantic_core-2.46.4-cp313-cp313-win_amd64.whl", hash = "sha256:6b3ace8194b0e5204818c92802dcdca7fc6d88aabbb799d7c795540d9cd6d292", size = 2071819, upload-time = "2026-05-06T13:38:49.139Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c6/1a/f4aee670d5670e9e148e0c82c7db98d780be566c6e6a97ee8035528ca0b3/pydantic_core-2.46.4-cp313-cp313-win_arm64.whl", hash = "sha256:184c081504d17f1c1066e430e117142b2c77d9448a97f7b65c6ac9fd9aee238d", size = 2027411, upload-time = "2026-05-06T13:40:45.796Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8d/74/228a26ddad29c6672b805d9fd78e8d251cd04004fa7eed0e622096cd0250/pydantic_core-2.46.4-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:428e04521a40150c85216fc8b85e8d39fece235a9cf5e383761238c7fa9b96fb", size = 2102079, upload-time = "2026-05-06T13:38:41.019Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ad/1f/8970b150a4b4365623ae00fc88603491f763c627311ae8031e3111356d6e/pydantic_core-2.46.4-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:23ace664830ee0bfe014a0c7bc248b1f7f25ed7ad103852c317624a1083af462", size = 1952179, upload-time = "2026-05-06T13:36:59.812Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/95/30/5211a831ae054928054b2f79731661087a2bc5c01e825c672b3a4a8f1b3e/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ce5c1d2a8b27468f433ca974829c44060b8097eedc39933e3c206a90ee49c4a9", size = 1978926, upload-time = "2026-05-06T13:37:39.933Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/57/e9/689668733b1eb67adeef047db3c2e8788fcf65a7fd9c9e2b46b7744fe245/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:7283d57845ecf5a163403eb0702dfc220cc4fbdd18919cb5ccea4f95ee1cdab4", size = 2046785, upload-time = "2026-05-06T13:38:01.995Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/60/d9/6715260422ff50a2109878fd24d948a6c3446bb2664f34ee78cd972b3acd/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8daafc69c93ee8a0204506a3b6b30f586ef54028f52aeeeb5c4cfc5184fd5914", size = 2228733, upload-time = "2026-05-06T13:40:50.371Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/18/ae/fdb2f64316afca925640f8e70bb1a564b0ec2721c1389e25b8eb4bf9a299/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cd2213145bcc2ba85884d0ac63d222fece9209678f77b9b4d76f054c561adb28", size = 2307534, upload-time = "2026-05-06T13:37:21.531Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/89/1d/8eff589b45bb8190a9d12c49cfad0f176a5cbd1534908a6b5125e2886239/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7a5f930472650a82629163023e630d160863fce524c616f4e5186e5de9d9a49b", size = 2099732, upload-time = "2026-05-06T13:39:31.942Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/06/d5/ee5a3366637fee41dee51a1fc91562dcf12ddbc68fda34e6b253da2324bb/pydantic_core-2.46.4-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:c1b3f518abeca3aa13c712fd202306e145abf59a18b094a6bafb2d2bbf59192c", size = 2129627, upload-time = "2026-05-06T13:37:25.033Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/94/33/2414be571d2c6a6c4d08be21f9292b6d3fdb08949a97b6dfe985017821db/pydantic_core-2.46.4-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1a7dd0b3ee80d90150e3495a3a13ac34dbcbfd4f012996a6a1d8900e91b5c0fb", size = 2179141, upload-time = "2026-05-06T13:37:14.046Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7b/79/7daa95be995be0eecc4cf75064cb33f9bbbfe3fe0158caf2f0d4a996a5c7/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:3fb702cd90b0446a3a1c5e470bfa0dd23c0233b676a9099ddcc964fa6ca13898", size = 2184325, upload-time = "2026-05-06T13:36:53.615Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9f/cb/d0a382f5c0de8a222dc61c65348e0ce831b1f68e0a018450d31c2cace3a5/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_armv7l.whl", hash = "sha256:b8458003118a712e66286df6a707db01c52c0f52f7db8e4a38f0da1d3b94fc4e", size = 2323990, upload-time = "2026-05-06T13:40:29.971Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/05/db/d9ba624cc4a5aced1598e88c04fdbd8310c8a69b9d38b9a3d39ce3a61ed7/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:372429a130e469c9cd698925ce5fc50940b7a1336b0d82038e63d5bbc4edc519", size = 2369978, upload-time = "2026-05-06T13:37:23.027Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f2/20/d15df15ba918c423461905802bfd2981c3af0bfa0e40d05e13edbfa48bc3/pydantic_core-2.46.4-cp314-cp314-win32.whl", hash = "sha256:85bb3611ff1802f3ee7fdd7dbff26b56f343fb432d57a4728fdd49b6ef35e2f4", size = 1966354, upload-time = "2026-05-06T13:38:03.499Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fc/b6/6b8de4c0a7d7ab3004c439c80c5c1e0a3e8d78bbae19379b01960383d9e5/pydantic_core-2.46.4-cp314-cp314-win_amd64.whl", hash = "sha256:811ff8e9c313ab425368bcbb36e5c4ebd7108c2bbf4e4089cfbb0b01eff63fac", size = 2072238, upload-time = "2026-05-06T13:39:40.807Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/32/36/51eb763beec1f4cf59b1db243a7dcc39cbb41230f050a09b9d69faaf0a48/pydantic_core-2.46.4-cp314-cp314-win_arm64.whl", hash = "sha256:bfec22eab3c8cc2ceec0248aec886624116dc079afa027ecc8ad4a7e62010f8a", size = 2018251, upload-time = "2026-05-06T13:37:26.72Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e8/91/855af51d625b23aa987116a19e231d2aaef9c4a415273ddc189b79a45fee/pydantic_core-2.46.4-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:af8244b2bef6aaad6d92cda81372de7f8c8d36c9f0c3ea36e827c60e7d9467a0", size = 2099593, upload-time = "2026-05-06T13:39:47.682Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fb/1b/8784a54c65edb5f49f0a14d6977cf1b209bba85a4c77445b255c2de58ab3/pydantic_core-2.46.4-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5a4330cdbc57162e4b3aa303f588ba752257694c9c9be3e7ebb11b4aca659b5d", size = 1935226, upload-time = "2026-05-06T13:40:40.428Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e8/e7/1955d28d1afc56dd4b3ad7cc0cf39df1b9852964cf16e5d13912756d6d6b/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:29c61fc04a3d840155ff08e475a04809278972fe6aef51e2720554e96367e34b", size = 1974605, upload-time = "2026-05-06T13:37:32.029Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/93/e2/3fedbf0ba7a22850e6e9fd78117f1c0f10f950182344d8a6c535d468fdd8/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:c50f2528cf200c5eed56faf3f4e22fcd5f38c157a8b78576e6ba3168ec35f000", size = 2030777, upload-time = "2026-05-06T13:38:55.239Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f8/61/46be275fcaaba0b4f5b9669dd852267ce1ff616592dccf7a7845588df091/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0cbe8b01f948de4286c74cdd6c667aceb38f5c1e26f0693b3983d9d74887c65e", size = 2236641, upload-time = "2026-05-06T13:37:08.096Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/60/db/12e93e46a8bac9988be3c016860f83293daea8c716c029c9ace279036f2f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:617d7e2ca7dcb8c5cf6bcb8c59b8832c94b36196bbf1cbd1bfb56ed341905edd", size = 2286404, upload-time = "2026-05-06T13:40:20.221Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e2/4a/4d8b19008f38d31c53b8219cfedc2e3d5de5fe99d90076b7e767de29274f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7027560ee92211647d0d34e3f7cd6f50da56399d26a9c8ad0da286d3869a53f3", size = 2109219, upload-time = "2026-05-06T13:38:12.153Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/88/70/3cbc40978fefb7bb09c6708d40d4ad1a5d70fd7213c3d17f971de868ec1f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:f99626688942fb746e545232e7726926f3be91b5975f8b55327665fafda991c7", size = 2110594, upload-time = "2026-05-06T13:40:02.971Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9d/20/b8d36736216e29491125531685b2f9e61aa5b4b2599893f8268551da3338/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:fc3e9034a63de20e15e8ade85358bc6efc614008cab72898b4b4952bea0509ff", size = 2159542, upload-time = "2026-05-06T13:39:27.506Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1d/a2/367df868eb584dacf6bf82a389272406d7178e301c4ac82545ab98bc2dd9/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:97e7cf2be5c77b7d1a9713a05605d49460d02c6078d38d8bef3cbe323c548424", size = 2168146, upload-time = "2026-05-06T13:38:31.93Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c1/b8/4460f77f7e201893f649a29ab355dddd3beee8a97bcb1a320db414f9a06e/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_armv7l.whl", hash = "sha256:3bf92c5d0e00fefaab325a4d27828fe6b6e2a21848686b5b60d2d9eeb09d76c6", size = 2306309, upload-time = "2026-05-06T13:37:44.717Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/64/c4/be2639293acd87dc8ddbcec41a73cee9b2ebf996fe6d892a1a74e88ad3f7/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:3ecbc122d18468d06ca279dc26a8c2e2d5acb10943bb35e36ae92096dc3b5565", size = 2369736, upload-time = "2026-05-06T13:37:05.645Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/30/a6/9f9f380dbb301f67023bf8f707aaa75daadf84f7152d95c410fd7e81d994/pydantic_core-2.46.4-cp314-cp314t-win32.whl", hash = "sha256:e846ae7835bf0703ae43f534ab79a867146dadd59dc9ca5c8b53d5c8f7c9ef02", size = 1955575, upload-time = "2026-05-06T13:38:51.116Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/40/1f/f1eb9eb350e795d1af8586289746f5c5677d16043040d63710e22abc43c9/pydantic_core-2.46.4-cp314-cp314t-win_amd64.whl", hash = "sha256:2108ba5c1c1eca18030634489dc544844144ee36357f2f9f780b93e7ddbb44b5", size = 2051624, upload-time = "2026-05-06T13:38:21.672Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f6/d2/42dd53d0a85c27606f316d3aa5d2869c4e8470a5ed6dec30e4a1abe19192/pydantic_core-2.46.4-cp314-cp314t-win_arm64.whl", hash = "sha256:4fcbe087dbc2068af7eda3aa87634eba216dbda64d1ae73c8684b621d33f6596", size = 2017325, upload-time = "2026-05-06T13:40:52.723Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pyee"
|
||||
version = "13.0.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/8b/04/e7c1fe4dc78a6fdbfd6c337b1c3732ff543b8a397683ab38378447baa331/pyee-13.0.1.tar.gz", hash = "sha256:0b931f7c14535667ed4c7e0d531716368715e860b988770fc7eb8578d1f67fc8", size = 31655, upload-time = "2026-02-14T21:12:28.044Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/a0/c4/b4d4827c93ef43c01f599ef31453ccc1c132b353284fc6c87d535c233129/pyee-13.0.1-py3-none-any.whl", hash = "sha256:af2f8fede4171ef667dfded53f96e2ed0d6e6bd7ee3bb46437f77e3b57689228", size = 15659, upload-time = "2026-02-14T21:12:26.263Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pygments"
|
||||
version = "2.20.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pypdf"
|
||||
version = "5.9.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/89/3a/584b97a228950ed85aec97c811c68473d9b8d149e6a8c155668287cf1a28/pypdf-5.9.0.tar.gz", hash = "sha256:30f67a614d558e495e1fbb157ba58c1de91ffc1718f5e0dfeb82a029233890a1", size = 5035118, upload-time = "2025-07-27T14:04:52.364Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/48/d9/6cff57c80a6963e7dd183bf09e9f21604a77716644b1e580e97b259f7612/pypdf-5.9.0-py3-none-any.whl", hash = "sha256:be10a4c54202f46d9daceaa8788be07aa8cd5ea8c25c529c50dd509206382c35", size = 313193, upload-time = "2025-07-27T14:04:50.53Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pytest"
|
||||
version = "8.4.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||
{ name = "iniconfig" },
|
||||
{ name = "packaging" },
|
||||
{ name = "pluggy" },
|
||||
{ name = "pygments" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/a3/5c/00a0e072241553e1a7496d638deababa67c5058571567b92a7eaa258397c/pytest-8.4.2.tar.gz", hash = "sha256:86c0d0b93306b961d58d62a4db4879f27fe25513d4b969df351abdddb3c30e01", size = 1519618, upload-time = "2025-09-04T14:34:22.711Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/a8/a4/20da314d277121d6534b3a980b29035dcd51e6744bd79075a6ce8fa4eb8d/pytest-8.4.2-py3-none-any.whl", hash = "sha256:872f880de3fc3a5bdc88a11b39c9710c3497a547cfa9320bc3c5e62fbf272e79", size = 365750, upload-time = "2025-09-04T14:34:20.226Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pyyaml"
|
||||
version = "6.0.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rich"
|
||||
version = "15.0.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "markdown-it-py" },
|
||||
{ name = "pygments" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz", hash = "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36", size = 230680, upload-time = "2026-04-12T08:24:00.75Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "shellingham"
|
||||
version = "1.5.4"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/58/15/8b3609fd3830ef7b27b655beb4b4e9c62313a4e8da8c676e142cc210d58e/shellingham-1.5.4.tar.gz", hash = "sha256:8dbca0739d487e5bd35ab3ca4b36e11c4078f3a234bfce294b0a0291363404de", size = 10310, upload-time = "2023-10-24T04:13:40.426Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl", hash = "sha256:7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686", size = 9755, upload-time = "2023-10-24T04:13:38.866Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "soupsieve"
|
||||
version = "2.8.4"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/47/2c/0a5f6f8ee0d5589e48c7640213ed5175d52cf540a06725b628cc1a45d6ce/soupsieve-2.8.4.tar.gz", hash = "sha256:e121fd02e975c695e4e9e8774a5ee35d74714b59307868dcc5319ad2d9e3328e", size = 121110, upload-time = "2026-05-24T13:55:57.154Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/5e/f5/0c41cb68dcae6b7de4fac4188a3a9589e21fb31df21ea3a2e888db95e6c9/soupsieve-2.8.4-py3-none-any.whl", hash = "sha256:e7e6b0769c8f51ed59acab6e994b00621096cfb1c640a7509295987388fbaf65", size = 37304, upload-time = "2026-05-24T13:55:55.406Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typer"
|
||||
version = "0.26.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "annotated-doc" },
|
||||
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||
{ name = "rich" },
|
||||
{ name = "shellingham" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/67/a5/756f2e6bc81a7dd79aa3c625dd01b74cabc4516628cace2caaec09ca6ff2/typer-0.26.2.tar.gz", hash = "sha256:9b4f19e08fcc9427a822d1ef467b1fe76737a2f65c7926bdeba2337d73569b68", size = 198991, upload-time = "2026-05-27T10:41:39.166Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/8b/a5/6ffd702beda8798b2b82ff70805ed4a66d963557e43a5d1823ab456251a4/typer-0.26.2-py3-none-any.whl", hash = "sha256:39beff72ffbb31978a5b545f677d57edb97c6f980f433b38556deb0af25f094d", size = 123123, upload-time = "2026-05-27T10:41:40.504Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typing-extensions"
|
||||
version = "4.15.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typing-inspection"
|
||||
version = "0.4.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/55/e3/70399cb7dd41c10ac53367ae42139cf4b1ca5f36bb3dc6c9d33acdb43655/typing_inspection-0.4.2.tar.gz", hash = "sha256:ba561c48a67c5958007083d386c3295464928b01faa735ab8547c5692e87f464", size = 75949, upload-time = "2025-10-01T02:14:41.687Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/dc/9b/47798a6c91d8bdb567fe2698fe81e0c6b7cb7ef4d13da4114b41d239f65d/typing_inspection-0.4.2-py3-none-any.whl", hash = "sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7", size = 14611, upload-time = "2025-10-01T02:14:40.154Z" },
|
||||
]
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user