T1: Deduplicator compile fix + unit tests [checkpoint]

2026-07-07 22:20:08 +02:00
commit 38a93d0c4d
114 changed files with 6942 additions and 0 deletions
@@ -0,0 +1,84 @@
+# Hospital Leadership Graph Database
+
+This repository contains the implementation for building a Neo4j graph database of Polish hospitals and their leadership structures.
+
+## Overview
+
+The project creates a graph database that links:
+- **Hospitals** in Poland → **Directors** (`[:DYREKTOR]`)
+- **Hospitals** → **Supervisory Boards** (`[:RADNIA]`)
+- **Supervisory Board members** → **Political parties** (`[:CZŁONEK_PARTII]`) 
+- **Supervisory Board members** → **Regional councilors** (`[:RADNY]`) when the same person holds both roles
+
+## Implementation
+
+The implementation consists of:
+
+1. **build_hospital_graph.py** - Main Python script that:
+   - Sets up Neo4j connection and schema
+   - Processes hospital data (using sample data as real API is not accessible)
+   - Processes councilor data (using sample data)
+   - Creates graph nodes and relationships
+   - Generates summary reports
+
+2. **PLAN.md** - Detailed execution plan outlining the steps and methodology
+
+## Usage
+
+### Prerequisites
+- Docker (to run Neo4j)
+- Python 3.6+
+- Required packages: requests, beautifulsoup4, neo4j, pandas, python-levenshtein
+
+### Setup
+```bash
+# Run Neo4j container
+docker run -d --name neo4j-test -p 7687:7687 -p 7474:7474 -e NEO4J_AUTH=neo4j/password neo4j:latest
+
+# Create virtual environment and install dependencies
+python3 -m venv hospital_graph_env
+source hospital_graph_env/bin/activate
+pip install requests beautifulsoup4 neo4j pandas python-levenshtein
+
+# Run the script
+python build_hospital_graph.py
+```
+
+## Data Sources
+
+1. **Hospital List**: 
+   - Primary: NFZ registry or Ministry of Health (currently using sample data due to API unavailability)
+   - Fallback: Wikipedia's "Lista szpitali w Polsce"
+
+2. **Hospital Staff**:
+   - Official hospital websites (often under "Zarząd" / "Rada Nadzorcza")
+
+3. **Regional Councilors**:
+   - Biuletyn Informacji Publicznej (BIP) for each voivodeship
+
+4. **Political Party Affiliation**:
+   - Same BIP pages or party registries
+
+## Graph Schema
+
+- **Hospital** node with properties: `name`, `city`, `website`, `nip`
+- **Person** node with properties: `name`, `party`, `sourceUrls` (array)
+- **Council** node with property: `region`
+- Relationships:
+  - `[:DYREKTOR]` from Hospital to Person
+  - `[:RADNIA]` from Hospital to Person
+  - `[:RADNY]` from Person to Council (with `region` property)
+  - `[:CZŁONEK_PARTII]` from Person to Party (if applicable)
+
+## Files Generated
+
+1. **scrape_log.json** - Log of all scraping attempts
+2. **overlap_report.csv** - Summary report of persons in multiple roles
+3. **PLAN.md** - Detailed execution plan
+
+## Validation
+
+The script includes validation to ensure:
+- Every hospital has at least one director relationship
+- All person nodes have their source URLs properly recorded
+- Graph schema constraints are enforced