Date : 13/01/2026 22:19 Duree : ~4h Statut : TERMINE
Transformation complete de NeedFinder d'un simple collecteur de besoins IT en plateforme de detection d'opportunites croisant besoins utilisateurs et decouvertes scientifiques.
┌─────────────────────────────────────────────────────────────────────────┐
│ NEEDFINDER v2.1 │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ COLLECTE BESOINS COLLECTE DISCOVERIES │
│ ───────────────── ──────────────────── │
│ Reddit (27 subreddits) GitHub Trending (OSS Insight API) │
│ - tech (8) ArXiv (26 categories sciences) │
│ - health (5) HuggingFace (12 types modeles) │
│ - finance (4) │
│ - productivity (3) │
│ - education (2) CATEGORIES SCIENCES │
│ - family (2) ─────────────────── │
│ - work (3) - AI/ML, NLP, Computer Vision │
│ - Quantum Physics │
│ HackerNews - Neuroscience │
│ - Biomedicine, Genomics │
│ │ - Robotics │
│ ▼ - Materials Science │
│ ┌─────────────┐ - Economics/Finance │
│ │ ANALYSE │ - Energy, Environment │
│ │ Ollama AI │ │ │
│ │ (mistral) │ │ │
│ └─────────────┘ │ │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ SCIENCE BRIDGE MATCHER │ │
│ │ - Mapping science → applications → domaines utilisateurs │ │
│ │ - Pre-filtrage semantique par mots-cles │ │
│ │ - Scoring IA avec prompt creatif │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────┐ │
│ │ OPPORTUNITES │ │
│ │ (760 matches) │ │
│ └───────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘github-trending.ts)api.ossinsight.io/v1/trends/repos)arxiv.ts)huggingface.ts)matcher.ts)Concept : Chaque domaine scientifique est mappe vers :
Mapping exemple :
'neuroscience': {
applications: ['mental health', 'sleep optimization', 'learning enhancement', 'focus improvement'],
userDomains: ['health', 'productivity', 'education', 'work'],
keywords: ['brain', 'sleep', 'focus', 'anxiety', 'stress', 'learn', 'memory', 'habit', 'mental', 'adhd']
},
'quantum': {
applications: ['optimization', 'cryptography', 'simulation', 'machine learning', 'drug discovery'],
userDomains: ['tech', 'finance', 'health', 'productivity'],
keywords: ['optimize', 'secure', 'encrypt', 'compute', 'simulate', 'portfolio', 'schedule', 'route']
}Queries separees pour garantir inclusion des sciences fondamentales (pas noyes par AI/ML).
needfinder-v2.html)4 onglets :
Features :
-- Sources de besoins
sources (id, name, type, last_collected_at)
-- Posts bruts
raw_posts (id, source_id, external_id, title, content, domain, ...)
-- Besoins detectes (apres analyse Ollama)
detected_needs (id, post_id, need_statement, need_category, need_type,
opportunity_score, willingness_to_pay, domain, ...)
-- Sources de discoveries
discovery_sources (id, name, url, last_collected_at)
-- Discoveries
discoveries (id, source_id, external_id, external_url, title, description,
category, tags, tech_stack, stars, forks, downloads,
impact_score, novelty_score, accessibility_score, ...)
-- Matches besoin <-> discovery
need_discovery_matches (id, need_id, discovery_id, relevance_score,
match_reason, is_opportunity, ...)-- Matrice opportunites
CREATE VIEW opportunity_matrix AS ...
-- Discoveries recentes
CREATE VIEW recent_discoveries AS ...
-- Insights opportunites
CREATE VIEW opportunity_insights AS ...| Endpoint | Methode | Description |
|---|---|---|
| /health | GET | Health check |
| /api/stats | GET | Stats globales |
| /api/needs | GET | Liste besoins (filtres: domain, min_persistence, limit, offset) |
| /api/discoveries | GET | Liste discoveries (filtres: category, source, min_impact) |
| /api/discoveries/recent | GET | Discoveries recentes |
| /api/discoveries/categories | GET | Stats par categorie |
| /api/opportunities | GET | Opportunites triees |
| /api/matches | GET | Matches bruts |
| /api/collect | POST | Declenche collecte besoins |
| /api/collect/discoveries | POST | Declenche collecte discoveries (toutes) |
| /api/collect/github | POST | Collecte GitHub Trending |
| /api/collect/arxiv | POST | Collecte ArXiv |
| /api/collect/huggingface | POST | Collecte HuggingFace |
| /api/analyze | POST | Analyse Ollama (limit en param) |
| /api/match | POST | Matching (limit en param) |
| /api/match/all | POST | Full matching |
| /api/logs | GET | Logs de collecte |
NEEDFINDER_PORT=5200
NEEDFINDER_DB_URL=postgresql://postgres:***@db:5432/postgres
OLLAMA_URL=http://192.168.1.30:11434
OLLAMA_MODEL=mistral:latest/app/
├── src/
│ ├── index.ts # API principale
│ ├── db/
│ │ └── client.ts # Connexion PostgreSQL
│ ├── collectors/
│ │ ├── reddit.ts # 27 subreddits, 7 domaines
│ │ ├── hackernews.ts
│ │ ├── github-trending.ts # OSS Insight API
│ │ ├── arxiv.ts # 26 categories sciences
│ │ └── huggingface.ts # 12 types modeles
│ ├── services/
│ │ ├── analyzer.ts # Analyse Ollama
│ │ └── matcher.ts # Science Bridge Matcher
│ └── cron/
│ └── scheduler.ts # Cron interne 6h
├── package.json
└── Dockerfile| Metrique | Valeur |
|---|---|
| Posts collectes | 386 |
| Besoins detectes | 359 |
| Discoveries totales | 1334 |
| - GitHub Trending | 178 |
| - ArXiv | 826 |
| - HuggingFace | 330 |
| Matches/Opportunites | 760 |
| Categorie | Matches | Score moyen |
|---|---|---|
| Neuroscience | 106 | 73 |
| Quantum | 90 | 66 |
| Robotics | 84 | 73 |
| Mathematics | 66 | 70 |
| Physics/Engineering | 64 | 70 |
| Statistics | 59 | 72 |
| AI/ML | 58 | 75 |
| Economics/Finance | 51 | 76 |
| NLP | 51 | 78 |
| Genomics | 44 | 70 |
| Biomedicine | 43 | 73 |
# Stats
curl http://192.168.1.51:5200/api/stats | jq
# Lancer collecte complete
curl -X POST http://192.168.1.51:5200/api/collect/discoveries
# Lancer matching
curl -X POST "http://192.168.1.51:5200/api/match?limit=50"
# Voir logs container
ssh gouroubleu@192.168.1.51 "docker logs --tail 50 needfinder-collector-dev"
# Restart container
ssh gouroubleu@192.168.1.51 "docker restart needfinder-collector-dev"
# GPU status (Ollama)
ssh gouro@192.168.1.30 "nvidia-smi"