Laya fine-tuned for climate-vulnerability group detection, v2 (multi-label)
convaiinnovations/laya (421M, ModernBERT-large backbone) fine-tuned with the official RLCD recipe on GIZ/vulnerability_training_data_full (380 train rows x 17 binary vulnerability-group questions; 36 all-negative rows included as negatives).
v2 change vs v1: the typed-question instructions now carry the label into context — each binary question reads "Does this text indicate that are targeted, supported, or affected? This group is specifically vulnerable to climate change. Answer true or false."
Each vulnerability group is asked as one binary (noul) typed question; all 17 are answered in a single forward pass.
Evaluate with laya.load("peter2000/laya-vulnerability-groups-v2") and agent.predict(state, questions).
Test-set metrics (held-out 95 rows, threshold 0.5)
| metric | value |
|---|---|
| macro-F1 | 0.7038 |
| micro-F1 | 0.7083 |
| ECE | 0.0242 |
| subset accuracy | 0.4737 |
Per-label F1:
| label | F1 |
|---|---|
| Agricultural communities | 0.9167 |
| Coastal communities | 0.5714 |
| Ethnic, racial or other minorities | 0.6000 |
| Fishery communities | 0.4000 |
| Informal sector workers | 1.0000 |
| Members of indigenous and local communities | 0.9333 |
| Migrants and displaced persons | 0.6667 |
| Older persons | 0.7778 |
| Other | 0.0000 |
| Persons living in poverty | 0.6087 |
| Persons with disabilities | 1.0000 |
| Persons with pre-existing health conditions | 1.0000 |
| Residents of drought-prone regions | 0.8000 |
| Rural populations | 0.8889 |
| Sexual minorities (LGBTQI+) | 0.3333 |
| Urban populations | 0.5455 |
| Women and other genders | 0.9231 |