Research

Trustworthy AI in healthcare, NLP for the Sustainable Development Goals, and — now — how to evaluate agents.

01Questions
Updated Sep 2026. Revised twice a year.
  1. How do you attribute an agent's failure to the step that caused it, rather than grading only the final state?
  2. What does a simulated world need to contain for an eval result to transfer to production behavior?
  3. What makes a scenario cheap enough to write that engineers write one when they find a bug, and expensive enough to be right that the suite can be trusted?
  4. What has to be true of a preference signal collected from real usage for training on it to improve the product rather than the metric?
  5. What is the right unit of an eval scenario for systems that both build automations and repair them when the world changes?
  6. Which properties of a benchmark predict that improvements on it will show up for real users?
02Papers
Peer-reviewed journals, chapters & conferences.
  • 2023
    Sailing the Data Sea to Advance Research on the Sustainable Development Goals
    Spezzatti A, Kheradmand E, Gupta K, Peras M, Zaminpeyma R
    Argues that progress on the SDGs is bottlenecked by data discoverability, and sets out the design and ethics of a global catalog to fix it.
    The Ethics of Artificial Intelligence for the Sustainable Development Goals, Springer · DOI
  • 2022
    To explain or not to explain? — Artificial intelligence explainability in clinical decision support systems
    Amann J, Vetter D, Blomberg SN, Christensen HC, Coffee M, Gerke S, … Spezzatti A, … (Z-Inspection initiative)
    Whether a clinical AI should explain itself depends on validation level, deployment context and who is deciding — explainability is not always the right answer.
    PLOS Digital Health · PDF · DOI
  • 2022
    Assessing trustworthy AI in times of COVID-19: deep learning for predicting a multiregional score conveying the degree of lung compromise
    Allahabadi H, Amann J, Balot I, Beretta A, Binkley C, Bozenhard J, … Spezzatti A, … Zicari RV
    A post-hoc trustworthiness assessment of a deployed chest-X-ray severity model in Brescia, applying the EU HLEG framework under pandemic pressure.
    IEEE Transactions on Technology and Society · DOI
  • 2022
    Note: Leveraging artificial intelligence to build a data catalog and support research on the sustainable development goals
    Spezzatti A, Kheradmand E, Gupta K, Peras M, Zaminpeyma R
    Presents the SDG Data Catalog system: NLP over open-access literature to build a global database of datasets, metadata and research networks.
    ACM SIGCAS/SIGCHI COMPASS · DOI
  • 2021
    Co-design of a trustworthy AI system in healthcare: deep learning based skin lesion classifier
    Zicari RV, Ahmed S, Amann J, Braun SA, Brodersen J, … Spezzatti A, … Wurth R
    Shows how an ethically-aligned co-design process shapes a skin-lesion classifier from the earliest design phase, with a multidisciplinary team.
    Frontiers in Human Dynamics · PDF · DOI
  • 2021
    On assessing trustworthy AI in healthcare: Machine learning as a supportive tool to recognize cardiac arrest in emergency calls
    Zicari RV, Brusseau J, Blomberg SN, Christensen HC, Coffee M, … Spezzatti A, … Amann J, et al.
    Applies the EU trustworthy-AI guidelines to a live emergency-call cardiac-arrest detector in Copenhagen and surfaces the trade-offs an interdisciplinary team actually faces.
    Frontiers in Human Dynamics · PDF · DOI
  • 2020
    Hidden in Plain Sight: Building a Global Sustainable Development Data Catalogue
    Hodson J, Spezzatti A
    Describes mining millions of open-access papers to build an open, extensible catalogue of SDG-relevant datasets.
    ICT Analysis and Applications, Springer · DOI
03Projects
Open-source & research initiatives.
  • 2018 — 2021
    SDG Data Catalog
    Led the UN SDG Data Catalog at the AI for Good Foundation: an NLP extraction pipeline (BiLSTM-CRF, fine-tuned BERT) over millions of academic papers that maps datasets, research, and organizations across the Sustainable Development Goals, reaching over 92% recall on dataset discovery. Secured $300K+ in Microsoft funding and was presented at SIGKDD 2019.
  • 2020 — 2023
    Z-Inspection
    Co-developed a process, grounded in applied ethics and the EU High-Level Expert Group definition of trustworthy AI, for assessing whether an AI system can be trusted in practice. Applied it to four production healthcare AI systems with clinicians, ethicists, and legal scholars; the assessments became four peer-reviewed publications.
  • 2024
    MicroBio LLM
    Fine-tuned Mistral and LLaMA on k-mer DNA sequences for metagenomic classification, reaching 82% family-level taxonomic accuracy by treating genomic sequences as a language.
  • 2019
    Maia — Music Generation
    Trained LSTM and Transformer architectures from scratch on tokenized MIDI corpora at UC Berkeley, with sparse attention for compute efficiency. The accompanying write-up became one of the most-read Towards Data Science pieces on music generation.
  • 2020 — 2021
    DE&I Tracker
    Monthly public tracking of workforce diversity across 1.3M+ US organizations, built with the AI4Good Foundation.
  • 2019
    SciKid-Learn
    An interactive tool that adapts student learning to cognitive preferences and learning styles. Presented at SIGKDD 2019.
04Coverage
Selected press & institutional notes.
  1. SDG Data Catalog funded by Microsoft
    AI4Good / Microsoft
    Apr 2021
  2. MEng alumnus Andy Spezzatti speaks at SIGKDD 2019
    Berkeley Fung Institute
    Jun 2019