Here are 310 public repositories matching this topic...

emcie-co / parlant

The conversational control layer for customer-facing AI agents - Parlant is a context-engineering framework optimized for controlling customer interactions.

python gemini openai customer-service hacktoberfest customer-success ai-agents ai-alignment llm genai llama3

Updated Mar 16, 2026
Python

agencyenterprise / PromptInject

PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Safety Workshop 2022

machine-learning agi language-models ai-safety adversarial-attacks ai-alignment ml-safety gpt-3 large-language-models prompt-engineering chain-of-thought agi-alignment

Updated Feb 26, 2024
Python

MinghuiChen43 / awesome-trustworthy-deep-learning

A curated list of trustworthy deep learning papers. Daily updating...

Updated Mar 13, 2026

Giskard-AI / awesome-ai-safety

Sponsor

📚 A curated list of papers & technical articles on AI Quality & Safety

Updated Apr 14, 2025

tomekkorbak / pretraining-with-human-feedback

Code accompanying the paper Pretraining Language Models with Human Preferences

reinforcement-learning gpt language-models ai-safety ai-alignment pretraining decision-transformers rlhf

Updated Feb 13, 2024
Python

lets-make-safe-ai / make-safe-ai

How to Make Safe AI? Let's Discuss! 💡|💬|🙌|📚

ai agi artificial-intelligence artificial-general-intelligence ai-safety ai-alignment

Updated Mar 29, 2023

tsinghua-fib-lab / AAAI2025_MIA-Tuner

[AAAI'25 Oral] "MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector".

ai-alignment membership-inference-attack large-language-models pretraining-data-detection

Updated Mar 17, 2025
Python

EzgiKorkmaz / adversarial-reinforcement-learning

Reading list for adversarial perspective and robustness in deep reinforcement learning.

Updated Mar 2, 2026

AthenaCore / AwesomeResponsibleAI

A curated list of awesome academic research, books, code of ethics, courses, databases, data sets, frameworks, institutes, maturity models, newsletters, principles, podcasts, regulations, reports, responsible scale policies, tools and standards related to Responsible, Trustworthy, and Human-Centered AI.

ai awesome-list ai-safety interpretable-ai explainable-ai xai ai-alignment fairness-ai responsible-ai human-centered-ai ethical-ai trustworthy-ai ai-regulation ai-governance ai-standards

Updated Mar 4, 2026

Xayan / Rules.txt

A rationalist ruleset for "debugging" LLMs, auditing their internal reasoning and uncovering biases; also a jailbreak.

Updated Nov 1, 2025

dit7ya / awesome-ai-alignment

A curated list of awesome resources for Artificial Intelligence Alignment research

awesome awesome-list ai-safety ai-alignment

Updated Jul 14, 2023

wesg52 / sparse-probing-paper

Sparse probing paper full code.

ai-safety interpretability ai-alignment mechanistic-interpretability

Updated Dec 17, 2023
Jupyter Notebook

RLHFlow / Directional-Preference-Alignment

Directional Preference Alignment

ai-alignment large-language-models rlhf

Updated Sep 23, 2024

lzzcd001 / nabla-gfn

Official Implementation of Nabla-GFlowNet (ICLR 2025)

generative-model finetuning ai-alignment diffusion-models gflownet

Updated May 3, 2025
Python

riceissa / aiwatch

Website to track people, organizations, and products (tools, websites, etc.) in AI safety

mysql php database dataset ai-safety data-portal aisafety ai-alignment

Updated Mar 17, 2026
HTML

knoveleng / redeval

[ICLR 2026] - Official repo for the paper: "RedBench: A Universal Dataset for Comprehensive Red Teaming of Large Language Models"

evaluation ai-safety red-teaming ai-alignment

Updated Mar 2, 2026
Python

pablo-chacon / Spoon-Bending

Educational analysis of LLM alignment, safety behavior, and framing-sensitive response patterns.

research model-alignment interpretability educational-project ai-alignment gpt4 large-language-models llm prompt-engineering reasoning-patterns gpt5 safety-research ai-bias context-engineering

Updated Nov 4, 2025

UCSC-VLAA / Sight-Beyond-Text

[TMLR 2024] Official implementation of "Sight Beyond Text: Multi-Modal Training Enhances LLMs in Truthfulness and Ethics"

alignment vlm ai-alignment vision-language vicuna llm mllm llava llama2

Updated Sep 15, 2023
Python

liondw / Signal-Alignment

An initiative to create concise and widely shareable educational resources, infographics, and animated explainers on the latest contributions to the community AI alignment effort. Boosting the signal and moving the community towards finding and building solutions.

education design ai ai-alignment

Updated Jul 9, 2023

RamyaLab / pluralistic-alignment

The open-source repository for PAL: Sample-Efficient Personalized Reward Modeling for Pluralistic Alignment.

ai-alignment rlhf pluralistic-alignment

Updated Aug 28, 2025
Python