Expected Behavior
We would like to have Guardrailing support for Input, Dailog, Retrieval, Execution and Output of the LLM application
Current Behavior
We are not having railing support for LLM applications.
Context
We have been building LLM powered applications that are not gaurdrailed that is concerning me a lot.
Comment From: csterwa
@radhakrishna67 could you provide more specifics on what Guardrails capabilities that you would expect in this case. Do you have any examples that would provide more details?
Comment From: rafal-dudek
I think Quarkus starter for Langchain4j has a nice concept of Guardrails: https://docs.quarkiverse.io/quarkus-langchain4j/dev/guardrails.html
They are generic, easy to use and enable to verify/modify/retry Input and Output of LLMs.
Comment From: ThomasVitale
It should probably be included in the documentation, but the Advisor API is how you can specify guardrails in a ChatClient workflow. Advisors intercept an LLM request before and after its execution, giving you the hooks to define input and output guardrails. Here's some examples of an input guardrail and an output guardrail.
It would be nice to get some specialisation of the Advisor API tailored towards guardrails, providing convenient utils for doing things like deny LLM requests, retry, or re-prompt. You can do all that today with the Advisor API, but less conveniently.