logo
Sign In
function

Granite Thinking Filter

Supports Granite 3.2, enabling thinking, and collapsing thought.

v0.1
granite_thinking_filter.py
Python · 90 lines
18    class Valves(BaseModel):19        pass  # Define any configurable options for your filter here20 21    def __init__(self):22        self.valves = self.Valves()23 24    def inlet(self, body: dict, __user__: Optional[dict] = None) -> dict:25        # Define the control message26        control_message = {"role": "control", "content": "thinking"}27 28        # Insert the control message at position -229        if "messages" in body and len(body["messages"]) >= 1:

Supports Granite 3.2 & 3.3 thinking models with a collapsable section for thought. This inserts the Granite3.2-required {"role": "control", "content": "thinking"}, and provides an output filter to collapse the thinking section when complete.

For Granite 3.3, it may attempt to collapse non-thinking blocks. Delete the outlet function for Granite 3.3.

0
0

Supports Granite 3.2 & 3.3 thinking models with a collapsable section for thought. This inserts the Granite3.2-required {"role": "control", "content": "thinking"}, and provides an output filter to collapse the thinking section when complete.

For Granite 3.3, it may attempt to collapse non-thinking blocks. Delete the outlet function for Granite 3.3.

0
0