Extending PyGPT

Overview

PyGPT exposes integration points through pygpt_net.app.run(). A custom launcher can register additional:

• plugins,

• GUI tools,

• LLM wrappers,

• vector store providers,

• data loaders,

• audio input providers,

• audio output providers,

• web search providers,

• agent providers.

The repository’s examples directory contains tutorial implementations for every add-on type:

• examples/custom_launcher.py

• examples/addons/plugins/example_plugin

• examples/addons/tools/example_tool

• examples/addons/agents/example_agent

• examples/addons/llms/example_llm

• examples/addons/vector_stores/example_vector_store

• examples/addons/loaders/example_loader

• examples/addons/audio_input/example_audio_input

• examples/addons/audio_output/example_audio_output

• examples/addons/web/example_web

The examples are intentionally small and are the recommended starting point for custom integrations.

For the complete external package format, lifecycle, Package Manager integration, publishing workflow and method/event reference for every Add-on type, see Add-ons API.

External Add-ons

PyGPT can load application-wide external add-ons directly from <application base workdir>/addons. This is the recommended way to distribute Python add-ons that must work with both source installations and compiled PyInstaller builds, because the add-on code remains outside the application bundle and is imported by the embedded Python runtime at startup.

Supported add-on types are:

• plugin -> addons/plugins

• llm -> addons/llms

• vector_store -> addons/vector_stores

• loader -> addons/loaders

• audio_input -> addons/audio_input

• audio_output -> addons/audio_output

• web -> addons/web

• tool -> addons/tools

• agent -> addons/agents

• theme -> addons/themes

• locale -> addons/locale

The Python add-on types are registered into the same runtime registries as built-in components. Theme and locale packages are static application-wide resources: the theme/translation loaders read them directly from addons/themes and addons/locale without copying files into a profile.

Installing ready-made Add-ons

Open Config -> Install Add-on.... The Installed tab shows add-on name, description, author, version, type, trusted status and official status. Right-click an installed item and choose Uninstall to remove it after confirmation.

The installer supports:

• Import from ZIP - the archive must contain exactly one add-on package; normal archive/repository wrapper directories are allowed as long as exactly one manifest.json is present.

• Import from directory - select a local directory whose root contains manifest.json.

• Import from GitHub - paste a GitHub repository URL, a /tree/<ref>/<path> URL, or a direct repository-subdirectory URL such as https://github.com/owner/repo/plugins/example_plugin.

• Explore - browse the configured public add-ons registry and install selected entries.

The default registry is:

https://raw.githubusercontent.com/szczyglis-dev/py-gpt-addons/master/addons.json

You can change the registry URL in the Explore tab. A normal GitHub repository URL in the short github.com/<owner>/<repo>/<file>.json form is accepted by PyGPT and resolved to raw content internally.

Warning

External Python add-ons execute with the same process permissions as PyGPT. They can import Python modules, access files available to the process, use the network, and call any APIs exposed by installed dependencies. The trusted and official flags are not a sandbox or a code-safety guarantee. Public-registry entries are content-pinned with SHA-256, and trusted entries must pass registry/manifest/content verification before installation, but this only proves that the downloaded tree is the reviewed tree. Review source code and repository ownership before installing an add-on.

Python add-ons are loaded on application startup. After installing, updating or uninstalling one, restart PyGPT before relying on the runtime change. The installed Add-on set is shared by all profiles, so switching profiles does not select another Python Add-on tree. Theme and locale packages are global too and are read in place from the same application-wide Add-ons tree.

An invalid add-on never aborts the whole application startup. Missing manifests, unsupported manifest versions, incompatible minimum PyGPT versions, broken imports, invalid entry points and registration errors are skipped individually and reported as [Add-ons] WARNING messages in the console/log output.

Add-on directory layout

A plugin installed manually can look like this:

<application base workdir>/addons/
└── plugins/
    └── my_plugin/
        ├── manifest.json
        ├── plugin.py
        └── helpers/
            └── api.py

The directory name must match the manifest id. The parent directory must match the manifest type. For example, an add-on with "type": "audio_input" belongs in addons/audio_input/<id>.

Every package requires a manifest.json. Packages without a valid manifest are ignored.

Manifest format

Manifest version 1 uses the following base structure:

{
  "manifest_version": 1,
  "id": "my_plugin",
  "type": "plugin",
  "name": "My Plugin",
  "description": "Example external PyGPT plugin.",
  "version": "1.2.0",
  "sha256": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
  "min_app_version": "2.8.32",
  "author": "Example Author",
  "contact": {
    "email": "author@example.com",
    "repo": "https://github.com/example/my-pygpt-add-on",
    "www": "https://example.com"
  },
  "entrypoint": "plugin.py:Plugin",
  "external_dependencies": [
    {"name": "example-package", "version": ">=1.0,<2.0"}
  ]
}

Required fields are manifest_version, id, type, name, description, version, min_app_version, author and contact. Python/runtime add-on types additionally require entrypoint. sha256 is optional for local/manual packages but required when the Add-on is published through the public py-gpt-addons registry; a trusted registry entry also requires the upstream manifest to contain the same digest.

id must contain only lowercase letters, digits, ., _ and - and must start with a letter or digit. version and min_app_version use normal PEP 440-compatible version strings.

Add-on IDs should be globally unique. For add-ons intended for distribution, use a GitHub-derived naming convention such as <github_user>_<repo_name>_<addon_name> and normalize it to the characters allowed above. For example: szczyglis_dev_py_gpt_example_plugin. Keep the ID stable across releases so updates replace the existing installation instead of creating a second add-on.

contact may be a string, object or list. Using an object is recommended so an email address, repository and website can be declared separately.

external_dependencies declares application-runtime Python dependencies. Version 1 accepts either normal requirement strings or objects with at least name and optional version / optional fields. Required missing dependencies are resolved through the shared Config -> Package Manager flow and installed into the application-wide, Python-version-specific extra_packages/<major.minor> directory. Optional dependencies are not installed automatically. Direct package URLs are rejected; use package names and version constraints.

For public releases, generate the content pin with bin/addon-sha256.sh <addon-dir> --write on Linux/macOS or bin\addon-sha256.bat <addon-dir> --write on Windows, commit the resulting manifest, create an immutable release tag such as v1.0.0 for that exact commit, and place both the same digest and that tag in the public registry entry. The digest covers the complete Add-on tree and a canonicalized manifest.json with only its own sha256 field omitted. Any later file change requires a new release tag, a new digest and a new registry PR. PyGPT verifies a pinned package before dependency installation and refuses a mismatching trusted package.

Entrypoints

An entry point uses relative/module.py:Symbol syntax. The symbol may be:

• a class - PyGPT instantiates it without arguments;

• a zero-argument factory function - PyGPT calls it;

• a factory returning a list/tuple of compatible objects.

A manifest can also use an array of entry points when one add-on package intentionally registers multiple components of the same add-on type:

{
  "entrypoint": [
    "providers.py:PrimaryProvider",
    "providers.py:SecondaryProvider"
  ]
}

The resulting object must inherit the same base class required by the built-in launcher registration method. The mapping is:

plugin       -> pygpt_net.plugin.base.plugin.BasePlugin
llm          -> pygpt_net.provider.llms.base.BaseLLM
vector_store -> pygpt_net.provider.vector_stores.base.BaseStore
loader       -> pygpt_net.provider.loaders.base.BaseLoader
audio_input  -> pygpt_net.provider.audio_input.base.BaseProvider
audio_output -> pygpt_net.provider.audio_output.base.BaseProvider
web          -> pygpt_net.provider.web.base.BaseProvider
tool         -> pygpt_net.tools.base.BaseTool
agent        -> pygpt_net.provider.agents.base.BaseAgent

The add-on loader creates an isolated Python package namespace for each installed add-on. Relative imports inside an add-on package are therefore supported without adding the whole addons directory to global sys.path.

Creating an external plugin

For a minimal plugin:

from pygpt_net.plugin.base.plugin import BasePlugin


class Plugin(BasePlugin):
    def __init__(self):
        super().__init__()
        self.id = "my_external_plugin"
        self.name = "My External Plugin"
        self.description = "Loaded from the application-wide addons directory."
        self.type = ["cmd"]

Save it as plugin.py next to the manifest and set "entrypoint": "plugin.py:Plugin". The normal plugin API, options, commands and event hooks are identical to those used by built-in plugins.

Creating other provider types

For other add-on families, subclass the same base class used by the corresponding built-in provider and implement the methods required by that interface. External objects are passed to the existing Launcher.add_llm(), add_vector_store(), add_loader(), add_audio_input(), add_audio_output(), add_web(), add_tool() or add_agent() path, so registration behavior is the same as for custom instances supplied to pygpt_net.app.run().

The repository contains ready-to-copy examples under examples for every supported type, including a theme and locale package. These examples are deliberately minimal and are intended to verify discovery/registration before implementing a real backend.

Theme Add-ons

A theme package uses "type": "theme" and does not require an entry point. Put theme files either directly in the add-on root or in a theme subdirectory:

addons/themes/my-theme-dark/
├── manifest.json
└── theme/
    ├── app.css
    ├── app.xml
    └── chat.css

The add-on ID becomes the theme directory ID under %workdir%/css. The normal -dark / -light theme naming and compatibility rules described later in this document still apply.

Locale Add-ons

A locale package uses "type": "locale" and does not require an entry point. Put one or more normal PyGPT locale .ini files in a locale subdirectory:

addons/locale/my-locale-pack/
├── manifest.json
└── locale/
    ├── locale.en.ini
    └── locale.pl.ini

They are read directly from <application base workdir>/addons/locale/<id> and extend the normal application locale domain. No locale files are copied into %workdir%/locale.

GitHub repositories and monorepos

A standalone add-on repository should keep manifest.json in its repository root. A repository containing multiple add-ons may keep them in subdirectories such as:

plugins/example_plugin/manifest.json
tools/example_tool/manifest.json
themes/example-theme-dark/manifest.json

Users can paste a normal GitHub tree URL that includes the subdirectory, for example https://github.com/owner/repo/tree/v1.0.0/plugins/example_plugin. PyGPT also accepts a direct repository-subdirectory URL such as https://github.com/owner/repo/plugins/example_plugin and resolves that path against the repository default branch. Registry entries can specify the repository, package path and source revision separately with github_url, github_path and ref. For add-ons stored directly in the official py-gpt-addons repository, github_url may be omitted and only github_path/path (for example ./plugins/example_plugin) is required.

Publishing in the official registry

The official registry repository is https://github.com/szczyglis-dev/py-gpt-addons. To publish an add-on, host the add-on source on GitHub, verify that its root contains a valid manifest.json, and submit a pull request adding an item to addons.json.

A registry entry can look like this:

{
  "id": "example_plugin",
  "name": "Example Plugin",
  "description": "Short catalog description.",
  "author": "Example Author",
  "version": "1.0.0",
  "type": "plugin",
  "github_url": "https://github.com/example/pygpt-add-ons",
  "github_path": "plugins/example_plugin",
  "ref": "v1.0.0",
  "trusted": false,
  "official": false
}

github_path is optional when github_url points directly at a standalone add-on repository. Conversely, when an entry points to a directory inside the official py-gpt-addons repository, github_url can be omitted and github_path or path can contain a relative path such as ./plugins/example_plugin. For manual/local GitHub imports ref may be omitted and PyGPT then resolves the repository’s default branch, but public registry submissions must use an immutable release ref, preferably a version tag such as v1.0.0 (an exact commit SHA is also acceptable). Do not publish a registry entry pointing to main/master or another moving branch. trusted and official are registry-maintainer metadata and should not be self-declared as a security guarantee by third-party authors.

Create and push the release tag before opening the registry PR and never move, overwrite or delete a tag referenced by an accepted registry entry. This keeps the currently approved version downloadable while development continues on the default branch and while a later release is waiting for registry review. A versioned GitHub subdirectory URL therefore looks like https://github.com/user/repo/tree/v1.0.0/plugin.

The top-level registry document has this shape:

{
  "addons": [
    { ... },
    { ... }
  ]
}

A compact entry for an add-on committed directly to the official registry repository can use only a relative path:

{
  "id": "example_plugin",
  "name": "Example Plugin",
  "description": "Example stored in the official registry repository.",
  "author": "Example Author",
  "version": "1.0.0",
  "type": "plugin",
  "path": "./plugins/example_plugin",
  "trusted": true,
  "official": true
}

When preparing the pull request, include a short description, add-on type, repository link, manifest/version information, dependency notes, and any security-sensitive capabilities such as filesystem, network, shell, desktop-control or credential access.

Registry files for Agent Skills and MCP Connectors are hosted in the same repository as skills.json and mcp_connectors.json. PyGPT still retains its bundled Skills/MCP catalogs as an offline fallback when the default online catalogs cannot be loaded.

Profile portability

The addons directory is not part of profile export/import starting with 2.8.36. Installed packages and their registry belong to the application base workdir and remain available when profiles are exported, imported, duplicated or switched. Older profile archives that still contain addons entries do not restore them into the imported profile.

Legacy custom launcher registration

The external Add-ons manager is the normal choice for redistributable application-wide packages. A custom launcher remains supported when you control application startup directly or need to construct add-on objects programmatically.

Registering components programmatically

pygpt_net.app.run() first registers PyGPT’s built-in components and then appends custom instances provided through keyword arguments.

A minimal launcher can register only one add-on type:

from pygpt_net.app import run
from example_plugin import Plugin

run(
    plugins=[
        Plugin(),
    ],
)

A larger launcher can register all currently supported add-on families:

from pygpt_net.app import run

from example_agent import ExampleAgent
from example_audio_input import ExampleAudioInput
from example_audio_output import ExampleAudioOutput
from example_data_loader import ExampleDataLoader
from example_llm import ExampleLlm
from example_plugin import Plugin
from example_tool import ExampleTool
from example_vector_store import ExampleVectorStore
from example_web_search import ExampleWebSearchEngine


def main():
    run(
        plugins=[Plugin()],
        tools=[ExampleTool()],
        llms=[ExampleLlm()],
        vector_stores=[ExampleVectorStore()],
        loaders=[ExampleDataLoader()],
        audio_input=[ExampleAudioInput()],
        audio_output=[ExampleAudioOutput()],
        web=[ExampleWebSearchEngine()],
        agents=[ExampleAgent()],
    )


if __name__ == "__main__":
    main()

run() owns the application lifecycle and should normally be called only once.

Every add-on ID should be unique. Reusing a built-in ID may replace or shadow a built-in entry in the corresponding registry. For distributable add-ons, prefer the GitHub-derived convention <github_user>_<repo_name>_<addon_name> (for example szczyglis_dev_py_gpt_example_plugin) to reduce collisions between independent authors and repositories.

During application initialization PyGPT attaches the main window or owning plugin to registered objects. Do not assume those references are already available inside your add-on’s __init__() unless the relevant base class explicitly provides them.

Plugins and GUI Tools are different

A plugin participates in application/model events and can expose commands callable by models.

A GUI Tool is an application component registered in the Tools subsystem. It can provide a Tools-menu entry, dialog, tabs, lifecycle hooks and event handling, but registering a GUI Tool does not automatically make it a callable model function.

Use a plugin when the model should call functionality as a tool/command. Use a GUI Tool when you want to extend the desktop application’s interface or provide a standalone utility.

Custom themes and styles

PyGPT themes use one directory per theme ID. Bundled theme assets are stored below pygpt_net/data/css using the same filenames for every theme:

data/css/
├── app.css
├── chat.css
├── chat.wide.css
├── fix_windows.css
├── fix_windows.dark.css
├── fix_windows.light.css
├── dark/
│   ├── app.css
│   ├── app.xml
│   └── chat.css
├── light/
│   ├── app.css
│   ├── app.xml
│   └── chat.css
└── ...

The files have the following roles:

• app.css at the root is the global native Qt stylesheet and is always loaded first.

• <theme-id>/app.css contains native Qt overrides for one theme.

• <theme-id>/app.xml defines the qt-material color palette for that theme.

• chat.css at the root is the global WebEngine chat stylesheet and is used by both the Standard and Wide chat layouts.

• <theme-id>/chat.css contains chat/WebEngine overrides for one theme.

• chat.wide.css is a small global override loaded only when the Wide chat layout is selected.

User themes

Profile-specific themes can be placed below %workdir%/css with exactly the same layout. The name of the directory is the persistent theme ID. For a new custom theme, append -dark or -light to the directory name to declare its compatibility type. The suffix controls runtime Dark/Light behavior but is normally omitted from the Themes menu title. For example, my_custom-dark is shown as My Custom.

For example:

%workdir%/css/
└── my_custom-dark/
    ├── app.css
    ├── app.xml
    └── chat.css

Only files that exist are loaded. Therefore a custom theme may contain only chat.css, only app.css, or a complete set of files. The runtime compatibility type is built dynamically from the active profile theme list: bundled IDs use their built-in type, a new ID ending in -light is Light-compatible, and a new ID ending in -dark is Dark-compatible. New custom IDs without either suffix remain Dark-compatible for backward compatibility.

The compatibility type is not only a CSS fallback. It is the runtime Dark/Light classification used by the application for qt-material inversion, platform-specific fixes, widget behavior, the legacy renderer, and fallback assets. A completely new custom theme inherits the bundled light or dark CSS/XML base matching this runtime type, then applies files from its own directory. If a new theme does not provide app.xml, the corresponding bundled Light or Dark material palette is used.

The -dark / -light suffix is part of the theme ID stored in configuration but is removed from the normal display name. Thus paper-light is shown as Paper. If both paper-dark and paper-light exist at the same time, PyGPT disambiguates them in the menu as Paper (Dark) and Paper (Light).

A profile theme can also use the same ID as a bundled theme, for example:

%workdir%/css/ocean/chat.css

In that case the user files override/extend the bundled ocean theme and keep the built-in ocean compatibility type. For CSS, PyGPT layers the files in this order so later files can override earlier rules:

1. bundled data/css/app.css or data/css/chat.css;

2. bundled data/css/<theme-id>/app.css or chat.css (or the Dark/Light compatibility base for a new custom ID);

3. profile %workdir%/css/app.css or chat.css when present;

4. profile %workdir%/css/<theme-id>/app.css or chat.css when present.

For the Wide chat layout, chat.wide.css is appended after the normal chat.css layers. A profile-level %workdir%/css/chat.wide.css can add final Wide-specific overrides.

For app.xml there is no layering: %workdir%/css/<theme-id>/app.xml replaces the bundled XML for that theme. If it is absent, the bundled XML is used when available; a new custom theme falls back to the bundled light or dark XML selected by its runtime compatibility type.

The built-in themes are discovered from their directories, and profile theme directories are added to the same Themes menu. Theme discovery is refreshed when the theme/profile is reloaded; a restart also refreshes the list.

A minimal custom app.xml can look like this:

<resources>
  <color name="primaryColor">#62d4ff</color>
  <color name="primaryLightColor">#9ce6ff</color>
  <color name="secondaryColor">#111820</color>
  <color name="secondaryLightColor">#18232d</color>
  <color name="secondaryDarkColor">#0b1016</color>
  <color name="primaryTextColor">#ffffff</color>
  <color name="secondaryTextColor">#c8d5df</color>
</resources>

Normal CSS syntax can be used in profile styles. qt-material placeholders such as {QTMATERIAL_PRIMARYCOLOR} and {QTMATERIAL_PRIMARYLIGHTCOLOR} are also expanded when available.

While developing or adjusting a custom theme, you can reload CSS without restarting PyGPT. Open the Logger from the application’s debug tools and enter the css command in its console. PyGPT will reload the active theme CSS immediately, which makes it convenient to edit app.css or chat.css and preview changes iteratively.

Adding a custom model

Models can be created in Config -> Models -> Edit or added to %workdir%/models.json.

For a local OpenAI-compatible server a minimal model entry can look like this:

{
    "id": "my-local-model",
    "name": "My local model",
    "mode": ["chat", "llama_index"],
    "provider": "local_ai",
    "input": ["text"],
    "output": ["text"],
    "ctx": 32768,
    "tokens": 4096,
    "tool_calls": true,
    "custom_api_endpoint": "http://127.0.0.1:1234/v1",
    "custom_api_key": "local",
    "llama_index": {
        "args": [
            {
                "name": "model",
                "value": "my-local-model",
                "type": "str"
            }
        ],
        "env": []
    }
}

custom_api_endpoint and custom_api_key correspond to API base and API key in the Models Editor’s Advanced section. When non-empty, the OpenAI-compatible Chat client uses these values for that model. The local_ai LlamaIndex wrapper also uses them for its OpenAILike client.

Leave the custom API fields empty if the model should continue using the provider/global API settings.

For built-in providers, provider API keys configured in Config -> Settings -> API Keys are normally used automatically if a LlamaIndex argument does not explicitly override them.

Adding a custom plugin

Custom plugins normally inherit from pygpt_net.plugin.base.plugin.BasePlugin. A plugin can:

• define settings with add_option(),

• expose model-callable commands with add_cmd(),

• respond to application events in handle(),

• return command results with reply().

Minimal plugin example

The example below adds one option, exposes one model-callable command, modifies the final system prompt, and handles command execution:

from pygpt_net.core.events import Event
from pygpt_net.plugin.base.plugin import BasePlugin


class Plugin(BasePlugin):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.id = "example_plugin"
        self.name = "Example Plugin"
        self.description = "Small tutorial plugin."
        self.prefix = "Example"

        self.allowed_cmds = ["funny_cmd"]

        self.add_option(
            "append_footer",
            type="bool",
            value=False,
            label="Append footer",
            description="Append an instruction to the final system prompt.",
        )

        self.add_cmd(
            "funny_cmd",
            instruction="return an example response for the provided query",
            params=[
                {
                    "name": "query",
                    "type": "str",
                    "description": "Topic to include in the response.",
                    "required": True,
                },
            ],
            enabled=True,
            description="Enable the example command.",
        )

    def handle(self, event: Event, *args, **kwargs):
        if event.name == Event.POST_PROMPT_END:
            if self.get_option_value("append_footer"):
                event.data["value"] += "\n\nKeep the final answer concise."

        elif event.name in (Event.CMD_SYNTAX, Event.CMD_SYNTAX_INLINE):
            for cmd_name in self.allowed_cmds:
                if self.has_cmd(cmd_name):
                    event.data["cmd"].append(self.get_cmd(cmd_name))

        elif event.name in (Event.CMD_EXECUTE, Event.CMD_INLINE):
            self.execute_commands(event)

    def execute_commands(self, event: Event):
        for item in event.data.get("commands", []):
            if item.get("cmd") != "funny_cmd":
                continue

            params = item.get("params") or {}
            query = str(params.get("query", "")).strip()

            self.reply(
                {
                    "request": {
                        "cmd": "funny_cmd",
                        "params": params,
                    },
                    "result": f"Example response for: {query}",
                },
                event.ctx,
            )
            return

add_cmd() is preferred over manually constructing the command schema. PyGPT can translate registered commands into the appropriate native function/tool format when native API function calls are enabled.

The complete tutorial implementation is available in examples/addons/plugins/example_plugin.

Handling events

Plugins can inspect and, for mutable events, modify data while a request moves through the application.

A basic handler:

from pygpt_net.core.events import Event


def handle(self, event: Event, *args, **kwargs):
    if event.name == Event.USER_SEND:
        # USER_SEND is mutable.
        event.data["value"] = event.data["value"].strip()

    elif event.name == Event.POST_PROMPT_END:
        # Final chance to update the system prompt before the API request.
        event.data["value"] += "\n\nAdditional instruction."

    elif event.name == Event.MODELS_CHANGED:
        # Refresh provider/plugin state after the models list changes.
        self.refresh_models()

Most generic events expose three useful attributes:

• event.name - event identifier.

• event.data - dictionary with event-specific data.

• event.ctx - current CtxItem when the event is associated with a conversation item.

event.data is intentionally mutable for many pre-processing hooks. If the caller reads the changed value after dispatch, modifying it changes the subsequent application flow.

Stopping propagation

A listener can stop the event from being passed to later listeners:

if event.name == Event.INPUT_BEFORE:
    if should_handle_input_myself(event.data["value"]):
        event.stop = True

event.stop stops further event propagation. It does not automatically cancel the operation that created the event unless the relevant caller also checks a cancellation field.

For INPUT_BEFORE, the supported way to abort sending is to set data["stop"]:

if event.name == Event.INPUT_BEFORE:
    if event.data["value"].startswith("/local-only"):
        event.data["stop"] = True
        event.data["silent"] = True

Event reference

Generic event constants are defined by pygpt_net.core.events.Event.

Some constants are retained as compatibility/reserved hooks even if the current core flow primarily uses a newer related event. This is noted below.

• AI_NAME - emitted while the assistant name is being prepared. data["value"] contains the current assistant name and can be replaced.

• AGENT_PROMPT - emitted immediately before an input prompt is passed to an agent runner. data["value"] contains the prompt and can be modified.

• AUDIO_INPUT_RECORD_START - compatibility/reserved hook for an explicit microphone recording-start notification. The current UI primarily uses AUDIO_INPUT_RECORD_TOGGLE for recording control.

• AUDIO_INPUT_RECORD_STOP - compatibility/reserved hook for an explicit microphone recording-stop notification. The current UI primarily uses AUDIO_INPUT_RECORD_TOGGLE.

• AUDIO_INPUT_RECORD_TOGGLE - requests that microphone recording be toggled. Internal callers may additionally provide data["state"] to request a specific state and data["auto"] to mark an automatic/internal transition.

• AUDIO_INPUT_STOP - requests an immediate stop of audio-input processing. Some callers provide data["value"] = True.

• AUDIO_INPUT_TOGGLE - enables or disables speech/audio input. data["value"] is the requested boolean state.

• AUDIO_INPUT_TRANSCRIBE - asks the active Audio input provider to transcribe an existing audio file. data["path"] contains the file path. event.ctx may be a temporary context when transcription is initiated from the Transcript tool.

• AUDIO_OUTPUT_STOP - requests that active TTS generation/playback be stopped. event.ctx may identify the context whose playback is being stopped.

• AUDIO_OUTPUT_TOGGLE - compatibility/reserved generic hook for changing audio-output state. Current UI code normally enables/disables the Audio output plugin through the plugin controller.

• AUDIO_PLAYBACK - requests playback of an already generated audio file. data["audio_file"] contains the path and event.ctx identifies the related context when available.

• AUDIO_READ_TEXT - asks the Audio output provider to synthesize/read text. data["text"] contains the text and data["cache_file"] may contain a preferred cache path. event.ctx identifies the related context.

• BRIDGE_BEFORE - emitted before a prepared conversation request is handed to the bridge/provider layer. Typical fields are data["mode"], data["context"] and data["extra"].

• CMD_EXECUTE - dispatches normal plugin commands/tools for execution. data["commands"] is a list of parsed command dictionaries. data["silent"] may be present for internal calls. event.ctx identifies the tool-call context.

• CMD_INLINE - inline-command equivalent of CMD_EXECUTE. It is used by plugins that intentionally expose inline commands independently of the normal Tools switch.

• CMD_SYNTAX - asks enabled plugins to publish normal command/tool definitions. Plugins append command definitions to data["cmd"]. Depending on the call site, data can also contain prompt, syntax, mode and is_expert.

• CMD_SYNTAX_INLINE - inline-plugin equivalent of CMD_SYNTAX. Plugins append their definitions to data["cmd"].

• CTX_AFTER - emitted after model output has been attached to the current context item and before the response lifecycle fully finishes. event.ctx is the context item; data["mode"] is normally available for text responses.

• CTX_BEFORE - emitted after a context item has been prepared but before it is committed/sent through the normal processing path. event.ctx contains that item and data["mode"] is normally present.

• CTX_BEGIN - compatibility hook for the beginning of context processing. Some plugins still implement it, while the current input pipeline relies mainly on INPUT_BEGIN, CTX_BEFORE, CTX_AFTER and CTX_END.

• CTX_END - emitted when processing of a context item is complete. event.ctx contains the finished item and data["mode"] is normally present.

• CTX_SELECT - emitted when a conversation context is selected. data["value"] contains the context metadata ID.

• DISABLE - emitted when a plugin is disabled. data["value"] contains the plugin ID. Dispatch may be sent to all plugins so a plugin can react to another plugin being disabled.

• ENABLE - emitted when a plugin is enabled. data["value"] contains the plugin ID.

• FORCE_STOP - global request to stop active plugin/background operations, for example after the user presses Stop. data["value"] may be True; listeners should treat the event itself as the stop signal.

• INPUT_ACCEPT - emitted after INPUT_BEFORE has accepted the message and immediately before normal send processing continues. Typical fields are data["value"], data["mode"] and data["multimodal_ctx"].

• INPUT_BEFORE - main mutable input pre-processing hook. It is emitted after raw input reaches the chat send pipeline but before the request is sent. Fields include data["mode"], data["value"], data["multimodal_ctx"], data["stop"] and data["silent"]. Modify value to rewrite input; set stop to True to abort the send.

• INPUT_BEGIN - earliest generic hook in a manual send operation, before the textarea content is fully processed. data["mode"] and data["force"] describe the request; setting data["stop"] can stop the normal manual-send path.

• MODE_BEFORE - emitted before an inline/temporary mode choice is finalized for a request. data["value"] contains the mode, data["prompt"] contains the prompt, and event.ctx is the current context. Replacing data["value"] can redirect the inline mode.

• MODE_SELECT - emitted when the application’s active mode is selected. data["value"] contains the selected mode ID.

• MODEL_BEFORE - emitted before the model instance used by an inline request is finalized. data["mode"] contains the mode and data["model"] contains the proposed ModelItem. Replacing data["model"] can override that choice.

• MODEL_SELECT - emitted when the active model changes. data["value"] contains the selected model ID.

• MODELS_CHANGED - emitted after the model registry is changed, for example by the Models Editor or importer. It has no required payload; listeners normally refresh cached model lists.

• PLUGIN_OPTION_GET - query hook for plugin-owned dynamic values. data["name"] identifies the requested value and the handling plugin writes the answer to data["value"].

• PLUGIN_SETTINGS_CHANGED - emitted after plugin settings are saved. There is no required payload; plugins use it to reload runtime configuration.

• POST_PROMPT - emitted after the main system prompt has been assembled and personalization has been applied, before command/tool syntax is appended. Typical fields are mode, reply, internal, value and is_expert. event.ctx is set. data["value"] is mutable.

• POST_PROMPT_ASYNC - emitted in the bridge worker after processing has moved to the asynchronous request path. It is a late mutable system-prompt hook. Typical fields are mode, reply and value; event.ctx is set.

• POST_PROMPT_END - final mutable system-prompt hook immediately before the provider request is executed. Typical fields are mode, reply and value; event.ctx is set. This is usually the safest event for a plugin that must append a final instruction.

• PRE_PROMPT - early hook for the system prompt before the main prompt builder finishes processing it. data["mode"] and data["value"] are provided; expert calls may additionally provide data["is_expert"].

• SETTINGS_CHANGED - emitted after the main Settings editor saves changes. It has no required payload. Plugins/providers can use it to refresh configuration-dependent state.

• SYSTEM_PROMPT - emitted while the base/final system prompt is being assembled. data["value"] is the mutable prompt text and data["mode"] identifies the current mode. Depending on the call site, silent or is_expert may also be present.

• TOOL_OUTPUT_RENDER - WebView rendering hook for custom tool output. Typical fields are data["tool"] (plugin/tool ID), data["content"] (tool output data), data["html"] (custom HTML result) and data["multiple"] (whether the renderer is handling one of multiple tool outputs). A plugin can populate html to provide custom rendering.

• UI_ATTACHMENTS - asks listeners whether attachment UI should be visible for a mode. data["mode"] identifies the mode and mutable data["value"] is the visibility boolean.

• UI_VISION - asks listeners whether the inline Vision availability indicator should be visible. data["mode"] identifies the mode and mutable data["value"] is the visibility boolean.

• USER_NAME - emitted while the user display/name value is prepared. data["value"] contains the current name and can be replaced.

• USER_SEND - emitted for a manual user send after the text has been read from the input widget but before attachment handling and the kernel/bridge input event. data["mode"] contains the current mode and mutable data["value"] contains the text.

Event ordering example

For a typical manual text message, a simplified sequence is approximately:

INPUT_BEGIN
    |
    v
USER_SEND
    |
    v
INPUT_BEFORE
    |
    v
INPUT_ACCEPT
    |
    v
CTX_BEFORE
    |
    v
PRE_PROMPT
    |
    v
SYSTEM_PROMPT
    |
    v
POST_PROMPT
    |
    v
CMD_SYNTAX / CMD_SYNTAX_INLINE
    |
    v
POST_PROMPT_ASYNC
    |
    v
POST_PROMPT_END
    |
    v
provider / model request
    |
    v
CTX_AFTER
    |
    v
CTX_END

Exact ordering can differ for image generation, Realtime, agents, internal calls and tool continuations.

Event debug logging

Event flow can be inspected by enabling:

Config -> Settings -> Debug -> Log events

Adding a custom GUI Tool

A custom application tool uses the Tools subsystem and is registered through tools=[...].

Minimal example:

from PySide6.QtGui import QAction
from pygpt_net.tools.base import BaseTool


class ExampleTool(BaseTool):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.id = "example_tool"
        self.opened = False

    def setup(self):
        # self.window is available here.
        pass

    def setup_menu(self) -> dict:
        action = QAction("Example Tool", self.window)
        action.triggered.connect(self.toggle)
        return {"example": action}

    def toggle(self):
        self.opened = not self.opened
        self.window.ui.dialogs.alert(
            f"Example tool opened: {self.opened}"
        )

    def on_reload(self):
        # Called when profile/workdir is reloaded.
        pass

    def on_exit(self):
        # Release external resources here.
        pass

A real GUI Tool can register dialogs, tabs, menus, theme hooks and update hooks. See examples/addons/tools/example_tool for a complete dialog-based tutorial.

Adding a custom LLM wrapper

LLM wrappers are used by the LlamaIndex integration and embeddings. They are not the same thing as the normal native Chat-provider path.

A wrapper derives from pygpt_net.provider.llms.base.BaseLLM and declares a unique id and supported types.

A compact OpenAI-backed example:

from llama_index.llms.openai import OpenAI as LlamaOpenAI

from pygpt_net.core.types import MODE_LLAMA_INDEX
from pygpt_net.provider.llms.base import BaseLLM


class ExampleLlm(BaseLLM):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.id = "example_llm"
        self.name = "Example LlamaIndex provider"
        self.type = [MODE_LLAMA_INDEX, "embeddings"]

    def llama(self, window, model, stream=False):
        args = self.parse_args(model.llama_index, window)
        args.setdefault("model", model.id)

        # Global fallback used by this OpenAI-based tutorial.
        args.setdefault(
            "api_key",
            window.core.config.get("api_key", ""),
        )

        # Per-model values have highest priority when configured.
        if model.custom_api_key:
            args["api_key"] = model.custom_api_key

        if model.custom_api_endpoint:
            args["api_base"] = model.custom_api_endpoint

        args = self.inject_llamaindex_http_clients(
            args,
            window.core.config,
        )
        return LlamaOpenAI(**args)

For embeddings, implement llama_embeddings() and return a LlamaIndex BaseEmbedding instance.

Depending on its purpose, a wrapper can implement:

• llama() - return a LlamaIndex LLM for RAG and LlamaIndex agents.

• llama_embeddings() - return a LlamaIndex embedding model.

• get_models() - optionally expose provider-side model discovery.

Legacy chat() and completion() methods remain in the base interface for compatibility, but current custom LlamaIndex integrations should normally follow the llama() path.

See examples/addons/llms/example_llm for a complete LLM + embeddings example.

Adding a custom vector store

Vector store providers derive from pygpt_net.provider.vector_stores.base.BaseStore and are registered through vector_stores=[...].

The current index creation path passes the LLM and embedding model separately. Older ServiceContext-based examples are no longer appropriate.

Example:

from llama_index.core import StorageContext, load_index_from_storage

from pygpt_net.provider.vector_stores.base import BaseStore


class ExampleVectorStore(BaseStore):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.id = "ExampleVectorStore"
        self.prefix = "example_"

    def create(self, id, embed_model=None):
        if self.exists(id):
            return

        index = self.index_from_empty(embed_model)
        self.store(id=id, index=index)

    def get(self, id, llm=None, embed_model=None):
        if not self.exists(id):
            self.create(id, embed_model)

        storage_context = StorageContext.from_defaults(
            persist_dir=self.get_path(id),
        )
        index = load_index_from_storage(
            storage_context,
            llm=llm,
            embed_model=embed_model,
        )
        self.indexes[id] = index
        return index

    def store(self, id, index=None):
        index = index or self.indexes[id]
        index.storage_context.persist(
            persist_dir=self.get_path(id),
        )
        self.indexes[id] = index

BaseStore already provides common helpers such as exists(), remove(), truncate(), remove_document(), attach() and get_path().

See examples/addons/vector_stores/example_vector_store for the complete tutorial.

Adding a custom data loader

Data loaders are registered through loaders=[...] and normally derive from pygpt_net.provider.loaders.base.BaseLoader.

The loader registers metadata/configuration and returns a LlamaIndex reader:

from llama_index.core.readers.base import BaseReader
from llama_index.core.schema import Document

from pygpt_net.provider.loaders.base import BaseLoader


class ExampleDataLoader(BaseLoader):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.id = "example_text"
        self.name = "Example text loader"
        self.extensions = ["exampletxt"]
        self.type = ["file"]

        self.init_args = {
            "encoding": "utf-8",
        }
        self.init_args_types = {
            "encoding": "str",
        }

    def get(self) -> BaseReader:
        return ExampleReader(**self.get_args())


class ExampleReader(BaseReader):
    def __init__(self, encoding="utf-8", **kwargs):
        super().__init__(**kwargs)
        self.encoding = encoding

    def load_data(self, file, extra_info=None):
        with open(file, "r", encoding=self.encoding) as handle:
            text = handle.read()

        return [
            Document(
                text=text,
                metadata=extra_info or {},
            )
        ]

Current LlamaIndex Document objects use the metadata field for metadata.

See examples/addons/loaders/example_loader for a runnable .example file reader that returns real LlamaIndex Document objects.

Adding audio providers

Custom speech-to-text providers are registered with audio_input=[...]. Custom text-to-speech providers are registered with audio_output=[...].

Audio input

A speech-to-text provider receives the path that PyGPT wants transcribed. Do not hard-code input.wav; application-managed recordings can live under %workdir%/tmp.

from pygpt_net.provider.audio_input.base import BaseProvider


class ExampleAudioInput(BaseProvider):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.id = "example_audio_input"
        self.name = "Example audio input"

    def transcribe(self, path: str) -> str:
        client = self.plugin.window.core.api.openai.get_client()

        with open(path, "rb") as audio_file:
            return client.audio.transcriptions.create(
                model="whisper-1",
                file=audio_file,
                response_format="text",
            )

    def is_configured(self) -> bool:
        return bool(
            self.plugin.window.core.config.get("api_key")
        )

See examples/addons/audio_input/example_audio_input for a runnable WAV-inspector provider with provider-specific settings.

Audio output

Use prepare_output_path() rather than a fixed filename. PyGPT stores generated TTS working files under %workdir%/tmp/audio_output and uses unique names, which also avoids file-lock issues on Windows.

from pygpt_net.provider.audio_output.base import BaseProvider


class ExampleAudioOutput(BaseProvider):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.id = "example_audio_output"
        self.name = "Example audio output"

    def speech(self, text: str) -> str:
        client = self.plugin.window.core.api.openai.get_client()
        path = self.prepare_output_path(extension="mp3")

        response = client.audio.speech.create(
            model="tts-1",
            voice="alloy",
            input=text,
        )
        response.stream_to_file(path)
        return str(path)

See examples/addons/audio_output/example_audio_output for a runnable provider that generates a WAV tone through the normal audio-output path.

Adding a web search provider

Web search providers are registered with web=[...]. They provide a search-engine implementation used by the Web search plugin.

A provider normally defines its own settings and implements search():

from pygpt_net.provider.web.base import BaseProvider


class ExampleWebSearchEngine(BaseProvider):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.id = "example_web_search"
        self.name = "Example search engine"
        self.type = ["search_engine"]

    def search(self, query: str, limit: int = 10, offset: int = 0):
        # Call the external search API here.
        # Return result URLs as a list of strings.
        return [
            "https://example.com/result-1",
            "https://example.com/result-2",
        ]

    def is_configured(self, cmds):
        return True

A production provider should define credentials/settings with plugin.add_option() and use self.plugin.get_url() or another PyGPT/network helper appropriate for the integration.

See examples/addons/web/example_web for a runnable Wikipedia/MediaWiki OpenSearch provider that performs a real HTTP search.

Adding a custom agent

Agent providers are registered with agents=[...]. The selectable Custom agents mode is LlamaIndex-backed; built-in implementations live under pygpt_net.provider.agents.llama_index.

For a completely new provider, derive from pygpt_net.provider.agents.base.BaseAgent and implement the current provider contract. A provider typically defines its ID, agent type/workflow mode, configuration options, and get_agent() implementation that returns the runtime workflow/agent object used by PyGPT.

For workflow experimentation that does not require a new Python provider, prefer the built-in Custom agent builder, which can construct and save node-based Custom agents workflows directly from the UI.

See examples/addons/agents/example_agent for the current extension pattern and use the base interfaces as the source of truth for method signatures.

Source code as API reference

The base interfaces are the source of truth for method signatures. Useful implementation directories are:

• pygpt_net.plugin

• pygpt_net.tools

• pygpt_net.provider.llms

• pygpt_net.provider.vector_stores

• pygpt_net.provider.loaders

• pygpt_net.provider.audio_input

• pygpt_net.provider.audio_output

• pygpt_net.provider.web

• pygpt_net.provider.agents

For complete runnable tutorial files, use the repository examples directory.