Accessibility
Since version 2.2.8
, PyGPT has added beta support for disabled people and voice control. This may be very useful for blind people.
In the Config / Accessibility
menu, you can turn on accessibility features such as:
activating voice control
translating actions and events on the screen with audio speech
setting up keyboard shortcuts for actions.
Using voice control
Voice control can be turned on in two ways: globally, through settings in Config -> Accessibility
, and by using the Voice control (inline)
plugin. Both options let you use the same voice commands, but they work a bit differently - the global option allows you to run commands outside of a conversation, anywhere, while the plugin option lets you execute commands directly during a conversation – allowing you to interact with the model and execute commands at the same time, within the conversation.
In the plugin (inline) option, you can also turn on a special trigger word that will be needed for content to be recognized as a voice command. You can set this up by going to Plugins -> Settings -> Voice Control (inline)
:
Magic prefix for voice commands
Tip: When the voice control is enabled via a plugin, simply provide commands while providing the content of the conversation by using the standard Microphone button.
Enabling voice control globally
Turn on the voice control option in Config / Accessibility
:
Enable voice control (using microphone)
Once you enable this option, an Voice Control
button will appear at the bottom right corner of the window. When you click on this button, the microphone will start listening; clicking it again stops listening and starts recognizing the voice command you said. You can cancel voice recording at any time with the ESC
key. You can also set a keyboard shortcut to turn voice recording on/off.
Voice command recognition works based on a model, so you don’t have to worry about saying things perfectly.
Here’s a list of commands you can ask for by voice:
Get the current application status
Exit the application
Enable audio output
Disable audio output
Enable audio input
Disable audio input
Add a memo to the calendar
Clear memos from calendar
Read the calendar memos
Enable the camera
Disable the camera
Capture image from camera
Create a new context
Go to the previous context
Go to the next context
Go to the latest context
Focus on the input
Send the input
Clear the input
Get current conversation info
Get available commands list
Stop executing current action
Clear the attachments
Read the last conversation entry
Read the whole conversation
Rename current context
Search for a conversation
Clear the search results
Send the message to input
Append message to current input without sending it
Switch to chat mode
Switch to chat with files (llama-index) mode
Switch to the next mode
Switch to the previous mode
Switch to the next model
Switch to the previous model
Add note to notepad
Clear notepad contents
Read current notepad contents
Switch to the next preset
Switch to the previous preset
Switch to the chat tab
Switch to the calendar tab
Switch to the draw (painter) tab
Switch to the files tab
Switch to the notepad tab
Switch to the next tab
Switch to the previous tab
Start listening for voice input
Stop listening for voice input
Toggle listening for voice input
More commands coming soon.
Just ask for an action that matches one of the descriptions above. These descriptions are also known to the model, and relevant commands are assigned to them. When you voice a command that fits one of those patterns, the model will trigger the appropriate action.
For convenience, you can enable a short sound to play when voice recording starts and stops. To do this, turn on the option:
Audio notify microphone listening start/stop
To enable a sound notification when a voice command is recognized and command execution begins, turn on the option:
Audio notify voice command execution
For voice translation of on-screen events and information about completed commands via speech synthesis, you can turn on the option:
Use voice synthesis to describe events on the screen.