How to use external Python packages

Using external Python package

ActiveGate plugins are standard Python scripts running in an embedded Python engine inside Remote Plugin Module. Every standard Python package can be used in a plugin. For example, you can use url.request for a simple import like this:

import urllib.request
from ruxit.api.base_plugin import RemoteBasePlugin

class MyPlugin (RemoteBasePlugin):
    def query(self, **kwargs):
        response = urllib.request.urlopen('https://www.google.com/')
        response_data = response.read()

Sometimes you may find it necessary to use an external package, for example Python driver for Apache Cassandra.

The installation of such a package requires two steps:

1. Development - prepare the environment for testing a plugin in ActiveGate Plugin Simulator.
2. Deployment - prepare the plugin package for an upload to DESK Server and the ActiveGate.

Development environment with external package

Let's create a plugin using driver for Apache Cassandra:

from cassandra.cluster import Cluster
from ruxit.api.base_plugin import RemoteBasePlugin

class MyCassandraPlugin(RemoteBasePlugin):
    def query(self, **kwargs):
        cluster = Cluster(self.config['cluster'].split(','))
        session = cluster.connect(self.config['space'])

In the deployment stage, the plugin should be tested with ActiveGate Plugin Simulator. Add the plugin.json definition for Python script:

{
	"name": "custom.remote.python.MyCassandraPlugin",
	"version": "1.0",
	"type": "python",
	"entity": "CUSTOM_DEVICE",
	"processTypeNames": ["PYTHON"],
	"technologies": ["CASSANDRA"],
	"source": {
		"package": "my_cassandra_plugin",
		"className": "MyCassandraPlugin",
		"activation": "Remote"
	},
	"configUI": {
		"displayName": "My Cassandra Active Plugin",
		"properties": [{
			"key": "cluster",
			"displayName": "Cluster IPs",
			"displayHint": "list of IP addresses"
		}, {
			"key": "space",
			"displayName": "Cassandra space"
        }]
	},
	"properties": [{
		"key": "cluster",
		"type": "String"
	}, {
		"key": "space",
		"type": "String"
    }]
}

Install the Cassandra driver in your development environment.

Every package should contain its documentation with instructions on installation process. For example, to install the Python driver for Apache Cassandra, execute the following command:

pip3 install cassandra-driver

This is sufficient for ActiveGate Plugin Simulator to run a plugin using an external package.

Plugin deployment with external package

In the deployment stage, you upload the developed plugin to DESK Server and production Environment ActiveGate.

For the deployment to work, the plugin.json has to refer to the Cassandra driver.

Update the plugin.json definition with install_requires field in the source object. You may add version condition if necessary e.g. cassadra_driver>=1.0.0:

{
	"name": "custom.remote.python.MyCassandraPlugin",
	"version": "1.0",
	"type": "python",
	"entity": "CUSTOM_DEVICE",
	"processTypeNames": ["PYTHON"],
	"technologies": ["CASSANDRA"],
	"source": {
		"package": "my_cassandra_plugin",
		"className": "MyCassandraPlugin",
		"activation": "Remote",
        "install_requires": ["cassandra-driver>=1.0.0"],
	},
	"configUI": {
		"displayName": "My Cassandra Active Plugin",
		"properties": [{
			"key": "cluster",
			"displayName": "Cluster IPs",
			"displayHint": "list of IP addresses"
		}, {
			"key": "space",
			"displayName": "Cassandra space"
        }]
	},
	"properties": [{
		"key": "cluster",
		"type": "String"
	}, {
		"key": "space",
		"type": "String"
    }]
}

The command plugin_sdk build_plugin builds the plugin zip archive and uploads it to DESK Server.

The zip file is located in

/opt/DESK/remotepluginmodule/plugin_deployment/custom.remote.python.MyCassandraPlugin.zip on Linux

or

c:\Program Files\DESK\remotepluginmodule\plugin_deployment\custom.remote.python.MyCassandraPlugin.zip on Windows

The command plugin_sdk build_plugin uses pip package manager to download and prepare any package needed by a plugin.

Building plugins with external packages

The command plugin_sdk build_plugin automatically prepares a plugin with the external libraries that are defined in the install_requires field of plugin.json. You don't need to place an external package in the source plugin directory.

The plugin zip file is located in

/opt/DESK/remotepluginmodule/plugin_deployment on Linux

or

c:\Program Files\DESK\remotepluginmodule\plugin_deployment on Windows

Built-in packages

There are a few external packages already used by the plugin module. These aren't included in plugin.json:

• certifi
• chardet
• docker
• docker_pycreds
• idna
• jsonchema
• plop
• request
• setuptools
• six
• urllib3
• websocket

You can find these libraries in the THIRDPARTYLICENSEREADME.html file located in

/opt/DESK/remotepluginmodule/agent on Linux

or

c:\Program Files\DESK\remotepluginmodule\agent on Windows

Native (C-compiled) components of Python packages

If the python package that you want to use contains any native dependencies these are resolved by pip automatically so there are are additional steps required during plugin deployment phase.

Note the you must build a plugin on the same system (Linux or Windows) and hardware platform (ActiveGate supports only 64-bit systems) as the destination production Environment ActiveGate. Native components of the Python package are not compatible between different systems and hardware platforms.

If you encounter any obstacles in installing a package using pip, copy the package folder together with the native libraries, but without the dist-info directory, to the plugin directory.

Loading external libraries

Keep in mind the order in which the libraries are loaded:

1. Engine libraries (Docker client, request, urllib3, etc.)
2. Built-in plugin libraries (in alphabetical order of directories)
3. Custom plugin libraries (in alphabetical order of directories)

Custom and built-in external packages may be of a different version. Loading of the newest version isn't guaranteed.