Welcome to the Hyrrokkin documentation

Introduction - Basic Concepts

Topologies

Hyrrokkin is a library which manages the execution of computational graphs, which are termed topologies.

Each topology consists of executable components called nodes.

Nodes, Ports and Links

Nodes are associated with a class which implements the node's behaviour, most notably the class will implement a run method which transforms a set of values received through that node's input ports into a set of values sent out via the node's output ports.

A topology will also contain links, which connect the output port of one node to the input port of another.

Packages and Package Configurations

Node types and link types are bundled together into a package containing related functionality.

A package must define a package configuration class, a single instance of which will be created when a topology is loaded.

Configuration instances can provide useful services to and can be accessed by any node or any other configuration instance. They must also define a factory method called create_node which will be called to create instances of nodes which belong to the package.

Persistent storage

As well as persisting the structure of a topology, Hyrrokkin manages the persistent storage of key-value pairs for each node and configuration within a topology.

There are two types of persistent storage:

• properties - key-value pairs where the values are JSON-serialisable
• data - key-value pairs where the values are binary data

Clients

Clients may be attached to specific nodes and configurations and interact with them by exchanging messages. Messages consist of one or more parts, each part may be binary, text or a JSON-serialisable object.

An Example Package

Consider a simple example package, NumberGraph, consisting of a configuration class and three node classes:

• NumberGraphConfiguration

Maintains a cache of previously computed factorisations, which can be accessed by all nodes. A client may attach and request that the cache be cleared.

• NumberInputNode

Stores an integer number in a property named value and outputs this value via its output port data_out.

Clients can send a new value to this node to update this property and trigger re-execution of downstream parts of the topology.

• PrimeFactorsNode

Expects numeric input values via its input port data_in, computes a list of its prime factors which are output as an integer-list value via port data_out.

If any clients are attached, they will be notified of the elapsed time after a set of prime factors have been calculated.

• NumberDisplayNode

Receives a list of numbers via its input port data_in, and communicates those to any attached clients.

Package Schema

Each Package, and the Links and Nodes it contains, is specified in a JSON formatted document that represents the schema of that Package.

{
    "id": "numbergraph",
    "metadata": {
        "name": "Number Graph",
        "version": "0.0.1",
        "description": "a small example package for manipulating numbers"
    },
    "node_types": {
        "number_input_node": {
            "metadata": {
                "name": "Number Input Node",
                "description": "Define an integer value"
            },
            "output_ports": {
                "data_out": {
                    "link_type": "numbergraph:integer"
                }
            }
        },
        "prime_factors_node": {
            "metadata": {
                "name": "Prime Factors Node",
                "description": "Calculate the prime factors of each input number"
            },
            "input_ports": {
                "data_in": {
                    "link_type": "numbergraph:integer",
                    "allow_multiple_connections": false
                }
            },
            "output_ports": {
                "data_out": {
                    "link_type": "numbergraph:integerlist"
                }
            }
        },
        "number_display_node": {
            "metadata": {
                "name": "Number Display Node",
                "description": "Display all input numbers"
            },
            "input_ports": {
                "data_in": {
                    "link_type": "numbergraph:integerlist",
                    "allow_multiple_connections": true
                }
            }
        }
    },
    "link_types": {
        "integer": {
            "metadata": {
                "name": "Integer",
                "description": "This type of link carries integer values"
            }
        },
        "integerlist": {
            "metadata": {
                "name": "IntegerList",
                "description": "This type of link carries values that are lists of integers"
            }
        }
    }
}

When refering to a link type, the package id should be used as a prefix, <package-id>:<link-type-id>. In this example, numbergraph:integer refers to the link type number defined in the numbergraph example package. This allows packages to refer to link types defined in other packages when defining nodes.

The package itself may define:

• metadata provides descriptive information, including name and description attributes
• optionally, a configuration and the names of any clients that may attach to the configuration

Each node is associated with the following information:

• input_ports and output_ports specify the names and link types of the ports attached to a node
• ports cannot accept multiple connections unless the allow_multiple_connections is set to true
• metadata provides descriptive information, including name and description attributes
• the names of any clients that may attach to the node

Each Link is associated with the following information:

• metadata provides descriptive information, including name and description attributes

Filesystem layout

All files that comprise a packages are stored under a root directory which contains the package schema, named schema.json

Hyrrokkin currently supports packages which implement nodes and configurations using javascript or python, but the interfaces are the same.

pythonjavascript

schema.json
python.json
python/
   number_input_node.py
   prime_factors_node.py
   number_display_node.py
   prime_factors_worker.py

schema.json
javascript.json
javascript/
   number_input_node.js
   prime_factors_node.js
   number_display_node.js
   prime_factors_worker.js

The file python.json / javascript.json defines how the engine will load the package configuration.

pythonjavascript

{
   "configuration_class": ".python.numbergraph_configuration.NumbergraphConfiguration"
}

{
  "source_paths": [
    "javascript/numbergraph_configuration.js",
    "javascript/number_input_node.js",
    "javascript/prime_factors_node.js",
    "javascript/number_display_node.js"
  ]
}

NumberGraphConfiguration

A package configuration is implemented as a class with a constructor accepting a services object. and an optional load method which is called to load up any additional resources which are needed by the configuration and its nodes.

In this example, the configuration can offer a service to store and retrieve factorisations, that can be re-used by all nodes.

pythonjavascript

#   Hyrrokkin - a library for building and running executable graphs
#
#   MIT License - Copyright (C) 2022-2025  Visual Topology Ltd

import pickle
import asyncio
import json

from hyrrokkin_engine.configuration_interface import ConfigurationInterface

from .number_display_node import NumberDisplayNode
from .prime_factors_node import PrimeFactorsNode
from .number_input_node import NumberInputNode

class NumbergraphConfiguration(ConfigurationInterface):

    def __init__(self, services):
        self.services = services
        self.client_services = {}
        self.prime_factor_cache = None
        self.last_save_cache_task = None

    async def load(self):
        cache_data = await self.services.get_data("prime_factors")
        self.prime_factor_cache = pickle.loads(cache_data) if cache_data else {}
        self.services.set_status(f"loaded cache ({len(self.prime_factor_cache)} items)","info")

    async def create_node(self, node_type_id, node_services):
        match node_type_id:
            case "number_display_node": return NumberDisplayNode(node_services)
            case "number_input_node": return NumberInputNode(node_services)
            case "prime_factors_node": return PrimeFactorsNode(node_services)
            case _: return None

    def update_clients(self):
        for id in self.client_services:
            self.client_services[id].send_message(len(self.prime_factor_cache))

    def get_prime_factors(self, n):
        if n in self.prime_factor_cache:
            return self.prime_factor_cache[n]
        else:
            return None

    async def save_cache(self):
        await self.services.set_data("prime_factors", pickle.dumps(self.prime_factor_cache))

    async def set_prime_factors(self, n, factors):
        self.prime_factor_cache[n] = factors
        await self.save_cache()
        self.update_clients()

    def open_client(self, session_id, client_id, client_options, client_service):
        self.client_services[session_id+":"+client_id] = client_service

        def message_handler(msg):
            if msg == "clear_cache":
                self.last_save_cache_task = asyncio.get_event_loop().create_task(self.save_cache())

        client_service.set_message_handler(message_handler)
        self.update_clients()

    # implement encode/decode for the link types defined in this package

    def encode(self, value, link_type):
        if link_type == "integer":
            v = str(value)
        elif link_type == "integerlist":
            v = list(map(lambda i: str(i), value))
        encoded_bytes = json.dumps(v).encode("utf-8")
        return encoded_bytes

    def decode(self, encoded_bytes, link_type):
        v = json.loads(encoded_bytes.decode("utf-8"))
        if link_type == "integer":
            return int(v)
        elif link_type == "integerlist":
            return list(map(lambda i: int(i), v))

//   Hyrrokkin - a library for building and running executable graphs
//
//   MIT License - Copyright (C) 2022-2025  Visual Topology Ltd

var numbergraph = numbergraph || {};

numbergraph.NumbergraphConfiguration = class {

    constructor(services) {
        this.services = services;
        this.prime_factor_cache = null;
        this.client_services = {};
        this.encoder = new TextEncoder();
        this.decoder = new TextDecoder();
    }

    async create_node(node_type_id, node_services) {
        switch (node_type_id) {
            case "number_display_node": return new numbergraph.NumberDisplayNode(node_services);
            case "number_input_node": return new numbergraph.NumberInputNode(node_services);
            case "prime_factors_node": return new numbergraph.PrimeFactorsNode(node_services);
            default: return null;
        }
    }

    async load() {
        let cache_data = await this.services.get_data("prime_factors")
        this.prime_factor_cache = cache_data ? JSON.parse(this.decoder.decode(cache_data)) : {};
        this.services.set_status("loaded cache (" + Object.keys(this.prime_factor_cache).length + " items)","info");
    }

    update_clients() {
        for(let id in this.client_services) {
            this.client_services[id].send_message(Object.keys(this.prime_factor_cache).length);
        }
    }

    get_prime_factors(n) {
        let key = String(n);
        if (key in this.prime_factor_cache) {
            return this.prime_factor_cache[key].map(item=>BigInt(item));
        } else {
            return null;
        }
    }

    async save_cache() {
        await this.services.set_data("prime_factors", this.encoder.encode(JSON.stringify(this.prime_factor_cache)).buffer);
    }

    async set_prime_factors(n, factors) {
        this.prime_factor_cache[String(n)] = factors.map(item=>String(item));
        await this.save_cache();
        this.update_clients();
    }

    async clear_cache() {
        this.prime_factor_cache = {};
        await this.save_cache();
        this.update_clients();
    }

    open_client(session_id, client_id, client_options, client_service) {
        this.client_services[session_id+":"+client_id] = client_service;
        client_service.set_message_handler((msg) => {
            if (msg === "clear_cache") {
                this.clear_cache().then(() => {});
            }
        });
        this.update_clients();
    }

    close_client(session_id, client_id) {
        delete this.client_services[session_id+":"+client_id];
    }

    // implement encode/decode for the link types defined in this package

    encode(value, link_type) {
        let v = "";
        if (link_type === "integer") {
            v = String(value);
        } else if (link_type === "integerlist") {
            v = value.map(n => String(n));
        }
        let encoded_bytes = (new TextEncoder()).encode(JSON.stringify(v)).buffer;
        return encoded_bytes
    }

    decode(encoded_bytes, link_type) {
        let v = JSON.parse((new TextDecoder()).decode(encoded_bytes));
        if (link_type == "integer") {
            return BigInt(v);
        } else if (link_type == "integerlist") {
            return v.map(n => BigInt(n));
        } else {
            return null;
        }
    }

}

hyrrokkin_engine.registry.register_configuration_factory("numbergraph",(configuration_services) => new numbergraph.NumbergraphConfiguration(configuration_services));

The services API provides methods for nodes and configurations to get and set binary data: get_data(name,value) and set_data(name,value) where values of type bytes.

NumberInputNode

When a node is constructed, the constructor is passed a service API object, providing various useful services. This services API is very similar to that passed to a configuration constructor.

Consider the NumberInputNode:

pythonjavascript

#   Hyrrokkin - a library for building and running executable graphs
#
#   MIT License - Copyright (C) 2022-2025  Visual Topology Ltd

from hyrrokkin_engine.node_interface import NodeInterface

class NumberInputNode(NodeInterface):

    DEFAULT_VALUE = "10"

    def __init__(self, services):
        self.services = services
        self.clients = {}

    def open_client(self, session_id, client_name, client_options, client_service):
        self.clients[(session_id,client_name)] = client_service
        client_service.set_message_handler(lambda *msg: self.__handle_message(session_id, client_name, *msg))
        client_service.send_message(self.services.get_property("value",NumberInputNode.DEFAULT_VALUE))

    def close_client(self, session_id, client_name):
        del self.clients[(session_id,client_name)]

    def __handle_message(self, session_id, client_name, value):
        self.services.set_property("value", value)
        self.services.request_run()
        for key in self.clients:
            if key != (session_id,client_name):
                self.clients[key].send_message(self.services.get_property("value",NumberInputNode.DEFAULT_VALUE))

    async def run(self, inputs):
        value = int(self.services.get_property("value", NumberInputNode.DEFAULT_VALUE))
        return { "data_out": value }

//   Hyrrokkin - a library for building and running executable graphs
//
//   MIT License - Copyright (C) 2022-2025  Visual Topology Ltd

var numbergraph = numbergraph || {};

numbergraph.NumberInputNode = class {

    static DEFAULT_VALUE="10";

    constructor(services) {
        this.services = services;
        this.clients = {};
    }

    open_client(session_id, client_name, client_options, client_service) {
        this.clients[session_id+":"+client_name] = client_service;
        client_service.set_message_handler((...msg) => this.handle_message(session_id, client_name, ...msg));
        client_service.send_message(this.services.get_property("value",numbergraph.NumberInputNode.DEFAULT_VALUE));
    }

    close_client(session_id, client_name) {
        delete this.clients[session_id+":"+client_name];
    }

    handle_message(session_id, client_id, value) {
        this.services.set_property("value",value);
        // update other clients
        for(let key in this.clients) {
            if (key !== session_id+":"+client_id) {
                this.clients[key].send_message(this.services.get_property("value",numbergraph.NumberInputNode.DEFAULT_VALUE));
            }
        }
        this.services.request_run();
    }

    async run(inputs) {
        return {"data_out": BigInt(this.services.get_property("value",numbergraph.NumberInputNode.DEFAULT_VALUE))};
    }
}

The integer value is stored in a value property and the services api get_property(name,value) and set_property(name,value) are used to retrieve and update the value.

Property names must be strings and values must be JSON-serialisable objects.

To communicate with clients, nodes (or configurations) implement open_client and close_client methods. In the example above, the NumberInputNode expects messages consisting of a single value, used to refresh the number stored by the node.

When the node is run, its stored value is output on port data_out

If the value passed by a client is not an integer, the node will issue a warning via the service api set_status. The following set of service APIs related to status updates:

service API	Purpose
set_status(msg,"info")	sets the status as INFORMATIONAL accompanied by message msg
set_status(msg,"warning")	sets the status as WARNING accompanied by message msg
set_status(msg,"error")	sets the status as ERROR accompanied by message msg
set_status("")	clears the status associated with this node

PrimeFactorsNode

This node performs processing on an input integer value to produce a list of integer values which are its prime factors.

pythonjavascript

#   Hyrrokkin - a library for building and running executable graphs
#
#   MIT License - Copyright (C) 2022-2025  Visual Topology Ltd


import asyncio
import sys
import json
import time

from hyrrokkin_engine.node_interface import NodeInterface

class PrimeFactorsNode(NodeInterface):

    def __init__(self,services):
        self.services = services
        self.sub_process = None

    def reset_run(self):
        if self.sub_process is not None:
            self.sub_process.terminate()

    async def run(self, inputs):
        start_time = time.time()
        n = inputs.get("data_in",None)
        if n is not None:
            self.services.set_status("calculating...", "info");
            if n < 2:
                raise Exception(f"input value {n} is invalid (< 2)")

            prime_factors = self.services.get_configuration().get_prime_factors(n)

            if not prime_factors:
                prime_factors = await self.find_prime_factors(n)

            elapsed_time_ms = int(1000*(time.time() - start_time))

            if prime_factors is not None:
                self.services.set_status(f"{elapsed_time_ms} ms", "info");
                await self.services.get_configuration().set_prime_factors(n, prime_factors)
                return { "data_out":prime_factors }
            else:
                self.services.set_status("Failed to calculate prime factors", "error");
                raise Exception("prime factors error")

    async def find_prime_factors(self,n):
        script_path = self.services.resolve_resource("python/prime_factors_worker.py").replace("file://", "")
        prime_factors = None
        try:
            self.sub_process = await asyncio.create_subprocess_exec(sys.executable, script_path, str(n),
                                                                   stdout=asyncio.subprocess.PIPE,
                                                                   stderr=asyncio.subprocess.PIPE)
            stdout, _ = await self.sub_process.communicate()
            prime_factors = json.loads(stdout.decode().strip("\n"))
        except:
            pass
        self.sub_process = None
        return prime_factors

//   Hyrrokkin - a library for building and running executable graphs
//
//   MIT License - Copyright (C) 2022-2025  Visual Topology Ltd

var numbergraph = numbergraph || {};

numbergraph.PrimeFactorsNode = class {

    constructor(services) {
        this.services = services;
        this.elapsed_time_ms = null;
        this.worker = null;
        this.reject_worker = null;
    }

    reset_run() {
        if (this.worker !== null) {
            this.worker.terminate();
            this.worker = null;
            this.reject_worker("interrupt");
            this.reject_worker = null;
        }
    }

    async run(inputs) {
        let start_time = Date.now();
        let input_value = inputs["data_in"];
        if (input_value !== undefined) {
            this.services.set_status("calculating...","info");
            if (input_value < 2) {
                throw Error(`input value ${input_value} is invalid (< 2)`);
            }
            let prime_factors = await this.services.get_configuration().get_prime_factors(input_value);
            if (prime_factors === null) {
                try {
                    prime_factors = await this.find_prime_factors(input_value);
                } catch(e) {
                }
                this.worker = null;
            }
            this.elapsed_time_ms = Date.now() - start_time;

            if (prime_factors !== null) {
                this.services.set_status(`${this.elapsed_time_ms} ms`,"info");
                this.services.get_configuration().set_prime_factors(input_value, prime_factors);
                return {"data_out": prime_factors};
            } else {
                this.services.set_status("Failed to calculate prime factors","error");
                throw Error("prime factors error");
            }
        }
    }

    async find_prime_factors(n) {
        return await new Promise((resolve,reject) => {
            this.reject_worker = reject;
            this.worker = new Worker(
                this.services.resolve_resource("javascript/prime_factors_worker.js"),
                {
                    type: "module"
                }
            );
            this.worker.onmessage = (e) => {
                resolve(e.data);
            }
            this.worker.onerror = (e) => {
                reject("error");
            }
            this.worker.postMessage(n);
        });
    }
}

Of note here is that the run method should not block whilst the calculation of prime factors is conducted.

For very large integers, this can be a computationally demanding task. The PrimeFactorsNode will offload this computation to a web-worker (javascript) or sub-process (python):

pythonjavascript

#   Hyrrokkin - a library for building and running executable graphs
#
#   MIT License - Copyright (C) 2022-2025  Visual Topology Ltd

def is_prime(n):
    i = 2
    while i*i <= n:
        if n % i == 0:
            return False
        i += 1
    return True

def compute_factors(n):
    i = 2
    r = n
    factors = []
    if is_prime(r):
        return [r]
    while True:
        if r % i == 0:
            factors.append(i)
            r //= i
            if is_prime(r):
                break
        else:
            i += 1
    if r > 1:
        factors.append(r)
    return factors

if __name__ == '__main__':
    import sys
    import json
    n = int(sys.argv[1])
    factors = compute_factors(n)
    print(json.dumps(factors))

//   Hyrrokkin - a library for building and running executable graphs
//
//   MIT License - Copyright (C) 2022-2025  Visual Topology Ltd

function is_prime(n) {
    for (let i = 2n; i*i <= n; i++) {
        if (n % i == 0n) {
            return false;
        }
    }
    return true;
}

function get_factors(for_number) {
    let i = 2n;
    let factors = [];
    let r = BigInt(for_number);
    if (is_prime(r)) {
        return [r];
    }
    while (true) {
        if (r % i == 0n) {
            factors.push(i);
            r = r / i;
            if (is_prime(r)) {
                break;
            }
        } else {
            i += 1n;
        }
    }
    if (r > 1n) {
        factors.push(r);
    }
    return factors;
}

self.onmessage = (e) => {
    let factors = get_factors(e.data);
    postMessage(factors);
}

NumberDisplayNode

The NumberDisplayNode implements some code to collect input values and report them to any connected clients.

pythonjavascript

#   Hyrrokkin - a library for building and running executable graphs
#
#   MIT License - Copyright (C) 2022-2025  Visual Topology Ltd

import json

from hyrrokkin_engine.node_interface import NodeInterface

class NumberDisplayNode(NodeInterface):

    def __init__(self, services):
        self.services = services
        self.clients = {}
        self.input_values = []

    def update_status(self):
        self.services.set_status(f"{len(self.input_values)} list(s)")

    def reset_run(self):
        self.input_values = []
        self.update_status()
        for client_service in self.clients.values():
            client_service.send_message(self.input_values)

    async def run(self, inputs):
        self.input_values = []
        for input_value in inputs.get("data_in",[]):
            self.input_values.append(list(map(lambda n: str(n),input_value)))
        self.update_status()
        for (id,client_service) in self.clients.items():
            client_service.send_message(self.input_values)
        self.services.request_open_client("default")

    def open_client(self, session_id, client_name, client_options, client_service):
        self.clients[(session_id,client_name)] = client_service
        client_service.send_message(self.input_values)

    def close_client(self, session_id, client_name):
        del self.clients[(session_id, client_name)]

//   Hyrrokkin - a library for building and running executable graphs
//
//   MIT License - Copyright (C) 2022-2025  Visual Topology Ltd

var numbergraph = numbergraph || {};

numbergraph.NumberDisplayNode = class {

    constructor(services) {
        this.services = services;
        this.clients = {};
        this.input_values = [];
    }

    update_status() {
        this.services.set_status(`${this.input_values.length} list(s)`,"info")
    }

    reset_run() {
        this.input_values = [];
        this.update_status()
        for(let id in this.clients) {
            this.clients[id].send_message(this.input_values);
        }
    }

    run(inputs) {
        if ("data_in" in inputs) {
            this.input_values = [];
            for(let idx=0; idx<inputs["data_in"].length; idx++) {
                // convert numeric values to strings
                this.input_values.push(inputs["data_in"][idx].map(n => n.toString()));
            }
        } else {
            this.input_values = [];
        }
        this.update_status();
        for(let id in this.clients) {
            this.clients[id].send_message(this.input_values);
        }
        this.services.request_open_client("default");
    }

    open_client(session_id, client_id, client_options, client_service) {
        this.clients[session_id+":"+client_id] = client_service;
        client_service.send_message(this.input_values);
    }

    close_client(session_id, client_id) {
        delete this.clients[session_id+":"+client_id];
    }

    close() {
    }
}

Node lifecycle - the load, reset_run, run and close methods.

When a topology is loaded, or when any upstream node in the topology is re-run, the node will be constructed and its load method, if implemented, will be called.

As the topology is executed, the node's inputs will be collected and its run method will be called. But, before this happens, the node's reset_run method will be called, if it is implemented. A node can implement this method to inform any clients that the node's current results are invalid and the node will soon be re-run.

The reset_run method is called as soon as the framework is aware that the node's run method will need to be called.

A node may implement a remove method to receive notifications when the node is removed from a topology

The configuration is then accessed by nodes via the get_configuration service method.

For more details on the methods that a node or configuration can implement, see

• Node abstract base class API documentation
• Configuration abstract base class API documentation

For more details on the services API passed to node or configuration constructors, see:

• Node services API documentation
• Configuration API documentation

Creating, loading and running topologies using the Hyrrokkin API

Hyrrokkin provides a Python API for creating, running and loading and saving topologies

from hyrrokkin.api.topology import Topology

# provide the resource path to the package containing the schema file
numbergraph_package = "hyrrokkin.example_packages.numbergraph"

t = Topology(execution_folder=tempfile.mkdtemp(),package_list=[numbergraph_package])

t.add_node("n0", "numbergraph:number_input_node", properties={"value": 99})
t.add_node("n1", "numbergraph:prime_factors_node")
t.add_node("n2", "numbergraph:number_display_node")

t.add_link("l0", "n0", "data_out", "n1", "data_in")
t.add_link("l1", "n1", "data_out", "n2", "data_in")
runner = t.create_runner()
runner.run()

The same topology can be expressed using a YAML file

metadata:
  name: test topology
nodes:
  n0:
    type: numbergraph:number_input_node
    properties:
      value: 99
  n1:
    type: numbergraph:prime_factors_node
  n2:
    type: numbergraph:number_display_node
links:
- n0:data_out => n1:data_in
- n1:data_out => n2:data_in

This YAML file can then be imported using the following API calls

from hyrrokkin.api.topology import Topology
from hyrrokkin.utils.yaml_importer import import_from_yaml

# provide the resource path to the package containing the schema file
numbergraph_package = "hyrrokkin.example_packages.numbergraph"

t = Topology(execution_folder=tempfile.mkdtemp(),package_list=[numbergraph_package])
with open("topology.yaml") as f:
    import_from_yaml(t,f)

Note that in the links section of the YAML file, where nodes have only one input or output port, the port name can be omitted in the links section:

metadata:
  name: test topology
configuration:
  ...
nodes:
  ...
links:
- n0 => n1
- n1 => n2

Saving and loading topologies

A topology including its properties and data can be saved to and loaded from a serialised zip format file, using the following API calls. Saving first:

from hyrrokkin.api.topology import Topology

# provide the resource path to the package containing the schema file
numbergraph_package = "hyrrokkin.example_packages.numbergraph"

t = Topology(execution_folder=tempfile.mkdtemp(),package_list=[numbergraph_package])
t.add_node("n0", "numbergraph:number_input_node", properties={"value": 99})
t.add_node("n1", "numbergraph:prime_factors_node")
t.add_node("n2", "numbergraph:number_display_node")

t.add_link("l0", "n0", "data_out", "n1", "data_in")
t.add_link("l1", "n1", "data_out", "n2", "data_in")

with open("topology.zip","wb") as f:
    t.save_zip(f)

To load from a saved topology:

from hyrrokkin.api.topology import Topology

# provide the resource path to the package containing the schema file
numbergraph_package = "hyrrokkin.example_packages.numbergraph"

t = Topology(execution_folder=tempfile.mkdtemp(),package_list=[numbergraph_package])
with open("topology.zip","rb") as f:
    t.load_zip(f)

A utility function is also provided to export a topology to YAML format. Note that the exported YAML file contains node and configuration properties but does not contain node and configuration data.

from hyrrokkin.api.topology import Topology
from hyrrokkin.utils.yaml_exporter import export_to_yaml

# provide the resource path to the package containing the schema file
numbergraph_package = "hyrrokkin.example_packages.numbergraph"

t = Topology(execution_folder=tempfile.mkdtemp(),package_list=[numbergraph_package])
with open("topology.zip","rb") as f:
    t.load(f)
with open("topology.yaml","w") as f:
    export_to_yaml(t,f)

For full details on the topology API, see:

• Topology API documentation

loading, saving and running topologies using topology_runner CLI

The Hyrrokkin package will install a hyrrokkin CLI command. Some typical usages include:

Import a topology from zip and run it:

hyrrokkin --packages hyrrokkin.example_packages.numbergraph \      
                --execution-folder /tmp/execution_test \
                --import-path topology.zip --run

Import a topology from yaml, run it and save the topology (including data) to a zip file:

hyrrokkin --packages hyrrokkin.example_packages.numbergraph \
                --execution-folder /tmp/execution_test \
                --import-path topology.yaml \
                --run --export-path topology.zip

Convert a topology from zip format to yaml format, but do not run it:

hyrrokkin --packages hyrrokkin.example_packages.numbergraph \
                --execution-folder /tmp/execution_test \
                --import-path topology.zip \
                --export-path topology.yaml

Using the Hyrrokkin expression parser

Often nodes need to work with string-based expressions, for example:

r * sin(theta)

Hyrrokkin provides a simple expression based parser which can be set up to parse simple string based expressions into a parse tree.

from hyrrokkin_engine.expr_parser import ExpressionParser
import json

ep = ExpressionParser()
ep.add_binary_operator("*",1)
print(json.dumps(ep.parse("10 * sin(pi)"),indent=2))

This program will print:

{
   "operator": "*",
   "args": [
     {
        "literal": 10
     },
     {
       "function": "sin",
       "args": [
         {
           "name": "pi"
         }
      ]
    }
  ]
}

Parser limitations

• unary and binary operators must be explicity registered with the parser
• unary operators have higher precedence than binary operators
• binary operators must be registered with a precedence