opensource/SpiffWorkflow
1
0
Fork 0
mirror of https://github.com/sartography/SpiffWorkflow.git synced 2026-03-13 08:00:46 +08:00
Code Issues Packages Projects Releases Wiki Activity

make documentation more developer-friendly

Browse Source
This commit is contained in:
Elizabeth Esswein
2024-02-07 12:57:40 -05:00
parent 14d081bfea
commit 0457022cd0
68 changed files with 1841 additions and 2384 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
doc/bpmn/Makefile
View File

@@ -1,25 +0,0 @@
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = _build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
.PHONY: apidoc
apidoc:
sphinx-apidoc -d5 -Mefo . ../venv/lib/python3.7/site-packages/SpiffWorkflow
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

492
doc/bpmn/advanced.rst
View File

@@ -1,492 +0,0 @@
A More In-Depth Look at Some of SpiffWorkflow's Features
========================================================
BPMN Task Specs
---------------
BPMN Tasks inherit quite a few attributes from :code:`SpiffWorkflow.specs.base.TaskSpec`, but only a few are used.
* `name`: the unique id of the TaskSpec, and it will correspond to the BPMN ID if that is present
* `description`: we use this attribute to provide a description of the BPMN type (the text that appears here can be overridden in the parser)
* `inputs`: a list of TaskSpec `names` that are parents of this TaskSpec
* `outputs`: a list of TaskSpec `names` that are children of this TaskSpec
* `manual`: :code:`True` if human input is required to complete tasks associated with this TaskSpec
BPMN Tasks have the following additional attributes.
* `bpmn_id`: the ID of the BPMN Task (this will be :code:`None` if the task is not visible on the diagram)
* `bpmn_name`: the BPMN name of the Task
* `lane`: the lane of the BPMN Task
* `documentation`: the contents of the BPMN `documentation` element for the Task
* `data_input_associations`: a list of incoming data object references
* `data_output_associtions`: a list of outgoing data object references
* `io_specification`: the BPMN IO specification of the Task
Filtering Tasks
---------------
Tasks by Lane
^^^^^^^^^^^^^
The :code:`workflow.get_ready_user_tasks` method optionally takes the argument `lane`, which can be used to
restrict the tasks returned to only tasks in that lane.
.. code:: python
ready_tasks = workflow.get_ready_user_tasks(lane='Customer')
will return only tasks in the 'Customer' lane in our example workflow.
Tasks by Spec Name
^^^^^^^^^^^^^^^^^^
To retrieve a list of tasks associated with a particular task spec, use :code:`workflow.get_tasks_from_spec_name`
.. code:: python
tasks = workflow.get_tasks_from_spec_name('customize_product')
will return a list containing the Call Actitivities for the customization of a product in our example workflow.
.. note::
The `name` paramter here refers to the task spec name, not the BPMN name (for visible tasks, this will
be the same as the `bpmn_id`)
Tasks by State
^^^^^^^^^^^^^^
We need to import the :code:`TaskState` object (unless you want to memorize which numbers correspond to which states).
.. code:: python
from SpiffWorkflow.util.task import TaskState
tasks = workflow.get_tasks(TaskState.COMPLETED)
will return a list of completed tasks.
See :doc:`../concepts` for more information about task states.
Tasks in a Subprocess or Call Activity
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The :code:`BpmnWorkflow` class maintains a dictionary of subprocesses (the key is the `id` of the Call Activity or
Subprocess Task). :code:`workflow.get_tasks` will start at the top level workflow and recurse through the subprocesses
to create a list of all tasks. It is also possible to start from a particular subprocess:
.. code:: python
tasks = workflow.get_tasks_from_spec_name('customize_product')
subprocess = workflow.get_subprocess(tasks[0])
subprocess_tasks = workflow.get_tasks(workflow=subprocess)
will limit the list of returned tasks to only those in the first product customization.
.. note::
Each :code:`Task` object has a reference to its workflow; so with a Task inside a subprocess, we can call
:code:`workflow.get_tasks(workflow=task.workflow)` to start from our current workflow.
Logging
-------
Spiff provides several loggers:
- the :code:`spiff` logger, which emits messages when a workflow is initialized and when tasks change state
- the :code:`spiff.metrics` logger, which emits messages containing the elapsed duration of tasks
- the :code:`spiff.data` logger, which emits a message when :code:`task.update_data` is called or workflow data is retrieved or set.
Log level :code:`INFO` will provide reasonably detailed information about state changes.
As usual, log level :code:`DEBUG` will probably provide more logs than you really want
to see, but the logs will contain the task and task internal data.
Data can be included at any level less than :code:`INFO`. In our example application,
we define a custom log level
.. code:: python
logging.addLevelName(15, 'DATA')
so that we can see the task data in the logs without fully enabling debugging.
The workflow runners take an `-l` argument that can be used to specify the logging level used when running the example workflows.
We'll write the logs to a file called `data.log` instead of the console to avoid printing very long messages during the workflow.
Our logging configuration code can be found in `runner/shared.py`. Most of the code is about logging
configuration in Python rather than anything specific to SpiffWorkflow, so we won't go over it in depth.
Parsing
-------
Each of the BPMN pacakges (:code:`bpmn`, :code:`spiff`, or :code:`camunda`) has a parser that is preconfigured with
the specs in that package (if a particular TaskSpec is not implemented in the package, :code:`bpmn` TaskSpec is used).
See the example in :doc:`synthesis` for the basics of creating a parser. The parser can optionally be initialized with
- a set of namespaces (useful if you have custom extensions)
- a BPMN Validator (the one in the :code:`bpmn` package validates against the BPMN 2.0 spec)
- a mapping of XML tag to Task Spec Descriptions. The default set of descriptions can be found in
:code:`SpiffWorkflow.bpmn.parser.spec_descriptions`. These values will be added to the Task Spec in the `description` attribute
and are intended as a user-friendly description of what the task is.
The :code:`BpmnValidator` can be used and extended independently of the parser as well; call :code:`validate` with
an :code:`lxml` parsed tree.
Loading BPMN Files
^^^^^^^^^^^^^^^^^^
In addition to :code:`load_bpmn_file`, there are similar functions :code:`load_bpmn_str` which can load the XML from a string, and
:code:`load_bpmn_io`, which can load XML from any object implementing the IO interface, and :code:`add_bpmn_xml`, which can load
BPMN specs from an :code:`lxml` parsed tree.
Dependencies
^^^^^^^^^^^^
The following methods are available for discovering the names of processes and DMN files that may be defined externally:
- :code:`get_subprocess_specs`: Returns a mapping of name -> :code:`BpmnWorkflowSpec` for any Call Activities referenced by the
provided spec (searches recursively)
- :code:`find_all_spec`: Returns a mapping of name -> :code:`BpmnWorkflowSpec` for all processes used in all files that have been
provided to the parser at that point.
- :code:`get_process_dependences`: Returns a list of process IDs referenced by the provided process ID
- :code:`get_dmn_dependencies`: Returns a list of DMN IDs referenced by the provided process ID
Serialization
-------------
The :code:`BpmnWorkflowSerializer` has two components
* the `workflow_spec_converter` (which handles serialization of objects that SpiffWorkflow knows about)
* the `data_converter` (which handles serialization of custom objects)
Unless you have overriden any of TaskSpecs with custom specs, you should be able to use the serializer
configuration from the package you are importing the parser from (:code:`bpmn`, :code:`spiff`, or :code:`camunda`).
See :doc:`synthesis` for an example.
Serializing Custom Objects
^^^^^^^^^^^^^^^^^^^^^^^^^^
In `Custom Script Engines`_ , we add some custom methods and objects to our scripting environment. We create a simple
class (a :code:`namedtuple`) that holds the product information for each product.
We'd like to be able to save and restore our custom object.
.. code:: python
ProductInfo = namedtuple('ProductInfo', ['color', 'size', 'style', 'price'])
def product_info_to_dict(obj):
return {
'color': obj.color,
'size': obj.size,
'style': obj.style,
'price': obj.price,
}
def product_info_from_dict(dct):
return ProductInfo(**dct)
registry = DictionaryConverter()
registry.register(ProductInfo, product_info_to_dict, product_info_from_dict)
Here we define two functions, one for turning our object into a dictionary of serializable objects, and one for recreating
the object from the dictionary representation we created.
We initialize a :code:`DictionaryConverter` and `register` the class and methods.
Registering an object sets up relationships between the class and the serialization and deserialization methods. We go
over how this works in a little more detail in `Custom Serialization in More Depth`_.
It is also possible to bypass using a :code:`DictionaryConverter` at all for the data serialization process (but not for
the spec serialization process). The only requirement for the the `data_converter` is that it implement the methods
- `convert`, which takes an object and returns something JSON-serializable
- `restore`, which takes a serialized version and returns an object
Serialization Versions
^^^^^^^^^^^^^^^^^^^^^^
As we make changes to Spiff, we may change the serialization format. For example, in 1.2.1, we changed
how subprocesses were handled interally in BPMN workflows and updated how they are serialized and we upraded the
serializer version to 1.1.
As we release SpiffWorkflow 2.0, there are several more substantial changes, and we'll upgrade the serializer version to 1.2.
Since workflows can contain arbitrary data, and even SpiffWorkflow's internal classes are designed to be customized in ways
that might require special serialization and deserialization, it is possible to override the default version number, to
provide users with a way of tracking their own changes. This can be accomplished by setting the `VERSION` attribute on
the :code:`BpmnWorkflowSerializer` class.
If you have not provided a custom version number, SpiffWorkflow wil attempt to migrate your workflows from one version
to the next if they were serialized in an earlier format.
If you've overridden the serializer version, you may need to incorporate our serialization changes with
your own. You can find our conversions in
`SpiffWorkflow/bpmn/serilaizer/migrations <https://github.com/sartography/SpiffWorkflow/tree/main/SpiffWorkflow/bpmn/serializer/migration>`_
These are broken up into functions that handle each individual change, which will hopefully make it easier to incoporate them
into your upgrade process, and also provides some documentation on what has changed.
Custom Serialization in More Depth
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Both of the serializer components mentioned in `Serialization`_ are based on the :code:`DictionaryConverter`. Let's import
it and create one and register a type:
.. code:: python
from datetime import datetime
from SpiffWorkflow.bpmn.serializer.helpers.dictionary import DictionaryConverter
registry = DictionaryConverter()
registry.register(
datetime.
lambda dt: {'value': dt.isoformat() },
lambda dct: datetime.fromisoformat(dct['value'])
)
The arguemnts to :code:`register` are:
* `cls`: the class to be converted
* `to_dict`: a function that returns a dictionary containing JSON-serializable objects
* `from_dict`: a function that take the output of `to_dict` and restores the original object
When the :code:`register` method is called, a `typename` is created and maps are set up between `cls` and `to_dict`
function, `cls` and `typename`, and `typename` and `from_dict`.
When :code:`registry.convert` is called on an object, the `cls` is use to retrieve the `to_dict` function and the
`typename`. The `to_dict` funciton is called on the object and the `typename` is added to the resulting dictionary.
When :code:`registry.restore` is called with a dictionary, it is checked for a `typename` key, and if one exists, it
is used to retrieve the `from_dict` function and the dictionary is passed to it.
If an object is not recognized, it will be passed on as-is.
Custom Script Engines
---------------------
You may need to modify the default script engine, whether because you need to make additional
functionality available to it, or because you might want to restrict its capabilities for
security reasons.
.. warning::
By default, the scripting environment passes input directly to :code:`eval` and :code:`exec`! In most
cases, you'll want to replace the default scripting environment with one of your own.
Files referenced in this section:
* `script_engine.py <https://github.com/sartography/spiff-example-cli/blob/main/runner/script_engine.py>`_
* `product_info.py <https://github.com/sartography/spiff-example-cli/blob/main/runner/product_info.py>`_
* `subprocess.py <https://github.com/sartography/spiff-example-cli/blob/main/runner/subprocess.py>`_
* `spiff-bpmn-runner.py <https://github.com/sartography/spiff-example-cli/blob/main/spiff-bpmn-runner.py>`_
The following example replaces the default global enviroment with the one provided by
`RestrictedPython <https://restrictedpython.readthedocs.io/en/latest/>`_
.. code:: python
from RestrictedPython import safe_globals
from SpiffWorkflow.bpmn.PythonScriptEngine import PythonScriptEngine
from SpiffWorkflow.bpmn.PythonScriptEngineEnvironment import TaskDataEnvironment
restricted_env = TaskDataEnvironment(safe_globals)
restricted_script_engine = PythonScriptEngine(environment=restricted_env)
Another reason you might want to customize the scripting environment is to provide access to custom
classes or functions.
In our example models so far, we've been using DMN tables to obtain product information. DMN
tables have a **lot** of uses so we wanted to feature them prominently, but in a simple way.
If a customer was selecting a product, we would surely have information about how the product
could be customized in a database somewhere. We would not hard code product information in
our diagram (although it is much easier to modify the BPMN diagram than to change the code
itself!). Our shipping costs would not be static, but would depend on the size of the order and
where it was being shipped -- maybe we'd query an API provided by our shipper.
SpiffWorkflow is obviously **not** going to know how to query **your** database or make API calls to
**your** vendors. However, one way of making this functionality available inside your diagram is to
implement the calls in functions and add those functions to the scripting environment, where they
can be called by Script Tasks.
We are not going to actually include a database or API and write code for connecting to and querying
it, but since we only have 7 products we can model our database with a simple dictionary lookup
and just return the same static info for shipping for the purposes of the tutorial.
We'll define some resources in `product_info.py`
.. code:: python
from collections import namedtuple
ProductInfo = namedtuple('ProductInfo', ['color', 'size', 'style', 'price'])
INVENTORY = {
'product_a': ProductInfo(False, False, False, 15.00),
'product_b': ProductInfo(False, False, False, 15.00),
'product_c': ProductInfo(True, False, False, 25.00),
'product_d': ProductInfo(True, True, False, 20.00),
'product_e': ProductInfo(True, True, True, 25.00),
'product_f': ProductInfo(True, True, True, 30.00),
'product_g': ProductInfo(False, False, True, 25.00),
}
def lookup_product_info(product_name):
return INVENTORY[product_name]
def lookup_shipping_cost(shipping_method):
return 25.00 if shipping_method == 'Overnight' else 5.00
We'll add these functions to our scripting environment in `script_engine.py`
.. code:: python
env_globals = {
'lookup_product_info': lookup_product_info,
'lookup_shipping_cost': lookup_shipping_cost,
'datetime': datetime,
}
custom_env = TaskDataEnvironment(env_globals)
custom_script_engine = PythonScriptEngine(environment=custom_env)
.. note::
We're also adding :code:`datetime`, because we added the timestamp to the payload of our message when we
set up the Message Event (see :doc:`events`)
When we initialize the runner in `spiff-bpmn-runner.py`, we'll import and use `custom_script_engine` as our
script engine.
We can use the custom functions in script tasks like any normal function. We've replaced the Business Rule
Task that determines product price with a script that simply checks the `price` field on our product.
.. figure:: figures/script_engine/top_level.png
:scale: 30%
:align: center
Top Level Workflow with Custom Script Engine
And we can simplify the gateways in our 'Call Activity' flows as well now too:
.. figure:: figures/script_engine/call_activity.png
:scale: 30%
:align: center
Call Activity with Custom Script Engine
To run this workflow (you'll have to manually change which script engine you import):
.. code-block:: console
./spiff-bpmn-runner.py -p order_product -b bpmn/tutorial/top_level_script.bpmn bpmn/tutorial/call_activity_script.bpmn
Another reason to customize the scripting enviroment is to allow it to run completely separately from
SpiffWorkflow. You might wish to do this for performance or security reasons.
In our example repo, we've created a simple command line script in `runner/subprocess.py` that takes serialized global
and local environments and a script or expression to execute or evaluate. In `runner/script_engine.py`, we create
a scripting environment that runs the current :code:`execute` or :code:`evaluate` request in a subprocess with this
script. We've imported our custom methods into `subprocess.py` so they are automatically available when it is used.
This example is needlessly complex for the work we're doing in this case, but the point of the example is to demonstrate
that this could be a Docker container with a complex environment, an HTTP API running somewhere else entirely.
.. note::
Note that our execute method returns :code:`True`. We could check the status of our process here and return
:code:`False` to force our task into an `ERROR` state if the task failed to execute.
We could also return :code:`None`
if the task is not finished; this will cause the task to go into the `STARTED` state. You would have to manually
complete a task that has been `STARTED`. The purpose of the state is to tell SpiffWorkflow your application will
handle monitoring and updating this task and other branches that do not depend on this task may proceed. It is
intended to be used with potentially long-running tasks.
See :doc:`../concepts` for more information about Task States and Workflow execution.
Service Tasks
-------------
Service Tasks are also executed by the workflow's script engine, but through a different method, with the help of some
custom extensions in the :code:`spiff` package:
- `operation_name`, the name assigned to the service being called
- `operation_params`, the parameters the operation requires
This is our script engine and scripting environment:
.. code:: python
service_task_env = TaskDataEnvironment({
'product_info_from_dict': product_info_from_dict,
'datetime': datetime,
})
class ServiceTaskEngine(PythonScriptEngine):
def __init__(self):
super().__init__(environment=service_task_env)
def call_service(self, operation_name, operation_params, task_data):
if operation_name == 'lookup_product_info':
product_info = lookup_product_info(operation_params['product_name']['value'])
result = product_info_to_dict(product_info)
elif operation_name == 'lookup_shipping_cost':
result = lookup_shipping_cost(operation_params['shipping_method']['value'])
else:
raise Exception("Unknown Service!")
return json.dumps(result)
service_task_engine = ServiceTaskEngine()
Instead of adding our custom functions to the enviroment, we'll override :code:`call_service` and call them directly
according to the `operation_name` that was given. The :code:`spiff` Service Task also evaluates the parameters
against the task data for us, so we can pass those in directly. The Service Task will also store our result in
a user-specified variable.
We need to send the result back as json, so we'll reuse the functions we wrote for the serializer.
The Service Task will assign the dictionary as the operation result, so we'll add a `postScript` to the Service Task
that retrieves the product information that creates a :code:`ProductInfo` instance from the dictionary, so we need to
import that too.
The XML for the Service Task looks like this:
.. code:: xml
<bpmn:serviceTask id="Activity_1ln3xkw" name="Lookup Product Info">
<bpmn:extensionElements>
<spiffworkflow:serviceTaskOperator id="lookup_product_info" resultVariable="product_info">
<spiffworkflow:parameters>
<spiffworkflow:parameter id="product_name" type="str" value="product_name"/>
</spiffworkflow:parameters>
</spiffworkflow:serviceTaskOperator>
<spiffworkflow:postScript>product_info = product_info_from_dict(product_info)</spiffworkflow:postScript>
</bpmn:extensionElements>
<bpmn:incoming>Flow_104dmrv</bpmn:incoming>
<bpmn:outgoing>Flow_06k811b</bpmn:outgoing>
</bpmn:serviceTask>
Getting this information into the XML is a little bit beyond the scope of this tutorial, as it involves more than
just SpiffWorkflow. I hand edited it for this case, but you can hardly ask your BPMN authors to do that!
Our `modeler <https://github.com/sartography/bpmn-js-spiffworkflow>`_ has a means of providing a list of services and
their parameters that can be displayed to a BPMN author in the Service Task configuration panel. There is an example of
hard-coding a list of services in
`app.js <https://github.com/sartography/bpmn-js-spiffworkflow/blob/0a9db509a0e85aa7adecc8301d8fbca9db75ac7c/app/app.js#L47>`_
and as suggested, it would be reasonably straightforward to replace this with a API call. `SpiffArena <https://www.spiffworkflow.org/posts/articles/get_started/>`_
has robust mechanisms for handling this that might serve as a model for you.
How this all works is obviously heavily dependent on your application, so we won't go into further detail here, except
to give you a bare bones starting point for implementing something yourself that meets your own needs.
To run this workflow (you'll have to manually change which script engine you import):
.. code-block:: console
./spiff-bpmn-runner.py -p order_product -b bpmn/tutorial/top_level_service_task.bpmn bpmn/tutorial/call_activity_service_task.bpmn

163
doc/bpmn/application.rst Normal file
View File

@@ -0,0 +1,163 @@
Overview
========
This section focuses on the example application rather than the library itself; it is intended to orient people
attempting to use this documentation, so we won't devote that much space to it (much of it is necessary for a
functioning app, but not directly relevant to library use); nonetheless, there is quite a bit of code and a general
idea of what's here will be helpful.
The application has several parts:
- an engine, which uses SpiffWorkflow to run, parse, and serialize workflows
- a curses UI for running and examining Workflows, which uses the engine
- a command line UI with some limited functionality, which also uses the engine
We'll mainly focus on the engine, as it contains the interface with the library, though a few examples will come from
the other components. The engine is quite small and simple compared to the code required to handle user input and
display information in a terminal.
Configuration is set up in a python module and passed into the application with the `-e` argument, which loads the
configured engine from this file. This setup should make it relatively to change the behavior of engine. The
following configurations are included:
- :code:`spiff_example.spiff.file`: uses spiff BPMN extensions and serializes to JSON files
- :code:`spiff_example.spiff.sqlite`: uses spiff BPMN extensions and serializes to SQLite
- :code:`spiff_example.camunda.default`: uses Camunda extensions and serializes to SQLite
.. _quickstart:
Quickstart
==========
There are several versions of a product ordering process of variying complexity located in the
:example:`bpmn/tutorial` directory of the repo which contain most of the elements that SpiffWorkflow supports. These
diagrams can be viewed in any BPMN editor, but many of them have custom extensions created with
`bpmn-js-spiffworflow <https://github.com/sartography/bpmn-js-spiffworkflow>`_.
To add a workflow via the command line and store serialized specs in JSON files:
.. code-block:: console
./runner.py -e spiff_example.spiff.file add \
-p order_product \
-b bpmn/tutorial/{top_level,call_activity}.bpmn \
-d bpmn/tutorial/{product_prices,shipping_costs}.dmn
To run the curses application using serialized JSON files:
.. code-block:: console
./runner.py -e spiff_example.spiff.file
Select the 'Start Workflow' screen and start the process.
The Application in Slightly More Depth
======================================
The application requires the name of a module to load that contains a configuration such as one of those defined above.
To start the curses UI using the JSON file serializer:
.. code-block:: console
./runner.py -e spiff_example.spiff.file
If the application is run with no other arguments, the curses UI will be loaded.
It is possible to add a workflow spec through the curses UI, but it is going to be somewhat painful to do so unless
you are a better typist and proofreader than I; therefore, there are also a few command line utilities for handling
some of the functionality, including adding workflow specs.
Command line options are
- :code:`add` to add a workflow spec (while taking advantage of your shell's file completion functionality)
- :code:`list` to list the available specs
- :code:`run` to run a workflow non-interactively
Each of these options has a help menu that describes how to use them.
Configuration Modules
=====================
The three main ways that users can customize the library are:
- the parser
- the script engine
- the serializer
We use the configuration module to allow these components to be defined outside the workflow engine itself and passed
in as parameters to make it easier to experiment. I am somewhat regularly asked questions about why a diagram doesn't
executed as expected, or how to get the script engine to work a particular way; this is a first pass at setting
something up that works better for me than configuring the library's test loader and running that in a debugger; I hope
other people will find it useful as well.
We'll go through the configuration in greater detail in later sections, but we'll take a brief look at the simplest
configuration, :app:`spiff/file.py` here.
In this file, we'll initialize our parser:
.. code-block:: python
parser = SpiffBpmnParser()
We don't need to further customize this parser -- this is a builtin parser that can handle DMN files as well as Spiff
BPMN extensions.
We also need to initialize a serializer:
.. code-block:: python
dirname = 'wfdata'
FileSerializer.initialize(dirname)
registry = FileSerializer.configure(SPIFF_CONFIG)
serializer = FileSerializer(dirname, registry=registry)
JSON specs and workflows will be stored in :code:`wfdata`. The :code:`registry` is the place where information about
converting Python objects to and from JSON-serializable dictionary form is maintained. :code:`SPIFF_CONFIG` tells the
serializer how to handle objects used internally by Spiff. Workflows can also contain arbitrary data, so this registry
can also tell the serializer how to handle any non-serializable data in your workflow. We'll go over this in more
detail in :ref:`serializing_custom_objects`.
We initialize a scripting enviroment:
.. code-block:: python
script_env = TaskDataEnvironment({'datetime': datetime })
>script_engine = PythonScriptEngine(script_env)
The :code:`PythonScriptEngine` handles execution of script tasks and evaluation of gateway and DMN conditions.
We'll create the script engine based on it; execution and evaluation will occur in the context of this enviroment.
SpiffWorkflow provides a default scripting environment that is suitable for simple applications, but a serious
application will probably need to extend (or restrict) it in some way. See :doc:`script_engine` for a few examples.
Therefore, we have the ability to optionally pass one in.
In this case, we'll include access to the :code:`datetime` module, because we'll use it in several of our script tasks.
We also specify some handlers:
.. code-block:: python
handlers = {
UserTask: UserTaskHandler,
ManualTask: ManualTaskHandler,
NoneTask: ManualTaskHandler,
}
This is a mapping of task spec to task handler and lets our application know how to handle these tasks.
.. note::
In our application, we're also passing in handlers, but this is not a typical use case. The library knows how to
handle all task types except for human (User and Manual) tasks, and those handlers would typically be built into
your application. However, this application needs to be able to deal with more than one set of human task specs,
and this is a convenient way to do this. The library treats None tasks (tasks with no specific type assigned)
like Manual Tasks by default.
We then create our BPMN engine (:app:`engine/engine.py`) using each of these components:
.. code-block:: python
from ..engine import BpmnEngine
engine = BpmnEngine(parser, serializer, handlers, script_env)

45
doc/bpmn/camunda/events.rst
View File

@@ -1,45 +0,0 @@
Events
======
Message Events
--------------
Configuring Message Events
^^^^^^^^^^^^^^^^^^^^^^^^^^
.. figure:: figures/throw_message_event.png
:scale: 60%
:align: center
Throw Message Event configuration
.. figure:: figures/message_start_event.png
:scale: 60%
:align: center
Message Catch Event configuration
The Throw Message Event Implementation should be 'Expression' and the Expression should
be a Python statement that can be evaluated. In this example, we'll just send the contents
of the :code:`reason_delayed` variable, which contains the response from the 'Investigate Delay'
Task.
We can provide a name for the result variable, but I have not done that here, as it does not
make sense to me for the generator of the event to tell the handler what to call the value.
If you *do* specify a result variable, the message payload (the expression evaluated in the
context of the Throwing task) will be added to the handling task's data in a variable of that
name; if you leave it blank, SpiffWorkflow will create a variable of the form <Handling
Task Name>_Response.
Running the Model
^^^^^^^^^^^^^^^^^
If you have set up our example repository, this model can be run with the
following command:
.. code-block:: console
./camunda-bpmn-runner.py -c order_collaboration \
-d bpmn/tutorial/product_prices.dmn bpmn/tutorial/shipping_costs.dmn \
-b bpmn/camunda/events.bpmn bpmn/camunda/call_activity.bpmn

131
doc/bpmn/camunda/index.rst Normal file
View File

@@ -0,0 +1,131 @@
Using the Camunda Configuration Module
======================================
.. warning:: There is a better way ...
SpiffWorkflow does not aim to support all of Camunda's proprietary extensions.
Many of of the items in the Camunda Properties Panel do not work. And
major features of SpiffWorkflow (Messages, Data Objects, Service Tasks, Pre-Scripts, etc...)
can not be configured in the Camunda editor. Use `SpiffArena <https://www.spiffworkflow.org/posts/articles/get_started/>`_
to build and test your BPMN models instead!
Earlier users of SpiffWorkflow relied heavily on Camunda's modeler and several of our task spec
implementations were based on Camunda's extensions. Support for these extensions has been moved
to the :code:`camunda` package. We are not actively maintaining this package (though we will
accept contributions from Camunda users!). Please be aware that many of the Camunda extensions
that will appear in the Camunda editor do not work with SpiffWorkflow.
In this repo, we provide the following configuration:
.. code-block:: console
./runner.py -e spiff_example.camunda.sqlite
Tasks
=====
User Tasks
----------
Creating a User Task
^^^^^^^^^^^^^^^^^^^^
When you click on a user task in the BPMN modeler, the Properties Panel includes a form tab. Use this
tab to build your questions.
The following example shows how a form might be set up in Camumda.
.. figure:: figures/user_task.png
:scale: 30%
:align: center
User Task configuration
Manual Tasks
------------
Creating a Manual Task
^^^^^^^^^^^^^^^^^^^^^^
We can use the BPMN element Documentation field to display more information about the context of the item.
Spiff is set up in a way that you could use any templating library you want, but we have used
`Jinja <https://jinja.palletsprojects.com/en/3.0.x/>`_.
In this example, we'll present an order summary to our customer.
.. figure:: figures/documentation.png
:scale: 30%
:align: center
Element Documentation
Example Code
------------
Example Human task handlers can be found in :app:`camunda/curses_handlers.py`.
Events
======
Message Events
--------------
Configuring Message Events
^^^^^^^^^^^^^^^^^^^^^^^^^^
.. figure:: figures/throw_message_event.png
:scale: 60%
:align: center
Throw Message Event configuration
.. figure:: figures/message_start_event.png
:scale: 60%
:align: center
Message Catch Event configuration
The Throw Message Event Implementation should be 'Expression' and the Expression should
be a Python statement that can be evaluated. In this example, we'll just send the contents
of the :code:`reason_delayed` variable, which contains the response from the 'Investigate Delay'
Task.
We can provide a name for the result variable, but I have not done that here, as it does not
make sense to me for the generator of the event to tell the handler what to call the value.
If you *do* specify a result variable, the message payload (the expression evaluated in the
context of the Throwing task) will be added to the handling task's data in a variable of that
name; if you leave it blank, SpiffWorkflow will create a variable of the form <Handling
Task Name>_Response.
MultiInstance Tasks
===================
Earlier versions of SpiffWorkflow relied on the properties available in the Camunda MultiInstance Panel.
.. figure:: figures/multiinstance_task_configuration.png
:scale: 60%
:align: center
MultiInstance Task configuration
SpiffWorkflow has a MultiInstance Task spec in the :code:`camunda` package that interprets these fields
in the following way:
* Loop Cardinality:
- If this is an integer, or a variable that evaluates to an integer, this number would be
used to determine the number of instances
- If this is a collection, the size of the collection would be used to determine the number of
instances
* Collection: the output collection (input collections have to be specified in the "Cardinality" field).
* Element variable: the name of the varible to copy the item into for each instance.
.. warning::
The spec in this package is based on an old version of Camunda, so the panel may have changed. The
properties might or might not have been the way Camunda used these fields, and may or may not be similar
to newer or current versions. *Use at your own risk!*

30
doc/bpmn/camunda/multiinstance.rst
View File

@@ -1,30 +0,0 @@
MultiInstance Tasks
===================
Earlier versions of SpiffWorkflow relied on the properties available in the Camunda MultiInstance Panel.
.. figure:: figures/multiinstance_task_configuration.png
:scale: 60%
:align: center
MultiInstance Task configuration
SpiffWorkflow has a MultiInstance Task spec in the :code:`camunda` package that interprets these fields
in the following way:
* Loop Cardinality:
- If this is an integer, or a variable that evaluates to an integer, this number would be
used to determine the number of instances
- If this is a collection, the size of the collection would be used to determine the number of
instances
* Collection: the output collection (input collections have to be specified in the "Cardinality" field).
* Element variable: the name of the varible to copy the item into for each instance.
.. warning::
The spec in this package is based on an old version of Camunda, which might or might not have been the
way Camunda uses these fields, and may or may not be similar to newer or current versions.
*Use at your own risk!*

23
doc/bpmn/camunda/support.rst
View File

@@ -1,23 +0,0 @@
Camunda Editor Support
======================
.. warning:: There is a better way ...
SpiffWorkflow does not aim to support all of Camunda's proprietary extensions.
Many of of the items in the Camunda Properties Panel do not work. And
major features of SpiffWorkflow (Messages, Data Objects, Service Tasks, Pre-Scripts, etc...)
can not be configured in the Camunda editor. Use `SpiffArena <https://www.spiffworkflow.org/posts/articles/get_started/>`_
to build and test your BPMN models instead!
Earlier users of SpiffWorkflow relied heavily on Camunda's modeler and several of our task spec
implementations were based on Camunda's extensions. Support for these extensions has been moved
to the :code:`camunda` package. We are not actively maintaining this package (though we will
accept contributions from Camunda users!). Please be aware that many of the Camunda extensions
that will appear in the Camunda editor do not work with SpiffWorkflow.
.. toctree::
:maxdepth: 3
tasks
events
multiinstance

104
doc/bpmn/camunda/tasks.rst
View File

@@ -1,104 +0,0 @@
Tasks
=====
User Tasks
----------
Creating a User Task
^^^^^^^^^^^^^^^^^^^^
When you click on a user task in the BPMN modeler, the Properties Panel includes a form tab. Use this
tab to build your questions.
The following example shows how a form might be set up in Camumda.
.. figure:: figures/user_task.png
:scale: 30%
:align: center
User Task configuration
Manual Tasks
------------
Creating a Manual Task
^^^^^^^^^^^^^^^^^^^^^^
We can use the BPMN element Documentation field to display more information about the context of the item.
Spiff is set up in a way that you could use any templating library you want, but we have used
`Jinja <https://jinja.palletsprojects.com/en/3.0.x/>`_.
In this example, we'll present an order summary to our customer.
.. figure:: figures/documentation.png
:scale: 30%
:align: center
Element Documentation
Running The Model
-----------------
If you have set up our example repository, this model can be run with the
following command:
.. code-block:: console
./camunda-bpmn-runner.py -p order_product -d bpmn/tutorial/product_prices.dmn -b bpmn/camunda/task_types.bpmn
Example Application Code
------------------------
Handling the User Task
^^^^^^^^^^^^^^^^^^^^^^
.. code:: python
dct = {}
for field in task.task_spec.form.fields:
if isinstance(field, EnumFormField):
option_map = dict([ (opt.name, opt.id) for opt in field.options ])
options = "(" + ', '.join(option_map) + ")"
prompt = f"{field.label} {options} "
option = input(prompt)
while option not in option_map:
print(f'Invalid selection!')
option = input(prompt)
response = option_map[option]
else:
response = input(f"{field.label} ")
if field.type == "long":
response = int(response)
update_data(dct, field.id, response)
DeepMerge.merge(task.data, dct)
The list of form fields for a task is stored in :code:`task.task_spec.form_fields`.
For Enumerated fields, we want to get the possible options and present them to the
user. The variable names of the fields were stored in :code:`field.id`, but since
we set labels for each of the fields, we'd like to display those instead, and map
the user's selection back to the variable name.
For other fields, we'll just store whatever the user enters, although in the case
where the data type was specified to be a :code:`long`, we'll convert it to a
number.
Finally, we need to explicitly store the user-provided response in a variable
with the expected name with :code:`update_data(dct, field.id, response)` and merge
the newly collected data into our task data with :code:`DeepMerge.merge(task.data, dct)`.
Our :code:`update_data` function handles "dot notation" in field names, which creates
nested dictionaries based on the path components.
.. code:: python
def update_data(dct, name, value):
path = name.split('.')
current = dct
for component in path[:-1]:
if component not in current:
current[component] = {}
current = current[component]
current[path[-1]] = value

60
doc/bpmn/conf.py
View File

@@ -1,60 +0,0 @@
# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
numfig = True
# -- Project information -----------------------------------------------------
project = 'SpiffWorkflow-BPMN Documentation'
copyright = '2020, Sartography'
author = 'Sartography'
# -- General configuration ---------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.autodoc', # 'sphinx.ext.coverage',
'sphinx.ext.viewcode',
'sphinx.ext.autosummary',
#'sphinx.ext.intersphinx',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'default'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
intersphinx_mapping = {'http://docs.python.org/': None}

70
doc/bpmn/custom_task_spec.rst
View File

@@ -11,11 +11,12 @@ First we'll create a new class
.. code:: python
from SpiffWorkflow.bpmn.specs.event_definitions import TimerEventDefinition, NoneEventDefinition
from SpiffWorkflow.bpmn.specs.mixins.events.start_event import StartEvent
from SpiffWorkflow.spiff.specs.spiff_task import SpiffBpmnTask
from SpiffWorkflow.bpmn.specs.event_definitions import NoneEventDefinition
from SpiffWorkflow.bpmn.specs.event_definitions.timer import TimerEventDefinition
from SpiffWorkflow.bpmn.specs.mixins import StartEventMixin
from SpiffWorkflow.spiff.specs import SpiffBpmnTask
class CustomStartEvent(StartEvent, SpiffBpmnTask):
class CustomStartEvent(StartEventMixin, SpiffBpmnTask):
def __init__(self, wf_spec, bpmn_id, event_definition, **kwargs):
@@ -26,13 +27,14 @@ First we'll create a new class
super().__init__(wf_spec, bpmn_id, event_definition, **kwargs)
self.timer_event = None
When we create our custom event, we'll check to see if we're creating a Start Event with a TimerEventDefinition, and if so,
we'll replace it with a NoneEventDefinition.
When we create our custom event, we'll check to see if we're creating a Start Event with a :code:`TimerEventDefinition`, and
if so, we'll replace it with a :code:`NoneEventDefinition`. There are three different types of Timer Events, so we'll use
the base class for all three to make sure we account for TimeDate, Duration, and Cycle.
.. note::
Our class inherits from two classes. We import a mixin class that defines generic BPMN Start Event behavior from
:code:`StartEvent` in the :code:`bpmn` package and the :code:`SpiffBpmnTask` from the :code:`spiff` package, which
:code:`StartEventMixin` in the :code:`bpmn` package and the :code:`SpiffBpmnTask` from the :code:`spiff` package, which
extends the default :code:`BpmnSpecMixin`.
We've split the basic behavior for specific BPMN tasks from the :code:`BpmnSpecMixin` to make it easier to extend
@@ -44,10 +46,10 @@ Whenever we create a custom task spec, we'll need to create a converter for it s
.. code:: python
from SpiffWorkflow.bpmn.serializer.workflow import BpmnWorkflowSerializer
from SpiffWorkflow.bpmn.serializer.task_spec import StartEventConverter
from SpiffWorkflow.bpmn.serializer import BpmnWorkflowSerializer
from SpiffWorkflow.bpmn.serializer.default import EventConverter
from SpiffWorkflow.spiff.serializer.task_spec import SpiffBpmnTaskConverter
from SpiffWorkflow.spiff.serializer.config import SPIFF_SPEC_CONFIG
from SpiffWorkflow.spiff.serializer import DEFAULT_CONFIG
class CustomStartEventConverter(SpiffBpmnTaskConverter):
@@ -63,21 +65,20 @@ Whenever we create a custom task spec, we'll need to create a converter for it s
return dct
SPIFF_SPEC_CONFIG['task_specs'].remove(StartEventConverter)
SPIFF_SPEC_CONFIG['task_specs'].append(CustomStartEventConverter)
wf_spec_converter = BpmnWorkflowSerializer.configure_workflow_spec_converter(SPIFF_SPEC_CONFIG)
serializer = BpmnWorkflowSerializer(wf_spec_converter)
DEFAULT_CONFIG['task_specs'].remove(StartEventConverter)
DEFAULT_CONFIG['task_specs'].append(CustomStartEventConverter)
registry = BpmnWorkflowSerializer.configure(DEFAULT_CONFIG)
serializer = BpmnWorkflowSerializer(registry)
Our converter will inherit from the :code:`SpiffBpmnTaskConverter`, since that's our base generic BPMN mixin class.
The :code:`SpiffBpmnTaskConverter` ultimately inherits from
The :code:`SpiffBpmnTaskConverter` itself inherits from
:code:`SpiffWorkflow.bpmn.serializer.helpers.task_spec.BpmnTaskSpecConverter`. which provides some helper methods for
extracting standard attributes from tasks; the :code:`SpiffBpmnTaskConverter` does the same for extensions from the
:code:`spiff` package.
We don't have to do much -- all we do is replace the event definition with the original. The timer event will be
moved when the task is restored.
moved when the task is restored, and this saves us from having to write a custom parser.
.. note::
@@ -86,31 +87,22 @@ moved when the task is restored.
way to make this a little easier to follow.
When we create our serializer, we need to tell it about this task. We'll remove the converter for the standard Start
Event and add the one we created to the confiuration and create the :code:`workflow_spec_converter` from the updated
config.
Event and add the one we created to the configuration. We then get a registry of classes that the serializer knows
about that includes our custom spec, as well as all the other specs and initialize the serializer with it.
.. note::
We have not instantiated our converter class. When we call :code:`configure_workflow_spec_converter` with a
configuration (which is essentially a list of classes, split up into sections for organizational purposes),
*it* instantiates the classes for us, using the same `registry` for every class. At the end of the configuration
if returns this registry, which now knows about all of the classes that will be used for SpiffWorkflow
specifications. It is possible to pass a separately created :code:`DictionaryConverter` preconfigured with
other converters; in that case, it will be used as the base `registry`, to which specification conversions will
be added.
Because we've built up the `registry` in such a way, we can make use of the :code:`registry.convert` and
:code:`registry.restore` methods rather than figuring out how to serialize them. We can use these methods on any
objects that SpiffWorkflow knows about.
See :doc:`advanced` for more information about the serializer.
The reason there are two steps involved (regurning a registry and *then* passing it to the serializer) rather
that using the configuration directly is to allow further customization of the :code:`registry`. Workflows
can contain arbtrary data, we want to provide developers the ability to serialization code for any object. See
:ref:`serializing_custom_objects` for more information about how this works.
Finally, we have to update our parser:
.. code:: python
from SpiffWorkflow.spiff.parser import SpiffBpmnParser
from SpiffWorkflow.spiff.parser.event_parsers import StartEventParser
from SpiffWorkflow.spiff.parser.process import SpiffBpmnParser
from SpiffWorkflow.bpmn.parser.util import full_tag
parser = SpiffBpmnParser()
@@ -122,12 +114,10 @@ will use. This is a bit unintuitive, but that's how it works.
Fortunately, we were able to reuse an existing Task Spec parser, which simplifies the process quite a bit.
Having created a parser and serializer, we could replace the ones we pass in the the :code:`SimpleBpmnRunner` with these.
Having created a parser and serializer, we could create a configuration module and instantiate an engine with these
components.
I am going to leave creating a script that makes use of them to readers of this document, as it should be clear enough
how to do.
There is a very simple diagram `bpmn/tutorial/timer_start.bpmn` with the process ID `timer_start` with a Start Event
There is a very simple diagram :bpmn:`timer_start.bpmn` with the process ID `timer_start` with a Start Event
with a Duration Timer of one day that can be used to illustrate how the custom task works. If you run this workflow
with `spiff-bpmn-runner.py`, you'll see a `WAITING` Start Event; if you use the parser and serializer we just created,
you'll be propmted to complete the User Task immediately.
with any of the configurations provided, you'll see a `WAITING` Start Event; if you use the parser and serializer we
just created, you'll be propmted to complete the User Task immediately.

94
doc/bpmn/data.rst
View File

@@ -1,65 +1,26 @@
Data
====
BPMN Model
----------
We'll be using the following files from `spiff-example-cli <https://github.com/sartography/spiff-example-cli>`_.
- `bpmn-spiff/events <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/events.bpmn>`_ workflow
- `bpmn-spiff/call_activity <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/call_activity.bpmn>`_ workflow
- `bpmn-spiff/data_output <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/data_output.bpmn>`_ workflow
- `product_prices <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/product_prices.dmn>`_ DMN table
- `shipping_costs <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/shipping_costs.dmn>`_ DMN table
Data Objects
^^^^^^^^^^^^
------------
Data Objects exist at the process level and are not visible in the diagram, but when you create a Data Object
Reference, you can choose what Data Object it points to.
Data Objects exist at the process level and are not visible to individual Tasks unless explicitly indicated (ie there
is a line from the Data Object to or from a Task).
.. figure:: figures/data/data_object_configuration.png
:scale: 50%
:align: center
All Workflows have a :code:`data` attribute; like :code:`Task.data`, it is just a dictionary, and this is where the
values for Data Objects are stored.
Configuring a Data Object Reference
If a Data Object is to be made available to a Task, the value is copied from the Workflow data into the Task data
immediately before the Task becomes :code:`READY`; when the Task completes, the Data Object value is removed from the
Task data.
When a Data Output association (a line) is drawn from a task to a Data Object Reference, the value is copied
from the task data to the workflow data and removed from the task. If a Data Input Association is created from
a Data Object Reference, the value is temporarily copied into the task data while the task is being executed,
and immediately removed afterwards.
This allows sensitive data to be removed from individual tasks (in our example, the customer's credit card
number). It can also be used to prevent large objects from being repeatedly copied from task to task.
Multiple Data Object References can point to the same underlying data. In our example, we use two references
to the same Data Object to pass the credit card info to both tasks that require it. On the right panel, we can
see that only one data object exists in the process.
.. figure:: figures/data/data_objects.png
:scale: 30%
:align: center
Data objects in a process
If you step through this workflow, you'll see that the card number is not contained in the task data after
the 'Enter Payment Info' has been completed but is available to the 'Charge Customer' task later on.
Running The Model
*****************
If you have set up our example repository, this model can be run with the following command:
.. code-block:: console
./spiff-bpmn-runner.py -c order_collaboration \
-d bpmn/tutorial/product_prices.dmn bpmn/tutorial/shipping_costs.dmn \
-b bpmn/tutorial/events.bpmn bpmn/tutorial/call_activity.bpmn
If value is to be written to a Data Object, it is stored in the Workflow data and removed from the Task data when the
Task completes.
Data Objects are always available to Gateways (even though you cannot draw a line on the diagram).
Data Inputs and Outputs
^^^^^^^^^^^^^^^^^^^^^^^
-----------------------
In complex workflows, it is useful to be able to specify required Data Inputs and Outputs, especially for Call Activities
given that they are external and might be shared across many different processes.
@@ -69,30 +30,23 @@ be copied into the activity and copy *only* the variables you've specified as in
SpiffWorkflow will copy *only* the variables you've specified from the Call Activity at the end of the process. If any
of the variables are missing, SpiffWorkflow will raise an error.
Our product customization Call Activity does not require any input, but the output of the process is the product
name and quantity. We can add corresponding Data Outputs for those.
.. figure:: figures/data/data_output.png
:scale: 30%
:align: center
Our product customization Call Activity (:bpmn:`call_activity.bpmn`) does not require any input, but the output of the
process is the product name and quantity. We can add corresponding Data Outputs for those.
Data Outputs in a Call Activity
If you use the alternate version of this Call Activity (:bpmn:`data_output.bpmn`) and choose a product that has
customizations, when you inspect the data after the Call Activity completes, you'll see that the customizations have been
removed.
If you use this version of the Call Activity and choose a product that has customizations, when you inspect the data
after the Call Activity completes, you'll see that the customizations have been removed. We won't continue to use this
version of the Call Activity, because we want to preserve all the data.
To load this example (product D has two customizations):
.. code-block:: console
./runner.py -e spiff_example.spiff.file add -p order_product \
-b bpmn/tutorial/{top_level,data_output}.bpmn \
-d bpmn/tutorial/{shipping_costs,product_prices}.dmn
.. note::
The BPMN spec allows *any* task to have Data Inputs and Outputs. Our modeler does not provide a way to add them to
arbitrary tasks, but SpiffWorkflow will recognize them on any task if they are present in the BPMN XML.
Running The Model
*****************
If you have set up our example repository, this model can be run with the following command:
.. code-block:: console
./spiff-bpmn-runner.py -p order_product \
-d bpmn/tutorial/product_prices.dmn bpmn/tutorial/shipping_costs.dmn \
-b bpmn/tutorial/top_level.bpmn bpmn/tutorial/data_output.bpmn

279
doc/bpmn/events.rst
View File

@@ -1,279 +0,0 @@
Events
======
BPMN Model
----------
We'll be using the following files from `spiff-example-cli <https://github.com/sartography/spiff-example-cli>`_.
- `transaction <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/transaction.bpmn>`_ workflow
- `signal_event <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/signal_event.bpmn>`_ workflow
- `events <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/events.bpmn>`_ workflow
- `call activity <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/call_activity.bpmn>`_ workflow
- `product_prices <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/product_prices.dmn>`_ DMN table
- `shipping_costs <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/shipping_costs.dmn>`_ DMN table
A general overview of events in BPMN can be found in the :doc:`overview`
section of the documentation.
SpiffWorkflow supports the following Event Definitions:
- `Cancel Events`_
- `Signal Events`_
- `Terminate Events`_
- `Error Events`_
- `Escalation Events`_
- `Timer Events`_
- `Message Events`_
We'll include examples of all of these types in this section.
.. note::
SpiffWorflow can also support Multiple Event definitions, but our modeler does not allow you to create them,
so we will not delve into them further here.
Transactions
^^^^^^^^^^^^
We also need to introduce the concept of a Transaction because certain events
can only be used in that context. A Transaction is essentially a Subprocess, but
it must fully complete before it affects its outer workflow.
We'll make our customer's ordering process through the point they review their order
into a Transaction. If they do not complete their order, then product selections and
customizations will be discarded; if they place the order, the workflow will proceed
as before.
We'll also introduce our first event type, the Cancel Event. Cancel Events can
only be used in Transactions.
Cancel Events
^^^^^^^^^^^^^
.. figure:: figures/events/transaction.png
:scale: 30%
:align: center
Workflow with a Transaction and Cancel Event
We changed our 'Review Order' Task to be a User Task and have added a form, so
that we can give the customer the option of cancelling the order. If the customer
answers 'Y', then the workflow ends normally and we proceed to collecting
payment information.
However, if the user elects to cancel their order, we use a Cancel End Event
instead, which generates a Cancel Event. We can then attach a Cancel Boundary
Event to the Transaction, and execute that path if the event occurs. Instead of
asking the customer for their payment info, we'll direct them to a form and ask
them why they cancelled their order.
If the order is placed, the workflow will contain the order data; if it is
cancelled, it will contain the reason for cancellation instead.
To run this workflow
.. code-block:: console
./spiff-bpmn-runner.py -p order_product \
-d bpmn/tutorial/product_prices.dmn bpmn/tutorial/shipping_costs.dmn \
-b bpmn/tutorial/transaction.bpmn bpmn/tutorial/call_activity.bpmn
Signal Events
^^^^^^^^^^^^^
.. figure:: figures/events/signal_event.png
:scale: 30%
:align: center
Workflow with Signal Events
Suppose we also want to give our customer the ability to cancel their order at
any time up until they are charged. We need to throw an event after the charge
is placed and catch this event before the user completes the 'Cancel Order' task.
Once the charge is placed, the task that provides the option to cancel will
itself be cancelled when the charge event is received.
We'll also need to detect the case that the customer cancels their order and
cancel the charge task if it occurs; we'll use a separate Signal for that.
Multiple tasks can catch the same Signal Event. Suppose we add a Manager role
to our Process, and allow the Employee to refer unsuccessful charges to the
Manager for resolution. The Manager's task will also need to catch the 'Order
Cancelled' Signal Event.
Signals are referred to by name.
.. figure:: figures/events/throw_signal_event.png
:scale: 60%
:align: center
Signal Event configuration
Terminate Events
^^^^^^^^^^^^^^^^
We also added a Terminate Event to the Manager Workflow. A regular End Event
simply marks the end of a path. A Terminate Event will indicate that the
entire Process is complete and any remaining tasks should be cancelled. Our
customer cannot cancel an order that has already been cancelled, and we won't ask
them for feedback about it (we know that is was because we were unable to charge
them for it), so we do not want to execute either of those tasks.
To run this workflow
.. code-block:: console
./spiff-bpmn-runner.py -p order_product \
-d bpmn/tutorial/product_prices.dmn bpmn/tutorial/shipping_costs.dmn \
-b bpmn/tutorial/signal_event.bpmn bpmn/tutorial/call_activity.bpmn
We'll now modify our workflow to add an example of each of the other types of
events that SpiffWorkflow supports.
Error Events
^^^^^^^^^^^^
Let's turn to our order fulfillment subprocess. Either of these steps could
potentially fail, and we may want to handle each case differently.
.. figure:: figures/events/events.png
:scale: 30%
:align: center
Workflow with multiple event types
One potential failure is that our product is unavailable. This actually might be
a temporary problem, but we'll assume that it is a show stopper for the sake of
this tutorial.
We ask the Employee to verify that they were able to retrieve the product; if they
were unable to do so, then we generate an Error End Event, which we will handle
with an Interrupting Error Boundary Event (Error events are *always* interrupting).
If the product is unavailable, our Manager will notify the customer, issue a refund,
and cancel the order.
Escalation Events
^^^^^^^^^^^^^^^^^
Escalation events are a lot like Error Events and as far as I can tell, which one
to use comes down to preference, with the caveat that if you want to use an Intermediate
Event, you'll have to use Escalation, because BPMN does not allow Intermediate Error Events,
and that Error Events cannot be Non-Interrupting.
In our example, we'll assume that if we failed to ship the product, we can try again later,
so, we will not end the Subprocess (Escalation events can be either Interrupting or
Non-Interrupting).
However, we still want to notify our customer of a delay, so we use a Non-Interrupting
Escalation Boundary Event.
Both Error and Escalation Events can be optionally associated with a code. Here is
Throw Event for our `product_not_shipped` Escalation.
.. figure:: figures/events/throw_escalation_event.png
:scale: 60%
:align: center
Throw Escalation Event configuration
Error Event configuration is similar.
If no code is provided in a Catch event, it can be caught by any Escalation with the same
name.
Timer Events
^^^^^^^^^^^^
In the previous section, we mentioned that that we would try again later if we were unable
to ship the order. We can use a Duration Timer Event to force our workflow to wait a certain
amount of time before continuing. We can use this as a regular Intermediate Event (in
'Try Again Later') or a Boundary Event. Timer Boundary Events can be Interrupting, but in
this case, we simply want to notify the customer of the delay while continuing to process
their order, so we use a Non-Interrupting Event.
.. figure:: figures/events/timer_event.png
:scale: 60%
:align: center
Duration Timer Event configuration
We express the duration as an ISO8601 duration.
.. note::
We enclosed the string in quotes, because it is possible to use a variable to determine
how long the timer should wait.
It is also possible to use a static date and time to trigger an event. It will also need to be
specified in ISO8601 format.
Timer events can only be caught, that is waited on. The timer begins implicitly when we
reach the event.
Message Events
^^^^^^^^^^^^^^
In BPMN, Messages are used to communicate across processes. Technically, Messages are not
intended to be used inside a single Process, but Spiff does support this use.
Messages are similar to Signals, in that they are referenced by name, but they have the
additional property that they may contain a payload. The payload is a bit of python code that will be
evaluated against the task data and sent along with the Message. In the corresponding Message Catch
Event or Receive Task, we define a variable name where we'll store the result.
We've added a QA process to our model, which will be initiated whenever an order takes too long
to fulfill. We'll send the reason for the delay in the Message.
Spiff Messages can also optionally use Correlation Keys. The Correlation Key is an expression or set of
expressions that are evaluated against a Message payload to create an additional identifier for associating
messages with Processes.
In our example, it is possible that multiple QA processes could be started (the timer event will fire every
two minutes until the order fulfillment process is complete, or more realistically, they could be
investigating many entirely different orders, even if our simple runner does not handle that case).
In this case, the Message name is insufficient, as there will be multiple Processes that can accept
Messages based on the name.
.. figure:: figures/events/correlation.png
:scale: 50%
:align: center
Defining a correlation key
We use the timestamp of the Message creation as a unique key that can be used to distinguish between multiple
QA Processes.
.. figure:: figures/events/throw_message_event.png
:scale: 50%
:align: center
Configuring a message throw event
When we receive the event, we assign the payload to :code:`order_info`.
.. figure:: figures/events/catch_message_event.png
:scale: 50%
:align: center
Configuring a message catch event
The correlation is visible on both the Throw and Catch Events, but it is associated with the message rather
than the tasks themselves; if you update the expression on either event, the changes will appear in both places.
Running The Model
^^^^^^^^^^^^^^^^^
.. code-block:: console
./spiff-bpmn-runner.py -c order_collaboration \
-d bpmn/tutorial/product_prices.dmn bpmn/tutorial/shipping_costs.dmn \
-b bpmn/tutorial/events.bpmn bpmn/tutorial/call_activity.bpmn
.. note::
We're specifying a collaboration rather than a process so that SpiffWorkflow knows that there is more than
one top-level process.

BIN
doc/bpmn/figures/data/data_object_configuration.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 17 KiB

BIN
doc/bpmn/figures/data/data_objects.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 242 KiB

BIN
doc/bpmn/figures/data/data_output.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 94 KiB

BIN
doc/bpmn/figures/events/catch_message_event.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 28 KiB

BIN
doc/bpmn/figures/events/correlation.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 29 KiB

BIN
doc/bpmn/figures/events/events.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 246 KiB

BIN
doc/bpmn/figures/events/signal_event.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 192 KiB

BIN
doc/bpmn/figures/events/throw_escalation_event.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 20 KiB

BIN
doc/bpmn/figures/events/throw_message_event.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 38 KiB

BIN
doc/bpmn/figures/events/throw_signal_event.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 22 KiB

BIN
doc/bpmn/figures/events/timer_event.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 32 KiB

BIN
doc/bpmn/figures/events/transaction.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 145 KiB

BIN
doc/bpmn/figures/gateways/exclusive_gateway.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 102 KiB

BIN
doc/bpmn/figures/gateways/parallel_gateway.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 99 KiB

BIN
doc/bpmn/figures/multiinstance/availability_flow.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 14 KiB

BIN
doc/bpmn/figures/multiinstance/call_activity_multi.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 118 KiB

BIN
doc/bpmn/figures/multiinstance/loop_task.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 142 KiB

BIN
doc/bpmn/figures/multiinstance/multiinstance_task_configuration.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 199 KiB

BIN
doc/bpmn/figures/organization/call_activity.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 90 KiB

BIN
doc/bpmn/figures/organization/dmn_table_updated.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 59 KiB

BIN
doc/bpmn/figures/organization/lanes.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 124 KiB

BIN
doc/bpmn/figures/organization/top_level.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 125 KiB

BIN
doc/bpmn/figures/overview/ExclusiveGateway.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 46 KiB

BIN
doc/bpmn/figures/overview/bpmnbook.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 3.6 KiB

BIN
doc/bpmn/figures/overview/events.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 264 KiB

BIN
doc/bpmn/figures/overview/simplestworkflow.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 22 KiB

BIN
doc/bpmn/figures/script_engine/call_activity.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 107 KiB

BIN
doc/bpmn/figures/script_engine/top_level.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 124 KiB

BIN
doc/bpmn/figures/tasks/business_rule_task.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 62 KiB

BIN
doc/bpmn/figures/tasks/dmn_table.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 39 KiB

BIN
doc/bpmn/figures/tasks/manual_task.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 64 KiB

BIN
doc/bpmn/figures/tasks/manual_task_instructions.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 78 KiB

BIN
doc/bpmn/figures/tasks/script_task.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 66 KiB

BIN
doc/bpmn/figures/tasks/user_task.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 75 KiB

BIN
doc/bpmn/figures/tasks/user_task_form.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 40 KiB

86
doc/bpmn/gateways.rst
View File

@@ -1,86 +0,0 @@
Gateways
========
BPMN Model
----------
In this section, we'll expand our model by creating alternate paths through the
workflow depending on the current workflow state, in this case, answers provided
by the user through forms.
We've also added a second DMN table to find the cost of the selected shipping
method, and we updated our order total calculations to incorporate that cost.
We'll be using the following files from `spiff-example-cli <https://github.com/sartography/spiff-example-cli>`_.
- `gateway_types <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/gateway_types.bpmn>`_ workflow
- `product_prices <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/product_prices.dmn>`_ DMN table
- `shipping_costs <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/shipping_costs.dmn>`_ DMN table
Exclusive Gateway
^^^^^^^^^^^^^^^^^
Exclusive Gateways are used when exactly one alternative can be selected.
Suppose our products are T-shirts and we offer product C in several colors. After
the user selects a product, we check to see it if is customizable. Our default
branch will be 'Not Customizable', but we'll direct the user to a second form
if they select 'C'; our condition for choosing this branch is a simple python
expression.
.. figure:: figures/gateways/exclusive_gateway.png
:scale: 30%
:align: center
Flow configuration
Parallel Gateway
^^^^^^^^^^^^^^^^
.. sidebar:: IDs vs Names
We've assigned descriptive names to all our tasks so far. Text added to
the Name field will appear in the diagram, so sometimes it's better to
leave it blank to avoid visual clutter. I've put a description of the
gateway into the ID field instead.
Parallel Gateways are used when the subsequent tasks do not need to be completed
in any particular order. The user can complete them in any sequence and the
workflow will wait for all tasks to be finished before advancing.
We do not care whether the user chooses a shipping method or enters their
address first, but they'll need to complete both tasks before continuing.
We don't need to do any particular configuration for this gateway type.
.. figure:: figures/gateways/parallel_gateway.png
:scale: 30%
:align: center
Parallel Gateway example
Inclusive Gateway
^^^^^^^^^^^^^^^^^
SpiffWorkflow also supports Inclusive Gateways, though we do not have an example of this gateway
type in this tutorial. Inclusive Gateways have conditions on outgoing flows like Exclusive Gateways,
but unlike Exclusive Gateways, multiple paths may be taken if more than one conition is met.
Event-Based Gateway
^^^^^^^^^^^^^^^^^^^
SpiffWorkflow supports Event-Based Gateways, though we do not use them in this tutorial. Event-Based
gateways select an outgoing flow based on an event. We'll discuss events in the next section.
Running The Model
^^^^^^^^^^^^^^^^^
If you have set up our example repository, this model can be run with the
following command:
.. code-block:: console
./spiff-bpmn-runner.py -p order_product \
-d bpmn/tutorial/product_prices.dmn bpmn/tutorial/shipping_costs.dmn \
-b bpmn/tutorial/gateway_types.bpmn

155
doc/bpmn/imports.rst Normal file
View File

@@ -0,0 +1,155 @@
Creating and Running Workflows
==============================
.. code-block:: python
from SpiffWorkflow.bpmn import BpmnWorkflow, BpmnEvent
from SpiffWorfkflow import TaskState
Parsing
=======
Basic Parsing
-------------
.. code-block:: python
from SpiffWorkflow.bpmn.parser import BpmnParser, BpmnValidator
Customized Parsing
------------------
.. code-block:: python
from SpiffWorkflow.bpmn.parser import TaskParser, EventDefinitionParser
Examples
--------
- :doc:`parsing`
- :doc:`custom_task_spec`
Script Engine
=============
To modify the default execution environment
-------------------------------------------
.. code-block:: python
from SpiffWorkflow.bpmn.script_engine import TaskDataEnvironment
To control how the engine interacts with the workflow
-----------------------------------------------------
.. code-block:: python
from SpiffWorkflow.bpmn.script_engine import PythonScriptEngine
To implement custom exec/eval
-----------------------------
.. code-block:: python
from SpiffWorkflow.bpmn.script_engine import BasePythonScriptEngineEnvironment
Examples
--------
- :doc:`script_engine`
Specs
=====
Using a Spec
------------
.. code-block:: python
from SpiffWorkflow.bpmn.specs import <TaskSpec>
from SpiffWorkflow.bpmn.specs.event_definition import <EventDefinition>
Extending a Spec
----------------
.. code-block:: python
from SpiffWorkflow.bpmn.specs import BpmnTaskSpec # Implements generic BPMN behavior
from SpiffWorkflow.bpmn.specs.mixins import <TaskSpecMixin> # Implements specific BPMN behavior
Implement a Datastore
---------------------
.. code-block:: python
from SpiffWorkflow.bpmn.spec import BpmnDataStoreSpecification
Examples
--------
- :doc:`workflows`
- :doc:`custom_task_spec`
Serializer
==========
Basic Usage
-----------
.. code-block:: python
from SpiffWorkflow.bpmn.serializer import BpmnWorkflowSerializer
Custom Data
-----------
.. code-block:: python
from SpiffWorkflow.bpmn.serializer import DefaultRegistry
Spec Customizations
-------------------
.. code-block:: python
from SpiffWorkflow.bpmn.serializer import DEFAULT_CONFIG
from SpiffWorkflow.bpmn.serializer.default import <TaskSpecConverter>
from SpiffWorkflow.bpmn.serializer.helpers import (
TaskSpecConverter,
EventDefinitionConverter,
BpmnDataSpecificationConverter,
)
Examples
--------
- :doc:`serialization`
- :doc:`custom_task_specs`
DMN
===
.. code-block:: python
from SpiffWorkflow.dmn.parser import BpmnDmnParser
from SpiffWorkflow.dmn.specs import BusinessRuleTaskMixin
from SpiffWorkflow.dmn.serializer import BaseBusinessRuleTaskConverter
Spiff
=====
.. code-block:: python
from SpiffWorkflow.spiff.parser import SpiffBpmnParser, VALIDATOR
from SpiffWorkflow.spiff.specs import <TaskSpec>
from SpiffWorkflow.spiff.serializer import DEFAULT_CONFIG
Camunda
=======
.. code-block:: python
from SpiffWorkflow.camunda.parser import CamundaParser
from SpiffWorkflow.camunda.specs import <TaskSpec>
from SpiffWorkfllw.camunda.serializer import DEFAULT_CONFIG

107
doc/bpmn/index.rst Normal file
View File

@@ -0,0 +1,107 @@
SpiffWorkflow and BPMN
======================
All the Python code and BPMN models used here are available in an example
project `spiff-example-cli <https://github.com/sartography/spiff-example-cli>`_
This example application serves two purposes:
- to illustate what a developer will need to do to use this library
- to let library users experiment without fully building out their own system
The Example Application
-----------------------
.. toctree::
:maxdepth: 2
application
Parsing
-------
.. toctree::
:maxdepth: 2
parsing
Workflows and Tasks
-------------------
.. toctree::
:maxdepth: 2
workflows
Scripting Environment
---------------------
.. toctree::
:maxdepth: 2
script_engine
Data
----
.. toctree::
:maxdepth: 2
data
Serialization
-------------
.. toctree::
:maxdepth: 2
serialization
Custom Tasks
------------
.. toctree::
:maxdepth: 2
custom_task_spec
Logging
-------
.. toctree::
:maxdepth: 2
logging
Exceptions
----------
.. toctree::
:maxdepth: 2
errors
What's in the Module and Where to Find It
-----------------------------------------
.. toctree::
:maxdepth: 1
imports
Supported BPMN Elements
-----------------------
.. toctree::
:maxdepth: 2
supported
Camunda Editor Support
----------------------
.. toctree::
:maxdepth: 2
camunda/index

103
doc/bpmn/intro.rst
View File

@@ -1,103 +0,0 @@
BPMN Workflows
==============
The basic idea of SpiffWorkflow is that you can use it to write an interpreter
in Python that creates business applications from BPMN models. In this section,
we'll develop a model of a reasonably complex process and show how to run it.
We expect that readers will fall into two general categories:
- People with a background in BPMN who might not be very familiar Python
- Python developers who might not know much about BPMN
This section of the documentation provides an example that (hopefully) serves
the needs of both groups. We will introduce some of the more common BPMN
elements and show how to build a simple workflow runner around them.
SpiffWorkflow does heavy-lifting such as keeping track of task dependencies and
states and providing the ability to serialize or deserialize a workflow that
has not been completed. The developer will write code for displaying workflow
state and presenting tasks to users of their application.
All the Python code and BPMN models used here are available in an example
project called `spiff-example-cli <https://github.com/sartography/spiff-example-cli>`_.
Quickstart
----------
Check out the code in `spiff-example-cli <https://github.com/sartography/spiff-example-cli>`_
and follow the instructions to set up an environment to run it in.
Run the sample workflow we built up using our example application with the following
command:
.. code-block:: console
./spiff-bpmn-runner.py -c order_collaboration \
-d bpmn/tutorial/{product_prices,shipping_costs}.dmn \
-b bpmn/tutorial/{top_level_multi,call_activity_multi}.bpmn
.. sidebar:: BPMN Runner
The example app provides a utility for running BPMN Diagrams from the command
line that will allow you to introspect a bit on a running process. You
can see the options available by running:
./spiff-bpmn-runner.py --help
The code in the workflow runner and the models in the bpmn directory of the
repository will be discussed in the remainder of this tutorial.
Supported BPMN Elements
-----------------------
.. toctree::
:maxdepth: 3
tasks
gateways
organization
events
data
multiinstance
Putting it All Together
-----------------------
.. toctree::
:maxdepth: 2
synthesis
Features in More Depth
----------------------
.. toctree::
:maxdepth: 2
advanced
Custom Task Specs
-----------------
.. toctree::
:maxdepth: 2
custom_task_spec
Exceptions
----------
.. toctree::
:maxdepth: 2
errors
Camunda Editor Support
----------------------
.. toctree::
:maxdepth: 2
camunda/support

45
doc/bpmn/logging.rst Normal file
View File

@@ -0,0 +1,45 @@
Logging
=======
Spiff provides several loggers:
- the :code:`spiff` logger, which emits messages when a workflow is initialized and when tasks change state
- the :code:`spiff.metrics` logger, which emits messages containing the elapsed duration of tasks
All log entries created during the course of running a workflow contain the following extra attributes:
- :code:`workflow_spec`: the name of the current workflow spec
- :code:`task_spec`: the name of the task spec
- :code:`task_id`: the ID of the task)
- :code:`task_type`: the name of the task's spec's class
If the log level is less than 20:
- :code:`data` the task data (this can be quite large and is only made available for debugging purposes)
If the log level is less than or equal to 10:
- :code:`internal_data`: the task internal data (only available at DEBUG or below because it is not typically useful)
The metrics logger additionally provides and only emits messages at the DEBUG level:
- :code:`elapsed`: the time it took the task to run after (ie, the duration of the :code:`task.run` method)
In our command line UI (:app:`cli/subcommands.py`), we've added a few of these extra attributes to the log records:
.. code-block:: python
spiff_logger = logging.getLogger('spiff')
spiff_handler = logging.StreamHandler()
spiff_handler.setFormatter('%(asctime)s [%(name)s:%(levelname)s] (%(workflow_spec)s:%(task_spec)s) %(message)s')
spiff_logger.addHandler(spiff_handler)
metrics_logger = logging.getLogger('spiff.metrics')
metrics_handler = logging.StreamHandler()
metrics_handler.setFormatter('%(asctime)s [%(name)s:%(levelname)s] (%(workflow_spec)s:%(task_spec)s) %(elasped)s')
metrics_logger.addHandler(metrics_handler)
In the configuration module :app:`spiff/file.py` that appears in many examples, we set the level of the :code:`spiff`
logger to :code:`INFO`, so that we'll see messages about task state changes, but we ignore the metrics log; however,
the configuration could easily be changed to include it (it can, however generate a high volume of very large records,
so consider yourself warned!).

35
doc/bpmn/make.bat
View File

@@ -1,35 +0,0 @@
@ECHO OFF
pushd %~dp0
REM Command file for Sphinx documentation
if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=.
set BUILDDIR=_build
if "%1" == "" goto help
%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
goto end
:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
:end
popd

143
doc/bpmn/multiinstance.rst
View File

@@ -1,143 +0,0 @@
MultiInstance Tasks
===================
BPMN Model
----------
We'll be using the following files from `spiff-example-cli <https://github.com/sartography/spiff-example-cli>`_.
- `multiinstance <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/top_level_multi.bpmn>`_ workflow
- `call activity multi <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/call_activity_multi.bpmn>`_ workflow
- `product_prices <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/product_prices.dmn>`_ DMN table
- `shipping_costs <https://github.com/sartography/spiff-example-cli/blob/main/bpmntutorial//shipping_costs.dmn>`_ DMN table
Loop Task
^^^^^^^^^
Suppose we want our customer to be able to select more than one product.
We'll run our 'Select and Customize Product' Call Activity as a Loop Task.
First we'll update the Call Activity's model to ask the customer if they would like to continue shopping.
.. figure:: figures/multiinstance/call_activity_multi.png
:scale: 30%
:align: center
Selecting more than one product
We've also added a *postScript* to the user task. Spiffworkflow provides extensions that allow scripts to be
run before and after tasks. It is often the case that data needs to be manipulated before and after a task.
We could add regular Script Tasks before and after, but diagrams quickly become cluttered with scripts, and
these extensions are intended to alleviate that.
We use a *postScript* to add the current product to a list of products.
.. code:: python
products.append({
'product_name': product_name,
'product_quantity': product_quantity,
'product_color': product_color,
'product_size': product_size,
'product_style': product_style,
'product_price': product_price,
})
We'll use a *preScript* on the first User Task (Select Product and Quantity) to initialize these variables to
:code:`None` each time we execute the task.
Loop Tasks run either a specified number of times or until a completion condition is met. Since we can't
know in advance how many products the customer will select, we'll add :code:`continue_shopping == 'Y'` as a
completion condition. We'll re-run this Call Activity as long as the customer indicates they want to choose
another product. We'll also set up the list of products that we plan on appending to.
We also added a postscript to this activity to delete the customization values so that we won't have to
look at them for the remainder of the workflow.
.. figure:: figures/multiinstance/loop_task.png
:scale: 30%
:align: center
Call Activity with Loop
We also needed to update our Script Task and the Instructions of the Review Order Task to handle an array
of products rather than a single product.
Here is our new script
.. code:: python
order_total = sum([ p['product_quantity'] * p['product_price'] for p in products ]) + shipping_cost
And our order summary
.. code:: python
Order Summary
{% for product in products %}
{{ product['product_name'] }}
Quantity: {{ product['product_quantity'] }}
Price: {{ product['product_price'] }}
{% endfor %}
Shipping Cost: {{ shipping_cost }}
Order Total: {{ order_total }}
Parallel MultiInstance
^^^^^^^^^^^^^^^^^^^^^^
We'll also update our 'Retrieve Product' Task and 'Product Not Available' flows to
accommodate multiple products. We can use a Parallel MultiInstance for this, since
it does not matter what order our Employee retrieves the products in.
.. figure:: figures/multiinstance/multiinstance_task_configuration.png
:scale: 30%
:align: center
MultiInstance Task configuration
We've specified :code:`products` as our Input Collection and :code:`product` as our Input Item. The
Input Collection should be an existing collection. We'll create a task instance for each element of
the collection, and copy the value into the Input Item; this is how we'll access the data of the
element.
.. :code::
Item: {{product['product_quantity']}} of {{product['product_name']}}
We also specified :code:`availability` as our Output Collection. Since this variable does not exist,
SpiffWorkflow will automatically create it. You can use an existing variable as an Output Collection;
in this case, its contents will be updated with new values. The Output Item will be copied out of the
child task into the Output Collection.
The 'Retrieve Product' task creates :code:`product_available` from the form input.
Since our input is a list, our output will also be a list. It is possible to generate different output
types if you create the output collections before referring to them.
We have to update our gateway condition to handle the list:
.. figure:: figures/multiinstance/availability_flow.png
:scale: 60%
:align: center
Gateway Condition
Sequential MultiInstance
^^^^^^^^^^^^^^^^^^^^^^^^
SpiffWorkflow also supports Sequential MultiInstance Tasks for collections, or if the loopCardinality
is known in advance, although we have not added an example of this to our workflow. Their configuraiton
is almost idenitcal to the configuration for Parallel MultiInstance Tasks.
Running The Model
^^^^^^^^^^^^^^^^^
If you have set up our example repository, this model can be run with the following command:
.. code-block:: console
./spiff-bpmn-runner.py -p order_product \
-d bpmn/tutorial/product_prices.dmn bpmn/tutorial/shipping_costs.dmn \
-b bpmn/tutorial/top_level_multi.bpmn bpmn/tutorial/call_activity_multi.bpmn

125
doc/bpmn/organization.rst
View File

@@ -1,125 +0,0 @@
Organizing More Complex Workflows
=================================
BPMN Model
----------
We'll be using the following files from `spiff-example-cli <https://github.com/sartography/spiff-example-cli>`_.
- `lanes <https://github.com/sartography/spiff-example-cli/blob/main/tutorial/bpmn/lanes.bpmn>`_ workflow
- `top_level <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/top_level.bpmn>`_ workflow
- `call_activity <https://github.com/sartography/spiff-example-cli/blob/main/tutorial/bpmn/call_activity.bpmn>`_ workflow
- `product_prices <https://github.com/sartography/spiff-example-cli/blob/main/tutorial/bpmn/product_prices.dmn>`_ DMN table
- `shipping_costs <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/shipping_costs.dmn>`_ DMN table
Lanes
^^^^^
Lanes are a method in BPMN to distinguish roles for the workflow and who is
responsible for which actions. In some cases this will be different business
units, and in some cases this will be different individuals - it really depends
on the nature of the workflow. Within the BPMN editor, this is done by choosing the
'Create pool/participant' option from the toolbar on the left hand side.
We'll modify our workflow to get the customer's payment information and send it
to an employee who will charge the customer and fulfill the order.
.. figure:: figures/organization/lanes.png
:scale: 30%
:align: center
Workflow with lanes
To run this workflow
.. code-block:: console
./spiff-bpmn-runner.py -p order_product \
-d bpmn/tutorial/product_prices.dmn bpmn/tutorial/shipping_costs.dmn \
-b bpmn/tutorial/lanes.bpmn
For a simple code example of displaying a tasks lane, see `Handling Lanes`_
Subprocesses
^^^^^^^^^^^^
Subprocesses allow us to conceptualize a group of tasks as a unit by creating a
mini-workflow inside a task. Subprocess Tasks come in two different flavors: expanded
or collapsed. The difference between the two types is visual rather than functional.
It also possible to refer to external processes via a Call Activity Task. This
allows us to 'call' a separate Process (which might be stored independently of the
Process we're implementing) by referencing the ID of the called Process, which can simplify
business logic and make it re-usable.
We'll expand 'Fulfill Order' into sub tasks -- retrieving the product and shipping
the order -- and create an Expanded Subprocess.
We'll also expand our selection of products, adding several new products and the ability
to customize certain products by size and style in addition to color.
.. figure:: figures/organization/dmn_table_updated.png
:scale: 60%
:align: center
Updated product list
.. note::
I've added what customizations are available for each product in the 'Annotations'
column of the DMN table. This is not actually used by Spiff; it simply provides
the option of documenting the decisions contained in the table.
Since adding gateways for navigating the new options will add a certain amount of
clutter to our diagram, we'll create a separate workflow for selecting and customizing
products and refer to that in our main workflow.
.. figure:: figures/organization/call_activity.png
:scale: 30%
:align: center
Subworkflow for product selection
We need to make sure the 'Called Element' matches the ID we assigned in the called Process.
.. figure:: figures/organization/top_level.png
:scale: 30%
:align: center
Parent workflow
Running the Model
^^^^^^^^^^^^^^^^^
.. code-block:: console
./spiff-bpmn-runner.py -p order_product \
-d bpmn/tutorial/product_prices.dmn bpmn/tutorial/shipping_costs.dmn \
-b bpmn/tutorial/top_level.bpmn bpmn/tutorial/call_activity.bpmn
Example Application Code
------------------------
Handling Lanes
^^^^^^^^^^^^^^
We are not required to handle lanes in our application, but most of the time we'll
probably want a way of filtering on lanes and selectively displaying tasks. In
our sample application, we'll simply display which lane a task belongs to.
.. code:: python
def get_task_description(self, task, include_state=True):
task_spec = task.task_spec
lane = f'{task_spec.lane}' if task_spec.lane is not None else '-'
name = task_spec.bpmn_name if task_spec.bpmn_name is not None else '-'
description = task_spec.description if task_spec.description is not None else 'Task'
state = f'{task.get_state_name()}' if include_state else ''
return f'[{lane}] {name} ({description}: {task_spec.bpmn_id}) {state}'
The tasks lane can be obtained from :code:`task.task_spec.lane`, which will be :code:`None`
if the task is not part of a lane.
See the Filtering Tasks Section of :doc:`advanced` more information about working with lanes in Spiff.

135
doc/bpmn/overview.rst
View File

@@ -1,135 +0,0 @@
Overview
========
BPMN and SpiffWorkflow
----------------------
.. sidebar:: BPMN Resources
This guide is a mere introduction to BPMN.
For more serious modeling, we recommend looking for more comprehensive
resources. We have used the `books by Bruce Silver <https://www.amazon.com/Bruce-Silver/e/B0062AXUFY/ref=dp_byline_cont_pop_book_1>`_
as a guide for our BPMN modeling.
.. image:: figures/overview/bpmnbook.jpg
:align: center
Business Process Model and Notation (BPMN) is a diagramming language for specifying business
processes. BPMN links the realms of business and IT, and creates a common process language that
can be shared between the two.
BPMN describes details of process behaviors efficiently in a diagram. The meaning is precise enough
to describe the technical details that control process execution in an automation engine.
SpiffWorkflow allows you to create code to directly execute a BPMN diagram.
When using SpiffWorkflow, a client can manipulate the BPMN diagram and still have their product work
without a need for you to edit the Python code, improving response and turnaround time.
.. sidebar:: BPMN Modelers
Currently the best way to build BPMN diagrams is through our SpiffArena project
which provides a custom BPMN Modeler, along with ways to test and run BPMN diagrams
from within a web browser. Please see our `getting started guide <https://www.spiffworkflow.org/posts/articles/get_started/>`_
for more information.
It is also possible to use version 7 of the Camunda Modeler to create BPMN diagrams.
However, be cautious of the properies panel settings, as many of these settings are
not a part of the BPMN Standard, and are not handled in the same way within SpiffWorkflow.
You can download the Camunda Modeler from `Camunda <https://camunda.com/download/modeler/>`_.
Today, nearly every process modeling tool supports BPMN in some fashion making it a great tool to
learn and use. This page provides a brief overview, and the following section provides a more
in-depth look. There are many resources for additional information about BPMN.
Most of the examples in this guide have been created with
`our modeler <https://github.com/sartography/bpmn-js-spiffworkflow>`_, which is based on
`bpmn.js <https://bpmn.io/toolkit/bpmn-js/>`_.
A Simple Workflow
-----------------
All BPMN models have a start event and at least one end event. The start event
is represented with a single thin border circle. An end event is represented
by a single thick border circle.
The following example also has one task, represented by the rectangle with curved corners.
.. figure:: figures/overview/simplestworkflow.png
:scale: 25%
:align: center
A simple workflow.
The sequence flow is represented with a solid line connector. When the node at
the tail of a sequence flow completes, the node at the arrowhead is enabled to start.
A More Complicated Workflow
---------------------------
.. figure:: figures/overview/ExclusiveGateway.png
:scale: 25%
:align: center
A workflow with a gateway
In this example, the diamond shape is called a gateway. It represents a branch
point in our flow. This gateway is an exclusive data-based gateway (also
called an XOR gateway). With an exclusive gateway, you must take one path or
the other based on some data condition. BPMN has other gateway types.
The important point is that we can use a gateway to add a branch in the
workflow **without** creating an explicit branch in our Python code.
An Even More Complicated Workflow
------
BPMN is a rich language that can describe many different types of processes. In
the following pages we'll cover lanes (a way to distribute work across different
roles) events (a way to handle asynchronous events), multi-instance tasks (that
can be executed many times in parallel or in sequence) and decomposition (the
many ways you can interconnect diagrams to build larger more complex processes)
We are just scratching the surface. For now let's take one more step and look
at what Events make possible.
Events
^^^^^^^
In the above simple workflows, all of the transitions are deterministic and we
have direct connections between tasks. We need to handle the cases where an event
may or may not happen, and link these events in different parts of the workflow or
across different workflows.
BPMN has a comprehensive suite of event elements. SpiffWorkflow does not support
every single BPMN event type, but it can handle many of them.
.. figure:: figures/overview/events.png
:scale: 25%
:align: center
A workflow containing events
We've already seen plain Start and End Events. BPMN also includes the concept
of Intermediate Events (standalone events that may be Throwing or Catching) as well
as Boundary Events (which are exclusively Catching).
All Start Events are inherently Catching Events (a workflow can be initiated if a
particular event occurs) and all End Events are Throwing Events (they can convey
the final state of a workflow or path to other tasks and workflows).
If an Intermediate Throwing Event is added to a flow, the event it represents
will be generated and the flow will continue immediately. If an Intermediate
Catching Event is added to a flow, the workflow will wait to catch the event it
represents before advancing.
A Boundary Event represents an event that may be caught only while a particular task
is being executed and comes in two types: Interrupting (in which case the task it is
attached to will be cancelled if the event is received) or Non-Interrupting (in
which case the task will continue). In both cases, flows may emanate from the
Boundary Event, which will trigger those paths if the events occur while the task
is being executed.

141
doc/bpmn/parsing.rst Normal file
View File

@@ -0,0 +1,141 @@
Parsing BPMN
============
The example application assumes that a :code:`BpmnProcessSpec` will be generated for each process independently of
starting a workflow and that these will be immediately serialized and provided with a ID. We'll discuss serialization
in greater detail later; for now we'll simply note that the file serializer simply writes a JSON representation of the
spec to a file and uses the filename as the ID.
.. note::
This is design choice -- it would be possible to re-parse the specs each time a process was run.
Default Parsers
===============
Importing
---------
Each of the BPMN modules (:code:`bpmn`, :code:`spiff`, or :code:`camunda`) has a parser that is preconfigured with
the specs in that module (if a particular TaskSpec is not implemented in the module, :code:`bpmn` TaskSpec is used).
- :code:`bpmn`: :code:`from SpiffWorkflow.bpmn.parser import BpmnParser`
- :code:`dmn`: :code:`from SpiffWorkflow.dmn.parser import BpmnDmnParser`
- :code:`spiff`: :code:`from SpiffWorkflow.spiff.parser import SpiffBpmnParser`
- :code:`camunda`: :code:`from SpiffWorkflow.camunda.parser import CamundaParser`
.. note::
The default parser cannot parse DMN files. The :code:`BpmnDmnParser` extends the default parser to add that
capability. Both the :code:`spiff` and :code:`camunda` parsers inherit from :code:`BpmnDmnParser`.
Instantiation of a parser has no required arguments, but there are several optional parameters.
Validation
----------
The :code:`SpiffWorkflow.bpmn.parser` module also contains a :code:`BpmnValidator`.
The default validator validates against the BPMN 2.0 spec. It is possible to import additional specifications (e.g.
for custom extensions) as well.
By default the parser does not validate, but if a validator is passed in, it will be used on any files added to the parser.
.. code-block:: python
from SpiffWorkflow.bpmn.parser import BpmnParser, BpmnValidator
parser = BpmnParser(validator=BpmnValidator())
Spec Descriptions
-----------------
A default set of :code:`decription` attributes for each Task Spec. The description is intended to be a user-friendly
representation of the task type. It is a mapping of XML tag to string.
The default set of descriptions can be found in :code:`SpiffWorkflow.bpmn.parser.spec_descriptions`.
Creating a BpmnProcessSpec from BPMN Process
--------------------------------------------
From the :code:`add_spec` method of our BPMN engine (:app:`engine/engine.py`):
.. code-block:: python
def add_spec(self, process_id, bpmn_files, dmn_files):
self.add_files(bpmn_files, dmn_files)
try:
spec = self.parser.get_spec(process_id)
dependencies = self.parser.get_subprocess_specs(process_id)
except ValidationException as exc:
self.parser.process_parsers = {}
raise exc
spec_id = self.serializer.create_workflow_spec(spec, dependencies)
logger.info(f'Added {process_id} with id {spec_id}')
return spec_id
def add_files(self, bpmn_files, dmn_files):
self.parser.add_bpmn_files(bpmn_files)
if dmn_files is not None:
self.parser.add_dmn_files(dmn_files)
The first step is adding BPMN and DMN files to the parser using the :code:`add_bpmn_files` and
:code:`add_dmn_files` methods.
We use the :code:`get_spec` to parse the BPMN process with the provided :code:`process_id` (*not* the process name).
.. note::
Ths parser was designed to load one set of files and parse a process and will raise a :code:`ValidationException`
if any duplicate iDs are present. The available processes are immediately added to :code:`process_parsers`, so
re-adding a file will generate an exception. Therefore, if we run into a problem (the specific case here) or wish
to reuse the same parser, we need to clear this attribute.
Other Methods for Adding Files
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- :code:`add_bpmn_files_by_glob`: Loads files from a glob instead of a list.
- :code:`add_bpmn_file`: Adds one file rather than a list.
- :code:`load_bpmn_str`: Loads and parses XML from a string.
- :code:`load_bpmn_io`: Loads and parses XML from an object implementing the IO interface.
- :code:`load_bpmn_xml`: Parses BPMN from an :code:`lxml` parsed tree.
.. _parsing_subprocesses:
Handling Subprocesses and Call Activities
-----------------------------------------
Internally, Call Activities and Subprocesses (as well as Transactional Subprocesses) are all treated as separate
specifications. This is to prevent a single specification from becoming too large, especially in the case where the
same process spec will be called more than once.
The :code:`get_subprocess_specs` method takes a process ID and recursively searches for Call Activities, Subprocesses,
etc used by or defined in the provided BPMN files. It returns a mapping of process ID to parsed specification.
Other Methods for Finding Dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- :code:`find_all_specs`: Returns a mapping of name -> :code:`BpmnWorkflowSpec` for all processes in all files that have been
provided to the parser at that point.
- :code:`get_process_dependencies`: Returns a list of process IDs referenced by the provided process ID
- :code:`get_dmn_dependencies`: Returns a list of DMN IDs referenced by the provided process ID
Creating a BpmnProcessSpec from a BPMN Collaboration
----------------------------------------------------
The parser can also generate a workflow spec based on a collaboration:
.. code-block:: python
def add_collaboration(self, collaboration_id, bpmn_files, dmn_files=None):
self.add_files(bpmn_files, dmn_files)
try:
spec, dependencies = self.parser.get_collaboration(collaboration_id)
except ValidationException as exc:
self.parser.process_parsers = {}
raise exc
A spec is created for each of the processes in the collaboration, and each of these processes is wrapped inside a
subworkflow. This means that a spec created this way will *always* require subprocess specs, and this method
returns the generated spec (which doesn't directly correspond to anything in the BPMN file) as well as the processes
present in the file, and theit dependencies.

209
doc/bpmn/script_engine.rst Normal file
View File

@@ -0,0 +1,209 @@
Script Engine Overview
======================
You may need to modify the default script engine, whether because you need to make additional
functionality available to it, or because you might want to restrict its capabilities for
security reasons.
.. warning::
By default, the scripting environment passes input directly to :code:`eval` and :code:`exec`! In most
cases, you'll want to replace the default scripting environment with one of your own.
Restricting the Script Environment
==================================
The following example replaces the default global enviroment with the one provided by
`RestrictedPython <https://restrictedpython.readthedocs.io/en/latest/>`_
We've modified our engine configuration to use the restricted environment in :app:`spiff/restricted.py`
.. code:: python
from RestrictedPython import safe_globals
from SpiffWorkflow.bpmn.PythonScriptEngineEnvironment import TaskDataEnvironment
restricted_env = TaskDataEnvironment(safe_globals)
restricted_script_engine = PythonScriptEngine(environment=restricted_env)
We've also included a dangerous process in :bpmn:`dangerous.bpmn`
If you run this process using the regular script enviromment, the BPMN process will get the OS process ID and
prompt you to kill it; if you answer 'Y', it will do so (it won't actually do anything more dangerous than screw
up your terminal settings, but hopefully proves the point).
.. code-block:: console
./runner.py -e spiff_example.spiff.file add -p end_it_all -b bpmn/tutorial/dangerous.bpmn
./runner.py -e spiff_example.spiff.file
If you load the restricted engine:
.. code-block:: console
./runner.py -e spiff_example.spiff.restricted
You'll get an error, because imports have been restricted.
.. note::
Since we used exactly the same parser and serializer, we can simply switch back and forth between these
two script engines (that is the only difference between the two configurations).
Making Custom Classes and Functions Available
=============================================
Another reason you might want to customize the scripting environment is to provide access to custom
classes or functions.
In many of our example models, we use DMN tables to obtain product information. DMN is a convenient
way of making table data available to our processes.
However, in a slightly more realistic scenario, we would surely have information about how the product
could be customized in a database somewhere. We would not hard code product information in our diagram
(although it is much easier to modify the BPMN and DMN models than to change the code itself!). Our
shipping costs would not be static, but would depend on the size of the order and where it was being
shipped -- maybe we'd query an API provided by our shipper.
SpiffWorkflow is obviously **not** going to know how to query **your** database or make API calls to
**your** vendors. However, one way of making this functionality available inside your diagram is to
implement the calls in functions and add those functions to the scripting environment, where they
can be called by Script Tasks.
We are not going to actually include a database or API and write code for connecting to and querying
it, but since we only have 7 products we can model our database with a simple dictionary lookup
and just return the same static info for shipping for the purposes of the tutorial.
We'll customize our scripting environment in :app:`spiff/custom_object.py`:
.. code:: python
from collections import namedtuple
ProductInfo = namedtuple('ProductInfo', ['color', 'size', 'style', 'price'])
INVENTORY = {
'product_a': ProductInfo(False, False, False, 15.00),
'product_b': ProductInfo(False, False, False, 15.00),
'product_c': ProductInfo(True, False, False, 25.00),
'product_d': ProductInfo(True, True, False, 20.00),
'product_e': ProductInfo(True, True, True, 25.00),
'product_f': ProductInfo(True, True, True, 30.00),
'product_g': ProductInfo(False, False, True, 25.00),
}
def lookup_product_info(product_name):
return INVENTORY[product_name]
def lookup_shipping_cost(shipping_method):
return 25.00 if shipping_method == 'Overnight' else 5.00
script_env = TaskDataEnvironment({
'datetime': datetime,
'lookup_product_info': lookup_product_info,
'lookup_shipping_cost': lookup_shipping_cost,
})
script_engine = PythonScriptEngine(script_env)
.. note::
We're also adding :code:`datetime`, because other parts of the process require it.
We can use the custom functions in script tasks like any normal function. To load the example diagrams that use the
custom script engine:
.. code-block:: console
./runner.py -e spiff_example.spiff.custom_object add -p order_product \
-b bpmn/tutorial/{top_level_script,call_activity_script}.bpmn
If you start the application in interactive mode and choose a product, you'll see tuple info reflected in the task data
after selecting a product.
Service Tasks
=============
We can also use Service Tasks to accomplish the same goal. Service Tasks are also executed by the workflow's script
engine, but through a different method, with the help of some custom extensions in the :code:`spiff` module:
- `operation_name`, the name assigned to the service being called
- `operation_params`, the parameters the operation requires
The advantage of a Service Task is that it is a bit more transparent what is happening (at least at a conceptual level)
than function calls embedded in a Script Task.
We implement the :code:`PythonScriptEngine.call_service` method in :app:`spiff/service_task.py`:
.. code:: python
service_task_env = TaskDataEnvironment({
'product_info_from_dict': product_info_from_dict,
'datetime': datetime,
})
class ServiceTaskEngine(PythonScriptEngine):
def __init__(self):
super().__init__(environment=service_task_env)
def call_service(self, operation_name, operation_params, task_data):
if operation_name == 'lookup_product_info':
product_info = lookup_product_info(operation_params['product_name']['value'])
result = product_info_to_dict(product_info)
elif operation_name == 'lookup_shipping_cost':
result = lookup_shipping_cost(operation_params['shipping_method']['value'])
else:
raise Exception("Unknown Service!")
return json.dumps(result)
service_task_engine = ServiceTaskEngine()
Instead of adding our custom functions to the environment, we'll override :code:`call_service` and call them directly
according to the `operation_name` that was given. The :code:`spiff` Service Task also evaluates the parameters
against the task data for us, so we can pass those in directly. The Service Task will also store our result in
a user-specified variable.
We need to send the result back as json, so we'll reuse the functions we wrote for the serializer (see
:ref:`serializing_custom_objects`).
The Service Task will assign the dictionary as the operation result, so we'll add a `postScript` to the Service Task
that retrieves the product information that creates a :code:`ProductInfo` instance from the dictionary, so we need to
add that to the scripting enviroment too.
The XML for the Service Task looks like this:
.. code:: xml
<bpmn:serviceTask id="Activity_1ln3xkw" name="Lookup Product Info">
<bpmn:extensionElements>
<spiffworkflow:serviceTaskOperator id="lookup_product_info" resultVariable="product_info">
<spiffworkflow:parameters>
<spiffworkflow:parameter id="product_name" type="str" value="product_name"/>
</spiffworkflow:parameters>
</spiffworkflow:serviceTaskOperator>
<spiffworkflow:postScript>product_info = product_info_from_dict(product_info)</spiffworkflow:postScript>
</bpmn:extensionElements>
<bpmn:incoming>Flow_104dmrv</bpmn:incoming>
<bpmn:outgoing>Flow_06k811b</bpmn:outgoing>
</bpmn:serviceTask>
Getting this information into the XML is a little bit beyond the scope of this tutorial, as it involves more than
just SpiffWorkflow. I hand edited it for this case, but you can hardly ask your BPMN authors to do that!
Our `modeler <https://github.com/sartography/bpmn-js-spiffworkflow>`_ has a means of providing a list of services and
their parameters that can be displayed to a BPMN author in the Service Task configuration panel. There is an example of
hard-coding a list of services in
`app.js <https://github.com/sartography/bpmn-js-spiffworkflow/blob/0a9db509a0e85aa7adecc8301d8fbca9db75ac7c/app/app.js#L47>`_
and as suggested, it would be reasonably straightforward to replace this with a API call.
`SpiffArena <https://www.spiffworkflow.org/posts/articles/get_started/>`_ has robust mechanisms for handling this that
might serve as a model for you.
How this all works is obviously heavily dependent on your application, so we won't go into further detail here, except
to give you a bare bones starting point for implementing something yourself that meets your own needs.
To run this workflow:
.. code-block:: console
./runner.py -e spiff_example.spiff.service_task add -p order_product \
-b bpmn/tutorial/{top_level_service_task,call_activity_service_task}.bpmn

280
doc/bpmn/serialization.rst Normal file
View File

@@ -0,0 +1,280 @@
Introduction
============
Given the long-running nature of many workflows, robust serialization capabilities are critical to any kind of
workflow execution library. We face several problems in serializing workflows:
- workflows may contain arbitrary data whose serialization mechanisms cannot be built into the library itself
- workflows may contain custom tasks and these also cannot be built into the library
- workflows may contain hundreds of tasks, generating very large serializations
- objects contained in the workflow data might also be very large
- the serialized data needs to be stored somewhere and there is no one-size-fits-all way of doing this
In the first section of this document, we'll show how to handle the first problem.
:doc:`custom_task_spec` contains an example of handling the second problem.
In the second section of this document, we'll discuss some of the ways the remaining problems might be alleviated
though creative use of the serializer's capabilities.
.. _serializing_custom_objects:
Serializing Custom Objects
==========================
In :doc:`script_engine`, we add some custom methods and objects to our scripting environment. We create a simple
class (a :code:`namedtuple`) that holds the product information for each product.
We'd like to be able to save and restore our custom object. This code lives in :app:`spiff/product_info.py`.
.. code:: python
ProductInfo = namedtuple('ProductInfo', ['color', 'size', 'style', 'price'])
def product_info_to_dict(obj):
return {
'color': obj.color,
'size': obj.size,
'style': obj.style,
'price': obj.price,
}
def product_info_from_dict(dct):
return ProductInfo(**dct)
And in :app:`spiff/custom_object.py`:
.. code-block:: python
from SpiffWorkflow.spiff.serializer.config import SPIFF_CONFIG
from ..serializer.file import FileSerializer
registry = FileSerializer.configure(SPIFF_CONFIG)
registry.register(ProductInfo, product_info_to_dict, product_info_from_dict)
serializer = FileSerializer(dirname, registry=registry)
We don't have any custom task specs in this example, so we can use the default serializer configuration for the
module we're using. We'll use the :app:`spiff/serializer/file/serializer.py` serializer. This is a very simple
serializer -- it converts the entire workflow to the default JSON format and writes it to disk in a readable
way.
.. note::
The default :code:`BpmnWorkflowSerializer` has a `serialize_json` method that essentially does the same thing,
except without formatting the JSON. We bypass this so we can intercept the JSON-serializable representation
and write it ourselves to a location of our choosing.
We initialize a :code:`registry` using the serializer; this registry contains the conversions for the objects
used workflow-internally.
Now we can add our custom serialization methods to this registry using the :code:`registry.register` method. The
arguments here are:
- the class that requires serialization
- a method that creates a dictionary representation of the object
- a method that recreates the object from that representation
Registering an object sets up relationships between the class and the serialization and deserialization methods.
The :code:`register` method assigns a :code:`typename` for the class, and generates partial functions that call the
appropriate methods based on the :code:`typename`, and stores *these* conversion mechanisms.
.. note::
The supplied :code:`to_dict` and :code:`from_dict` methods must always return and accept dictionaries, even if
they might have been serialized some other way.
If you're interested in how this works, the heart of the registry is the
`DictionaryConverter <https://github.com/sartography/SpiffWorkflow/blob/main/SpiffWorkflow/bpmn/serializer/helpers/dictionary.py>`_.
The price is a slightly less customizable serialized format; the benefit is that these partial functions can
replace humongous :code:`if/else` blocks that test for specific classes and attributes.
Optimizing Serializations
=========================
File Serializer
---------------
Now we'll turn to the customizations we made in the :app:`serializer/file/serializer.py`.
We've extended the :code:`BpmnWorkflowSerializer` to take a directory where we'll write our files, and additionally
we'll impose some structure inside this dictionary. We'll separate serialized workflow specs from instance data, and
set an output format that we can actually read.
Our engine requires a certain API from our serializer, and that's what the remainder of the methods are. We won't
go into these method here, as they don't *actually* have much to do with the library. We made few (the
:app:`spiff/custom_object.py`) or no modifications (the :app:`spiff/file.py`) so there isn't much to discuss.
We call `self.to_dict` and `self.from_dict`, which handle all conversions based on how we've set up the
:code:`registry`.
.. note::
We haven't referenced any particular code, as almost all the code here is about managing our directory
structure and formatting the JSON output appropriately.
The file serializer is actually *not* particularly optimized, but it is simple to understand, while also providing
the evidence that you probably want to do more. The output here is essentially the what you get by default. This
useful to be able to easily see in and of itself, and if you examine it, you'll see there would be a lot of
opportunity for splitting the output into its components and handling them separately.
SQLite Serializer
-----------------
We have a second example serializer that stores serializations in a SQLite database in
:app:`serializer/sqlite/serializer.py`. This might be a slightly more realistic use case of what you want to do,
so we'll discuss this in a little more detail (but it is also a considerably more complex example).
Our database schema actually takes care of much of the work, but since this isn't an SQL tutorial, I'll just refer you
to the file that contains it: :app:`serializer/sqlite/schema.sql`. Of course, you do not have to interact with the
database directly (or even use a database at all) and some of all of the triggers and views and so forth could be
replaced with Python code (or simplified quite a bit if using a more robust DB).
This is intended to be a somewhat extreme example in order to make it clear that you really aren't bound to
retrieving and storing a gigantic blob, and the logic for dealing with it does not have to be interspersed with the
rest of your code.
In addition to our triggers, we also rely pretty heavily on SQLite adapters. Between these two things, we hardly
have to worry about the types of objects we get back at all!
From our :code:`execute` method:
.. code-block:: python
conn = sqlite3.connect(self.dbname, detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
conn.execute("pragma foreign_keys=on")
sqlite3.register_adapter(UUID, lambda v: str(v))
sqlite3.register_converter("uuid", lambda s: UUID(s.decode('utf-8')))
sqlite3.register_adapter(dict, lambda v: json.dumps(v))
sqlite3.register_converter("json", lambda s: json.loads(s))
We use :code:`UUID` for spec and instance IDs and store all our workflow data as JSON. Our serializer guarantees
that its output will be JSON-serializable, so when we store it, we can just drop its output right into the DB, and
feed the DB output back into the serializer.
To help this process along, we've customized a few of the default conversions for our specs.
.. code-block:: python
class WorkflowConverter(BpmnWorkflowConverter):
def to_dict(self, workflow):
dct = super(BpmnWorkflowConverter, self).to_dict(workflow)
dct['bpmn_events'] = self.registry.convert(workflow.bpmn_events)
dct['subprocesses'] = {}
dct['tasks'] = list(dct['tasks'].values())
return dct
class SubworkflowConverter(BpmnSubWorkflowConverter):
def to_dict(self, workflow):
dct = super().to_dict(workflow)
dct['tasks'] = list(dct['tasks'].values())
return dct
class WorkflowSpecConverter(BpmnProcessSpecConverter):
def to_dict(self, spec):
dct = super().to_dict(spec)
dct['task_specs'] = list(dct['task_specs'].values())
return dct
We aren't making extensive customizations here, mainly just switching some dictionaries to lists; this is because we
store these items in separate tables, so it's convenient to get an output that can be passed directly to an
:code:`insert` statement.
When we configure our engine, we update the serializer configuration to use these classes (this code is from
:app:`spiff/sqlite.py`:
.. code-block:: python
from SpiffWorkflow.spiff.serializer import DEFAULT_CONFIG
from ..serializer.sqlite import (
SqliteSerializer,
WorkflowConverter,
SubworkflowConverter,
WorkflowSpecConverter
)
DEFAULT_CONFIG[BpmnWorkflow] = WorkflowConverter
DEFAULT_CONFIG[BpmnSubWorkflow] = SubworkflowConverter
DEFAULT_CONFIG[BpmnProcessSpec] = WorkflowSpecConverter
dbname = 'spiff.db'
with sqlite3.connect(dbname) as db:
SqliteSerializer.initialize(db)
registry = SqliteSerializer.configure(DEFAULT_CONFIG)
serializer = SqliteSerializer(dbname, registry=registry)
Finally, let's look at two of the methods we've implemented for the API required by our engine:
.. code-block:: python
def _create_workflow(self, cursor, workflow, spec_id):
dct = super().to_dict(workflow)
wf_id = uuid4()
stmt = "insert into workflow (id, workflow_spec_id, serialization) values (?, ?, ?)"
cursor.execute(stmt, (wf_id, spec_id, dct))
if len(workflow.subprocesses) > 0:
cursor.execute("select serialization->>'name', descendant from spec_dependency where root=?", (spec_id, ))
dependencies = dict((name, id) for name, id in cursor)
for sp_id, sp in workflow.subprocesses.items():
cursor.execute(stmt, (sp_id, dependencies[sp.spec.name], self.to_dict(sp)))
return wf_id
def _get_workflow(self, cursor, wf_id, include_dependencies):
cursor.execute("select workflow_spec_id, serialization as 'serialization [json]' from workflow where id=?", (wf_id, ))
row = cursor.fetchone()
spec_id, workflow = row[0], self.from_dict(row[1])
if include_dependencies:
workflow.subprocess_specs = self._get_subprocess_specs(cursor, spec_id)
cursor.execute(
"select descendant as 'id [uuid]', serialization as 'serialization [json]' from workflow_dependency where root=? order by depth",
(wf_id, )
)
for sp_id, sp in cursor:
task = workflow.get_task_from_id(sp_id)
workflow.subprocesses[sp_id] = self.from_dict(sp, task=task, top_workflow=workflow)
return workflow
We store subprocesses in the same table as top level processes because they are essentially the same thing.
We maintain a table that stores the parent/child relationships in a separate spec dependency table. While we don't do
this currently, we could modify our queries to ignore subprocesses that have been completed when we retrieve a workflow:
they could potentially contain many tasks that will never be revisited. Or, conversely, we could limit what we restore
to subprocesses that had :code:`READY` tasks to avoid loading something that is waiting for a timer that will fire in
two weeks.
We did not show the code for serializing workflow specs, but it is similar -- all specs, whether top-level or for
subprocesses and call activities live in one table, with a second that keeps track of dependencies between them. This
would make it possible to wait to load a spec until the task it was associated with needed to be executed.
We also maintain task data separately from information about workflow state; so while we're not doing this now, it provides
the potential to selectively retrieve it -- for example, it could be omitted from :code:`COMPLETED` tasks.
What I aim to get across here is that there are quite a few possiblities for customizing how your application serializes
its workflows -- you're not limited to a giant JSON blob that you get by default.
Serialization Versions
======================
As we make changes to Spiff, we may change the serialization format. For example, in 1.2.1, we changed
how subprocesses were handled interally in BPMN workflows and updated how they are serialized and we upraded the
serializer version to 1.1.
Since workflows can contain arbitrary data, and even SpiffWorkflow's internal classes are designed to be customized in ways
that might require special serialization and deserialization, it is possible to override the default version number, to
provide users with a way of tracking their own changes.
If you have not provided a custom version number, SpiffWorkflow will attempt to migrate your workflows from one version
to the next if they were serialized in an earlier format.
If you've overridden the serializer version, you may need to incorporate our serialization changes with
your own. You can find our migrations in
`SpiffWorkflow/bpmn/serilaizer/migrations <https://github.com/sartography/SpiffWorkflow/tree/main/SpiffWorkflow/bpmn/serializer/migration>`_
These are broken up into functions that handle each individual change, which will hopefully make it easier to incoporate them
into your upgrade process, and also provides some documentation on what has changed.

67
doc/bpmn/supported.rst Normal file
View File

@@ -0,0 +1,67 @@
List of Supported Elements
==========================
Tasks
-----
* User Task
* Manual Task
* Business Rule Task
* Script Task
* Service Task
.. note::
Spiff's implementation of Service Tasks is abstract, so while they will be parsed, the
library provides no built-in mechanism for executing them.
Gateways
--------
* Parallel Gateway
* Exclusive Gateway
* Inclusive Gateway
* Event-Based Gateway
Subrocesses and Call Activities
-------------------------------
* Subprocess
* Call Activity
* Transaction Subprocess
Events
------
* Cancel Event
* Escalation Event
* Error Event
* Message Event
* Signal Event
* Terminate Event
* Timer Event
Data
----
* Data Object
* Data Store
.. note::
Spiff's Data Store implementation is abstract; Spiff can parse a Data Store, but does not
provide any built-in mechanism for reading and writing to it.
Loops
-----
* Loop Task
* Parallel MultiInstance Task
* Sequential MultiInstance Task
.. note::
Parallel MultiInstance tasks are *not* executed by SpiffWorkflow in parallel. SpiffWorkflow
merely indicates that parallel tasks become ready at the same time and that the workflow
engine may execute them in parallel.

321
doc/bpmn/synthesis.rst
View File

@@ -1,321 +0,0 @@
Putting it All Together
=======================
In this section we'll be discussing the overall structure of the workflow
runner we developed in `spiff-example-cli <https://github.com/sartography/spiff-example-cli>`_.
Our example application contains two different workflow runners, one that uses tasks with with Spiff extensions
(`spiff-bpmn-runner.py <https://github.com/sartography/spiff-example-cli/blob/main/spiff-bpmn-runner.py>`_)
and one that uses the **deprecated** Camunda extensions
(`camunda-bpmn-runner.py <https://github.com/sartography/spiff-example-cli/blob/main/camunda-bpmn-runner.py>`_).
The primary differences between the two are in handling User and MultiInstance Tasks. We have some documentation
about how we interpret Camunda forms in :doc:`camunda/tasks`. That particular page comes from an earlier version of
our documentation, and `camunda-bpmn-runner.py` can run workflows with these tasks. However, we are not actively
maintaining the :code:`camunda` package, and it should be considered deprecated.
Base Application Runner
-----------------------
The core functions your application will have to accomodate are
* parsing workflows
* serializing workflows
* running workflows
Task specs define how tasks are executed, and creating the task specs depends on a parser which initializes a spec of
the appropriate class. And of course serialization is also heavily dependent on the same information needed to create
the instance. To that end, our BPMN runner requires that you provide a parser and serializer; it can't operate unless
it knows what to do with each task spec it runs across.
Here is the initialization for the :code:`runner.SimpleBpmnRunner` class that is used by both scripts.
.. code:: python
def __init__(self, parser, serializer, script_engine=None, handlers=None):
self.parser = parser
self.serializer = serializer
self.script_engine = script_engine
self.handlers = handlers or {}
self.workflow = None
If you read the introduction to BPMN, you'll remember that there's a Script Task; the script engine executes scripts
against the task data and updates it. Gateway conditions are also evaluated against the same context by the engine.
SpiffWorkflow provides a default scripting environment that is suitable for simple applications, but a serious application
will probably need to extend (or restrict) it in some way. See :doc:`advanced` for a few examples. Therefore, we have the
ability to optionally pass one in.
The `handlers` argument allows us to let our application know what to do with specific task spec types. It's a mapping
of task spec class to its handler. Most task specs won't need handlers outside of how SpiffWorkflow executes them
(that's probably why you are using this library). You'll only have to be concerned with the task spec types that
require human interaction; Spiff will not handle those for you. In your application, these will probably be built into
it and you won't need to pass anything in.
However, here we're trying to build something flexible enough that it can at least deal with two completely different
mechanisms for handling User Tasks, and provide a means for you to experiment with this application.
Parsing Workflows
-----------------
Here is the method we use to parse the workflows;
.. code:: python
def parse(self, name, bpmn_files, dmn_files=None, collaboration=False):
self.parser.add_bpmn_files(bpmn_files)
if dmn_files:
self.parser.add_dmn_files(dmn_files)
if collaboration:
top_level, subprocesses = self.parser.get_collaboration(name)
else:
top_level = self.parser.get_spec(name)
subprocesses = self.parser.get_subprocess_specs(name)
self.workflow = BpmnWorkflow(top_level, subprocesses, script_engine=self.script_engine)
We add the BPMN and DMN files to the parser and use :code:`parser.get_spec` to create a workflow spec for a process
model.
SpiffWorkflow needs at least one spec to create a workflow; this will be created from the name of the process passed
into the method. It also needs specs for any subprocesses or call activities. The method
:code:`parser.get_subprocess_specs` will search recursively through a starting spec and collect specs for all
referenced resources.
It is possible to have two processes defined in a single model, via a Collaboration. In this case, there is no "top
level spec". We can use :code:`self.parser.get_collaboration` to handle this case.
.. note::
The only required argument to :code:`BpmnWorkflow` is a single workflow spec, in this case `top_level`. The
parser returns an empty dict if no subprocesses are present, but it is not required to pass this in. If there
are subprocess present, `subprocess_specs` will be a mapping of process ID to :code:`BpmnWorkflowSpec`.
In :code:`simple_bpmn_runner.py` we create the parser like this:
.. code:: python
from SpiffWorkflow.spiff.parser.process import SpiffBpmnParser, BpmnValidator
parser = SpiffBpmnParser(validator=BpmnValidator())
The validator is an optional argument, which can be used to validate the BPMN files passed in. The :code:`BpmnValidator`
in the :code:`spiff` package is configured to validate against the BPMN 2.0 spec and our spec describing our own
extensions.
The parser we imported is pre-configured to create task specs that know about Spiff extensions.
There are parsers in both the :code:`bpmn` and :code:`camunda` packages that can be similarly imported. There is a
validator that uses only the BPMN 2.0 spec in the :code:`bpmn` package (but no similar validator for Camunda).
It is possible to override particular task specs for specific BPMN Task types. We'll cover an example of this in
:doc:`advanced`.
Serializing Workflows
---------------------
In addition to the pre-configured parser, each package has a pre-configured serializer.
.. code:: python
from SpiffWorkflow.spiff.serializer.config import SPIFF_SPEC_CONFIG
from runner.product_info import registry
wf_spec_converter = BpmnWorkflowSerializer.configure_workflow_spec_converter(SPIFF_SPEC_CONFIG)
serializer = BpmnWorkflowSerializer(wf_spec_converter, registry)
The serializer has two components:
* the `workflow_spec_converter`, which knows about objects inside SpiffWorkflow
* the `registry`, which can tell SpiffWorkflow how to handle arbitrary data from your scripting environment
(required only if you have non-JSON-serializable data there).
We discuss the creation and use of `registry` in :doc:`advanced` so we'll ignore it for now.
`SPIFF_SPEC_CONFIG` has serialization methods for each of the task specs in its parser and we can create a
converter from it directly and pass it into our serializer.
Here is our deserialization code:
.. code:: python
def restore(self, filename):
with open(filename) as fh:
self.workflow = self.serializer.deserialize_json(fh.read())
if self.script_engine is not None:
self.workflow.script_engine = self.script_engine
We'll just pass the contents of the file to the serializer and it will restore the workflow. The scripting environment
was not serialized, so we have to make sure we reset it.
And here is our serialization code:
.. code:: python
def dump(self):
filename = input('Enter filename: ')
with open(filename, 'w') as fh:
dct = self.serializer.workflow_to_dict(self.workflow)
dct[self.serializer.VERSION_KEY] = self.serializer.VERSION
fh.write(json.dumps(dct, indent=2, separators=[', ', ': ']))
The serializer has a companion method :code:`serialize_json` but we're bypassing that here so that we can make the
output readable.
The heart of the serialization process actually happens in :code:`workflow_to_dict`. This method returns a
dictionary representation of the workflow that contains only JSON-serializable items. All :code:`serialize_json`
does is add a serializer version and call :code:`json.dumps` on the returned dict. If you are developing a serious
application, it is unlikely you want to store the entire workflow as a string, so you should be aware that this method
exists.
The serializer is fairly complex: not only does it need to handle SpiffWorkflow's own internal objects that it
knows about, it needs to handle arbitrary Python objects in the scripting environment. The serializer is covered in
more depth in :doc:`advanced`.
Defining Task Handlers
----------------------
In :code:`spiff-bpmn-runner.py`, we also define the functions :code:`complete_user_task`. and
:code:`complete_manual_task`.
We went over these handlers in :doc:`tasks`, so we won't delve into them here.
We create a mapping of task type to handler, which we'll pass to our workflow runner.
.. code:: python
handlers = {
UserTask: complete_user_task,
ManualTask: complete_manual_task,
NoneTask: complete_manual_task,
}
In SpiffWorkflow the :code:`NoneTask` (which corresponds to the `bpmn:task` is treated as a human task, and therefore
has no built in way of handling them. Here we treat them as if they were Manual Tasks.
Running Workflows
-----------------
Our application's :code:`run_workflow` method takes one argument: `step` is a boolean that lets the runner know
if if should stop and present the menu at every step (if :code:`True`) or only where there are human tasks to
complete.
.. code:: python
def run_workflow(self, step=False):
while not self.workflow.is_completed():
if not step:
self.advance()
tasks = self.workflow.get_tasks(TaskState.READY|TaskState.WAITING)
runnable = [t for t in tasks if t.state == TaskState.READY]
human_tasks = [t for t in runnable if t.task_spec.manual]
current_tasks = human_tasks if not step else runnable
self.list_tasks(tasks, 'Ready and Waiting Tasks')
if len(current_tasks) > 0:
action = self.show_workflow_options(current_tasks)
else:
action = None
if len(tasks) > 0:
input("\nPress any key to update task list")
In the code above we first get the list of all `READY` or `WAITING` tasks; these are the currently active tasks.
`READY` tasks can be run, and `WAITING` tasks may change to `READY` (see :doc:`../concepts` for a discussion of task
states). We aren't going to do anything with the `WAITING` tasks except display them.
We can further filter our runnable tasks on the :code:`task_spec.manual` attribute. If we're stepping though the
workflow, we'll present the entire list; otherwise only the human tasks. There are actually many points where no
human tasks are available to execute; the :code:`advance` method runs the other runnable tasks if we've opted to
skip displaying them; we'll look at that method after this one.
There may also be points where there are no runnable tasks at all (for example, if the entire process is waiting
on a timer). In that case, we'll do nothing until the user indicates we can proceeed (the timer will fire
regardless of what the user does -- we're just preventing this loop from executing repeatedly when there's nothing
to do).
.. code:: python
if action == 'r':
task = self.select_task(current_tasks)
handler = self.handlers.get(type(task.task_spec))
if handler is not None:
handler(task)
task.run()
In the code above, we present a menu of runnable tasks to the user and run the one they chose, optionally
calling one of our handlers.
Each task has a `data` attribute, which can by optionally updated when the task is `READY` and before it is
run. The task `data` is just a dictionary. Our handler modifies the task data if necessary (eg adding data
collected from forms), and :code:`task.run` propogates the data to any tasks following it, and changes its state to
one of the `FINISHED` states; nothing more will be done with this task after this point.
We'll skip over most of the options in :code:`run_workflow` since they are pretty straightforward.
.. code:: python
self.workflow.refresh_waiting_tasks()
At the end of each iteration, we call :code:`refresh_waiting_tasks` to ensure that any currently `WAITING` tasks
will move to `READY` if they are able to do so.
After the workflow finishes, we'll give the user a few options for looking at the end state.
.. code:: python
while action != 'q':
action = self.show_prompt('\nSelect action: ', {
'a': 'List all tasks',
'v': 'View workflow data',
'q': 'Quit',
})
if action == 'a':
self.list_tasks([t for t in self.workflow.get_tasks() if t.task_spec.bpmn_id is not None], "All Tasks")
elif action == 'v':
dct = self.serializer.data_converter.convert(self.workflow.data)
print('\n' + json.dumps(dct, indent=2, separators=[', ', ': ']))
Note that we're filtering the task lists with :code:`t.task_spec.bpmn_id is not None`. The workflow contains
tasks other than the ones visible on the BPMN diagram; these are tasks that SpiffWorkflow uses to manage execution
and we'll omit them from the displays. If a task is visible on a diagram it will have a non-null value for its
`bpmn_id` attribute (because all BPMN elements require IDs), otherwise the value will be :code:`None`. See
:doc:`advanced` for more information about BPMN task spec attributes.
When a workflow completes, the task data from the "End" task, which has built up through the operation of the
workflow, is copied into the workflow data, so we want to give the option to display this end state. We're using
the serializer's `data_converter` to handle the workflow data (the `registry`) we passed in earlier, because
it may contain arbitrary data.
Let's take a brief look at the advance method:
.. code:: python
def advance(self):
engine_tasks = [t for t in self.workflow.get_tasks(TaskState.READY) if not t.task_spec.manual]
while len(engine_tasks) > 0:
for task in engine_tasks:
task.run()
self.workflow.refresh_waiting_tasks()
engine_tasks = [t for t in self.workflow.get_tasks(TaskState.READY) if not t.task_spec.manual]
This method is really just a condensed version of :code:`run_workflow` that ignore human tasks and doesn't need to
present a menu. We use it to get to a point in our workflow where there are only human tasks left to run.
In general, an application that uses SpiffWorkflow will use these methods as a template. It will consist of a
loop that:
* runs any `READY` engine tasks (where :code:`task_spec.manual == False`)
* presents `READY` human tasks to users (if any)
* updates the human task data if necessary
* runs the human tasks
* refreshes any `WAITING` tasks
until there are no tasks left to complete.
The rest of the code is all about presenting the tasks to the user and dumping the workflow state. These are the
parts that you'll want to customize in your own application.

234
doc/bpmn/tasks.rst
View File

@@ -1,234 +0,0 @@
Tasks
=====
BPMN Model
----------
In this example, we'll model a customer selecting a product to illustrate the basic task types that
can be used with SpiffWorkflow.
We'll be using the following files from `spiff-example-cli <https://github.com/sartography/spiff-example-cli>`_:
- `task_types <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/task_types.bpmn>`_ workflow
- `product_prices <https://github.com/sartography/spiff-example-cli/blob/main/bpmn/tutorial/product_prices.dmn>`_ DMN table
User Tasks
^^^^^^^^^^
User Tasks would typically be used in the case where the task would be completed from within the
application. Our User tasks present forms that collect data from users.
We'll ask our hypothetical user to choose a product and quantity.
.. figure:: figures/tasks/user_task.png
:scale: 30%
:align: center
User Task
We can use the form builder to create the form.
.. figure:: figures/tasks/user_task_form.png
:scale: 30%
:align: center
User Task form
See the `Handling User Tasks`_ section for a discussion of sample code.
We have also retained some limited support for the now deprecated
camunda forms, which you can read about in our Camunda Specific section on :doc:`camunda/tasks`.
Business Rule Tasks
^^^^^^^^^^^^^^^^^^^
In our Business Rule Task, we'll use a DMN table to look up the price of the
product the user chose.
We'll need to create a DMN table.
What is DMN?
++++++++++++
Decision Model and Notation (DMN) is a standard for business decision
modeling. DMN allows modelers to separate decision logic from process logic
and maintain it in a table format. DMN is linked into BPMN with a *decision
task*.
With DMN, business analysts can model the rules that lead to a decision
in an easy to read table. Those tables can be executed directly by SpiffWorkflow.
This minimizes the risk of misunderstandings between business analysts and
developers, and allows rapid changes in production.
BPMN includes a decision task that refers to the decision table. The outcome of
the decision lookup allows the next gateway or activity to route the flow.
Our Business Rule Task will make use of a DMN table.
.. figure:: figures/tasks/dmn_table.png
:scale: 30%
:align: center
DMN Table
.. note::
We add quote marks around the product names in the table. Spiff will
create an expression based on the exact contents of the table, so if
the quotes are omitted, the content will be interpreted as a variable
rather than a string.
Then we'll refer to this table in the task configuration.
.. figure:: figures/tasks/business_rule_task.png
:scale: 30%
:align: center
Business Rule Task configuration
Script Tasks
^^^^^^^^^^^^
The total order cost will need to be calculated on the fly. We can do this in
a Script Task. We'll configure the task with some simple Python code.
.. figure:: figures/tasks/script_task.png
:scale: 30%
:align: center
Script Task configuration
The code in the script will have access to the task data, so variables that
have been defined previously will be available to it.
Manual Tasks
^^^^^^^^^^^^
Our final task type is a Manual Task. Manual Tasks represent work that occures
outside of SpiffWorkflow's control. Say that you need to include a step in a
process where the participant needs to stand up, walk over to the coffee maker,
and poor the cup of coffee. Manual Tasks pause the process, and wait for
confirmation that the step was completed.
Text that will be displayed to the user is added in the "Instructions" panel.
.. figure:: figures/tasks/manual_task.png
:scale: 30%
:align: center
Manual Task
Spiff's manual tasks may contain references to data inside the workflow. We have used
`Jinja <https://jinja.palletsprojects.com/en/3.0.x/>`_, but Spiff is set up in a way that
you could use any templating library you want, as well as Markdown formatting directives
(we won't implement those here though, because it doesn't make sense for a command
line app).
.. figure:: figures/tasks/manual_task_instructions.png
:scale: 30%
:align: center
Editing Instructions
See the `Handling Manual Tasks`_ section for a discussion of sample code.
For information about how Spiff handles Manual Tasks created with Camunda please
refer to the Camunda Specific section on :doc:`camunda/tasks`.
Running The Model
^^^^^^^^^^^^^^^^^
If you have set up our example repository, this model can be run with the following command:
.. code-block:: console
./spiff-bpmn-runner.py -p order_product -d bpmn/tutorial/product_prices.dmn -b bpmn/tutorial/task_types.bpmn
Example Application Code
------------------------
Handling User Tasks
^^^^^^^^^^^^^^^^^^^
We will need to provide a way to display the form data and collect the user's
responses.
.. code:: python
filename = task.task_spec.extensions['properties']['formJsonSchemaFilename']
schema = json.load(open(os.path.join(forms_dir, filename)))
for field, config in schema['properties'].items():
if 'oneOf' in config:
option_map = dict([ (v['title'], v['const']) for v in config['oneOf'] ])
options = "(" + ', '.join(option_map) + ")"
prompt = f"{field} {options} "
option = input(prompt)
while option not in option_map:
print(f'Invalid selection!')
option = input(prompt)
response = option_map[option]
else:
response = input(f"{config['title']} ")
if config['type'] == 'integer':
response = int(response)
task.data[field] = response
SpiffWorkflow uses JSON Schema to represent forms, specifically
`react-jsonschema-form <https://react-jsonschema-form.readthedocs.io/en/latest/>`_.
Our forms are really intended to be displayed in a browser, and attempting to handle them in a command
line appliction is a little awkward. The form specifications can be quite complex.
This simple implementation will present a list of options for simple enumerated fields and simply
directly stores whatever the user enters otherwise, with integer conversions if the field is so
specified. This is robust enough to collect enough information from a user to make it through our example.
SpiffWorkflow provides a mechanism for you to provide your own form specification and leaves it up to you
to decide how to present it.
Handling Business Rule Tasks
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We do not need to do any special configuration to handle these Business Rule Tasks. SpiffWorkflow does it all for us.
Handling Script Tasks
^^^^^^^^^^^^^^^^^^^^^
We do not need to do any special configuration to handle Script Tasks, although it
is possible to implement a custom script engine. We demonstrate that process in
Custom Script Engines section :doc:`advanced` features. However, the default script
engine will be adequate for now.
Handling Manual Tasks
^^^^^^^^^^^^^^^^^^^^^
Our code for manual tasks simply asks the user to confirm that the task has been
completed.
.. code:: python
def complete_manual_task(task):
display_instructions(task)
input("Press any key to mark task complete")
:code:`display_instructions` handles presenting the task to the user.
.. code:: python
def display_instructions(task):
text = task.task_spec.extensions.get('instructionsForEndUser')
print(f'\n{task.task_spec.bpmn_name}')
if text is not None:
template = Template(text)
print(template.render(task.data))
The template string can be obtained from :code:`task.task_spec.extensions.get('instructionsForEndUser')`.
As noted above, our template class comes from Jinja. We render the template
using the task data, which is just a dictionary.
.. note::
Most of Spiff's task specifications contain this extension, not just Manual Tasks. We also use it to display
information along with forms, and about certain events.

375
doc/bpmn/workflows.rst Normal file
View File

@@ -0,0 +1,375 @@
Instantiating a Workflow
========================
From the :code:`start_workflow` method of our BPMN engine (:app:`engine/engine.py`):
.. code-block:: python
def start_workflow(self, spec_id):
spec, sp_specs = self.serializer.get_workflow_spec(spec_id)
wf = BpmnWorkflow(spec, sp_specs, script_engine=self._script_engine)
wf_id = self.serializer.create_workflow(wf, spec_id)
return wf_id
We'll use our serializer to recreate the workflow spec based on the id. As discussed in :ref:`parsing_subprocesses`,
a process has a top level specification and dictionary of process id -> spec containing any other processes referenced
by the top level process (Call Actitivies and Subprocesses).
Running a Workflow
==================
In the simplest case, running a workflow involves implementing the following loop:
* runs any `READY` engine tasks (where :code:`task_spec.manual == False`)
* presents `READY` human tasks to users (if any)
* updates the human task data if necessary
* runs the human tasks
* refreshes any `WAITING` tasks
until there are no tasks left to complete.
Here are our engine methods:
.. code-block:: python
def run_until_user_input_required(self, workflow):
task = workflow.get_next_task(state=TaskState.READY, manual=False)
while task is not None:
task.run()
self.run_ready_events(workflow)
task = workflow.get_next_task(state=TaskState.READY, manual=False)
def run_ready_events(self, workflow):
workflow.refresh_waiting_tasks()
task = workflow.get_next_task(state=TaskState.READY, spec_class=CatchingEvent)
while task is not None:
task.run()
task = workflow.get_next_task(state=TaskState.READY, spec_class=CatchingEvent)
In the first, we retrieve and run any tasks that can be executed automatically, including processing any events that
might have occurred.
The second method handles processing events. A task that corresponds to an event remains in state :code:`WAITING` until
it catches whatever event it is waiting on, at which point it becomes :code:`READY` and can be run. The
:code:`workflow.refresh_waiting_tasks` method iterates over all the waiting tasks and changes the state to :code:`READY`
if the conditions for doing so have been met.
We'll cover using the `workflow.get_next_task` method and handling Human tasks later in this document.
Tasks
=====
In this section, we'll give an overview of some of the general attributes of Task Specs and then delve into a few
specific types. See :ref:`specs_vs_instances` to read about Tasks vs Task Specs.
BPMN Task Specs
---------------
BPMN Task Specs inherit quite a few attributes from :code:`SpiffWorkflow.specs.base.TaskSpec`, but you probably
don't have to pay much attention to most of them. A few of the important ones are:
* `name`: the unique id of the TaskSpec, and it will correspond to the BPMN ID if that is present
* `description`: we use this attribute to provide a description of the BPMN task type
* `manual`: :code:`True` if human input is required to complete tasks associated with this Task Spec
BPMN Task Specs have the following additional attributes.
* `bpmn_id`: the ID of the BPMN Task (this will be :code:`None` if the task is not visible on the diagram)
* `bpmn_name`: the BPMN name of the Task
* `lane`: the lane of the BPMN Task
* `documentation`: the contents of the BPMN `documentation` element for the Task
In the example application, we use these :code:`bpmn_name` (or :code:`name` when a :code:`bpmn_name` isn't specified),
and :code:`lane` to display information about the tasks in a workflow (see the :code:`update_task_tree` method of
:app:`curses_ui/workflow_view.py`).
The :code:`manual` attribute is particularly important, because SpiffWorkflow does not include built-in
handling of these tasks so you'll need to implement this as part of your application. We'll go over how this is
handled in this application in the next section.
.. note::
NoneTasks (BPMN tasks with no more specific type assigned) are treated as Manual Tasks by SpiffWorkflow.
Instantiated Tasks
------------------
Actually all Tasks are instantiated -- that is what distinguishes a Task from a Task Spec; however, it is impossible to
belabor this point too much.
Tasks have a few additional attributes that contain important details about particular instances:
* :code:`id`: a UUID that uniquely identifies the Task (remember that a Task Spec may be reached more than once, but a new
Task is created each time)
* :code:`task_spec`: the Task Spec associated with this Task
* :code:`state`: the state of the Task, represented as one of the values in :code:`TaskState`
* :code:`last_state_change`: the timestamp of the last time this Task changed state
* :code:`data`: a dictionary that holds task/workflow data
Human (User and Manual) Tasks
-----------------------------
Remember that the :code:`bpmn` module does not provide any default capability for gathering information from a user,
and this is something you'll have to implement. In this example, we'll assume that we are using Task Specs from the
:code:`spiff` module (there is an alternative implementation in the :code:`camunda` module).
Spiff Arena uses JSON schemas to define forms associated with User Tasks and
`react-jsonschema-form <https://github.com/rjsf-team/react-jsonschema-form>`_ to render them. Additionally, our User
and Manual tasks have a custom extension :code:`instructionsForEndUser` which stores a Jinja template with Markdown
formatting that is rendered using the task data. A different format for defing forms could be used and Jinja and
Markdown could be easily replaced by other templating and rendering schemes depending on your application's needs.
Our User and Manual Task handlers render the instructions (this code is from :app:`spiff/curses_handlers.py`):
.. code-block:: python
from jinja2 import Template
def get_instructions(self):
instructions = f'{self.task.task_spec.bpmn_name}\n\n'
text = self.task.task_spec.extensions.get('instructionsForEndUser')
if text is not None:
template = Template(text)
instructions += template.render(self.task.data)
instructions += '\n\n'
return instructions
We're not going to attempt to handle Markdown in a curses UI, so we'll assume we just have text. However, we do
want to be able to incorporate data specific to the workflow in information that is presented to a user; this is
something that your application will certainly need to do. Here, we use the :code:`data` attribute of the Task
(recall that this is a dictionary) to render the template.
Our application contains a :code:`Field` class (defined in :app:`curses_ui/user_input.py`) that tells us
how to convert to and from a string representation that can be displayed on the screen and can interact with the form
display screen. Our User Task handler also has a method for translating a couple of basic JSON schema types into
something that can be displayed (supporting only text, integers, and 'oneOf'). The form screen collects and validates
the user input and collects the results in a dictionary.
We won't go into the details about how the form screen works, as it's specific to this application rather than the
library itself; instead we'll skip to the code that runs the task after it has been presented to the user; any
application needs to do this.
Simply running the task is sufficient for Manual Tasks.
.. code-block:: python
def on_complete(self, results):
self.task.run()
However, we need to extend this method for User Tasks, to incorporate the user-submitted data into the workflow:
.. code-block:: python
def on_complete(self, results):
self.task.set_data(**results)
super().on_complete(results)
Here we are setting a key for each field in the form. Other possible options here are to set one key that contains
all of the form data, or map the schema to Python class and use that in lieu of a dictionary. It's up to you to
decide the best way of managing this.
The key points here are that your application will need to have the capability to display information, potentially
incorporating data from the workflow instance, as well as update this data based on user input. We'll go through a
simple example next.
We'll refer to the process modeled in :bpmn:`task_types.bpmn` contains a simple form which asks a user to input a
product and quantity as well a manual task presenting the order information at the end of the process (the form is
defined in :form:`select_product_and_quantity.json`
After the user submits the form, we'll collect the results in the following dictionary:
.. code-block:: python
{
'product_name': 'product_a',
'product_quantity': 2,
}
We'll add these variables to the task data before we run the task. The Business Rule task looks up the price from a
DMN table based on :code:`product_name` and the Script Task sets :code:`order_total` based on the price and quantity.
Our Manual Task's instructions look like this:
.. code-block::
Order Summary
{{ product_name }}
Quantity: {{ product_quantity }}
Order Total: {{ order_total }}
and when rendered against the instance data, reflects the details of this particular order.
Business Rule Tasks
-------------------
Business Rule Tasks are not implemented in the :code:`SpiffWorkflow.bpmn` module; however, the library does contain
a DMN implementation of a Business Rule Task in the :code:`SpiffWorkflow.dmn` module. Both the :code:`spiff` and
:code:`camunda` modules include DMN support.
Gateways
--------
You will not need special code to handle gateways (this is one of the things this library does for you), but it is
worth emphasizing that gateway conditions are treated as Python expressions which are evaluated against the context of
the task data. See :doc:`script_engine` for more details.
Script and Service Tasks
------------------------
See :doc:`script_engine` for more information about how Spiff handles these tasks. There is no default Service Task
implementation, but we'll go over an example of one way this might be implemented there. Script tasks assume the
:code:`script` attribute contains the text of a Python script, which is executed in the context of the task's data.
.. _task_filters:
Filtering Tasks
===============
SpiffWorkflow has two methods for retrieving tasks:
- :code:`workflow.get_tasks`: returns a list of matching tasks, or an empty list
- :code:`workflow.get_next_task`: returns the first matching task, or None
Both of these methods use the same helper classes and take the same arguments -- the only difference is the return
type.
These methods return a :code:`TaskIterator`, which in turn uses a :code:`TaskFilter` to determine what tasks match.
Tasks can be filtered by:
- :code:`state`: a :code:`TaskState` value (see :ref:`states` for the possible states)
- :code:`spec_name`: the name of a Task Spec (this will typically correspond to the BPMN ID)
- :code:`manual`: whether the Task Spec requires manual input
- :code:`updated_ts`: limits results to after the provided timestamp
- :code:`spec_class`: limits results to a particular Task Spec class
- :code:`lane`: the lane of the Task Spec
- :code:`catches_event`: Task Specs that catch a particular :code:`BpmnEvent`
Examples
--------
We reference the following processes here:
- :bpmn:`top_level.bpmn`
- :bpmn:`call_activity.bpmn`
To filter by state, We need to import the :code:`TaskState` object (unless you want to memorize which numbers
correspond to which states).
.. code-block:: python
from SpiffWorkflow.util.task import TaskState
Ready Human Tasks
^^^^^^^^^^^^^^^^^
.. code-block:: python
tasks = workflow.get_tasks(state=TaskState.READY, manual=False)
Completed Tasks
^^^^^^^^^^^^^^^
.. code-block:: python
tasks = workflow.get_tasks(state=TaskState.COMPLETED)
Tasks by Spec Name
^^^^^^^^^^^^^^^^^^
.. code-block:: python
tasks = workflow.get_tasks(spec_name='customize_product')
will return a list containing the Call Activities for the customization of a product in our example workflow.
Tasks Updated After
^^^^^^^^^^^^^^^^^^^
.. code-block:: python
ts = datetime.now() - timedelta(hours=1)
tasks = workflow.get_tasks(state=TaskState.WAITING, updated_ts=ts)
Returns Tasks that changed to :code:`WAITING` in the past hour.
Tasks by Lane
^^^^^^^^^^^^^
.. code:: python
ready_tasks = workflow.get_tasks(state=TaskState.READY, lane='Customer')
will return only Tasks in the 'Customer' lane in our example workflow.
Subprocesses and Call Activities
================================
In the first section of this document, we noted that :code:`BpmnWorkflow` is instantiated with a top level spec as
well as a collection of specs for any referenced processes. The instantiated :code:`BpmnSubWorkflows` are maintained
as mapping of :code:`task.id` to :code:`BpmnSubworkflow` in the :code:`subprocesses` attribute.
Both classes inherit from :code:`Workflow` and maintain tasks in separate task trees. However, only
:code:`BpmnWorkflow` maintains subworkflow information; even deeply nested workflows are stored at the top level (for
ease of access).
Task iteration also works differently as well. :code:`BpmnWorkflow.get_tasks` has been extended to retrieve
subworkflows associated with tasks and iterate over those as well; when iterating over tasks in a
:code:`BpmnSubWorkflow`, only tasks from that workflow will be returned.
.. code-block:: python
task = workflow.get_next_task(spec_name='customize_product')
subprocess = workflow.get_subprocess(task)
subprocess_tasks = subprocess.get_tasks()
This code block finds the first product customization of our example workflow and gets only the tasks inside that
workflow.
A :code:`BpmnSubworkflow` always uses the top level workflow's script engine, to ensure consistency.
Additionally, the class has a few extra attributes to make it more convenient to navigate across nested workflows:
- :code:`subworkflow.top_workflow` returns the top level workflow
- :code:`subworkflow.parent_task_id` returns the UUID of the task the workflow is associated with
- :code:`parent_workflow`: returns the workflow immediately above it in the stack
These methods exist on the top level workflow as well, and return :code:`None`.
Events
======
BPMN Events are represented by :code:`BpmnEvent` class. An instance of this class contains an :code:`EventDefinition`,
an optional payload, message correlations for Messages that define them, and (also optionally) a target subworkflow.
The last property is used internally by SpiffWorkflow by subworkflows that need to communicate with other subworkflows
and can be safely ignored.
The relationship between the :code:`EventDefinition` and :code:`BpmnEvent` is analagous to that of :code:`TaskSpec`
and :code:`Task`: a :code:`TaskSpec` defining a BPMN Event has an additional :code:`event_definition` attribute that
contains the information about the Event that will be caught or thrown.
When an event is thrown, a :code:`BpmnEvent` will be created using the :code:`EventDefinition` associated with the
task's spec, and payload, if applicable. For events with payloads, the :code:`EventDefinition` will define how to
create the payload based on the workflow instance and include this with the event. A Timer Event will know how to
parse and evaluate the provided expression. And so forth.
The event will be passed to the :code:`workflow.catch` method, which will iterate over the all the tasks and pass the
event to any tasks that are waiting for that event. If no tasks that catch the event are present in the workflow, the
event will placed in a pending event queue and these events can be retrieved with the :code:`workflow.get_events`
method.
.. note::
This method clears the event queue, so if your application retrieves the event and does not handle it, it is gone
forever!
The application in this repo is designed to run single workflows, so it does not have any external event handling.
If you implement such functionality, you'll need a way of identifying which processes any retrieved events should be
sent to.
The :code:`workflow.waiting_events` will return a list of :code:`PendingBpmnEvents`, which contain the name and type
of event and might be used to help determine this.
Once you have determined which workflow should receive the event, you can pass it to :code:`workflow.catch` to handle
it.

45
doc/concepts.rst
View File

@@ -1,13 +1,25 @@
SpiffWorkflow Concepts
======================
Fundamental SpiffWorkflow Concepts
==================================
Overview
--------
The purpose of this library is to provide a general workflow execution enviroment with a wide variety of built-in task
types that support many different workflow patterns.
SpiffWorkflow keeps track of task dependencies and states and provides the ability to serialize or deserialize a
workflow that has not been completed. Developers using this library can focus on displaying a workflow’s state and
presenting its tasks to users of their application.
.. _specs_vs_instances:
Specifications vs. Instances
----------------------------
SpiffWorkflow consists of two different categories of objects:
- **Specification objects**, which represent the definitions and derive from :code:`WorkflowSpec` and :code:`TaskSpec`
- **Instance objects**, which represent the state of a running workflow (:code:`Workflow`, :code:`BpmnWorkflow` and :code:`Task`)
- **Specification objects**, which represent definitions of structure and behavior and derive from :code:`WorkflowSpec` and :code:`TaskSpec`
- **Instance objects**, which represent the state of a running workflow (:code:`Workflow`/:code:`BpmnWorkflow` and :code:`Task`)
In the workflow context, a specification is model of the workflow, an abstraction that describes *every path that could
be taken whenever the workflow is executed*. An instance is a particular instantiation of a specification. It describes *the
@@ -23,21 +35,28 @@ Specifications are unique, whereas instances are not. There is *one* model of a
Imagine a workflow with a loop. The loop is defined once in the specification, but there can be many tasks associated with
each of the specs that comprise the loop.
In our BPMN example, described a product selection process.::
In our BPMN example (:ref:`quickstart`), we described a product selection process:
Start -> Select and Customize Product -> Continue Shopping?
Since the customer can potentially select more than one product, how our instance looks depends on the customer's actions. If
they choose three products, then we get the following tree::
they choose three products, then we get the following path::
Start --> Select and Customize Product -> Continue Shopping?
|-> Select and Customize Product -> Continue Shopping?
Start --> Select and Customize Product -> Continue Shopping? -> |
/-------------------------------------------------------- /
|-> Select and Customize Product -> Continue Shopping? -> |
/-------------------------------------------------------- /
|-> Select and Customize Product -> Continue Shopping?
There is *one* TaskSpec describing product selection and customization and *one* TaskSpec that determines whether to add more
items, but it may execute any number of imes, resulting in as many Tasks for these TaskSpecs as the number of products the
items, but it may execute any number of times, resulting in as many Tasks for these TaskSpecs as the number of products the
customer selects.
A Task Spec may have multiple inputs (if there are multiple paths to reach it) but a Task has only one parent. A specification
may contains cycles, but an instantiated workflow is *always* a tree.
.. _states:
Understanding Task States
-------------------------
@@ -52,7 +71,7 @@ Understanding Task States
* **DEFINITE** Tasks
Definite tasks are certain to run as the workflow pregresses.
Definite tasks are certain to run as the workflow progresses.
- **FUTURE**: The task will definitely run.
- **WAITING**: A condition must be met before the task can become **READY**
@@ -67,13 +86,11 @@ Understanding Task States
- **ERROR**: The task finished unsucessfully.
- **CANCELLED**: The task was cancelled before it ran or while it was running.
Tasks start in either a **PREDICTED** or **FUTURE** state, move through one or more **DEFINITE** states, and end in a
**FINISHED** state. State changes are determined by several task spec methods:
**FINISHED** state. State changes are determined by task spec methods.
Hooks
=======
-----
SpiffWorkflow executes a Task by calling a series of hooks that are tightly coupled
to Task State. These hooks are:

19
doc/conf.py
View File

@@ -18,24 +18,33 @@
# -- Project information -----------------------------------------------------
project = 'SpiffWorkflow'
copyright = '2023, Sartography'
copyright = '2024, Sartography'
author = 'Sartography'
# The full version, including alpha/beta/rc tags
release = '2.0.0rc0'
release = '3.0.0rc0'
# -- General configuration ---------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.autodoc', # 'sphinx.ext.coverage',
extensions = ['sphinx.ext.autodoc',
'sphinx.ext.viewcode',
'sphinx.ext.autosummary',
'sphinx.ext.extlinks',
'sphinx_rtd_theme',
#'sphinx.ext.intersphinx',
]
# Configure links to example repo
branch = 'improvement/better-interactive-workflow-runner'
extlinks = {
'example': (f'https://github.com/sartography/spiff-example-cli/tree/{branch}/' + '%s', '%s'),
'bpmn': (f'https://github.com/sartography/spiff-example-cli/tree/{branch}/bpmn/tutorial/' + '%s', '%s'),
'form': (f'https://github.com/sartography/spiff-example-cli/tree/{branch}/bpmn/tutorial/forms/' + '%s', '%s'),
'app': (f'https://github.com/sartography/spiff-example-cli/tree/{branch}/spiff_example/' + '%s', '%s'),
}
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
@@ -55,7 +64,7 @@ html_theme = "sphinx_rtd_theme"
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
#html_static_path = ['_static']
# Set the master index file.
master_doc = 'index'

7
doc/core/index.rst
View File

@@ -1,10 +1,9 @@
Core Library
============
SpiffWorkflow's BPMN support is built on top of a core library that aims to be a general workflow
execution environment. Workflow specifications can be created from a simple XML format, or even
easily in python code. It supports a wide range of task specifications and workflow patterns, making
it amenable to adaptation to many different schemas for defining workflow behavior.
Workflow specifications can be created from a simple XML format, or even directly in python code. The library includes
a wide range of task specifications and supports many different workflow patterns, making it amenable to adaptation to
many different schemas for defining workflow behavior.
.. toctree::
:maxdepth: 2

23
doc/index.rst
View File

@@ -20,15 +20,17 @@ What is SpiffWorkflow?
**SpiffWorkflow is a library that provides a flexible workflow execution environment.**
Recent development has largely focused on allowing Python applications to process
BPMN diagrams (think of them as very powerful flow charts; see :doc:`bpmn/intro`). to
accomplish what would otherwise require writing a lot of complex business logic in your
code. You can use these diagrams to accomplish a number of tasks, such as:
SpiffWorkflow is the workflow library underlying `Spiff Arena <https://github.com/sartography/spiff-arena>`_.
- Creating a questionnaire with multiple complex paths
- Implement an approval process that requires input from multiple users
- Allow non-programmers to modify the flow and behavior of your application
- Visualize and manage long running data processing tasks
It consists of a generic core library, with packages supporting parsing and execution of BPMN diagrams that extend
this core.
Extensive documentation about BPMN and how SpiffWorkflow interprets it, as well as information about custom extensions
implemented in the :code:`spiff` package can be found in the
`Spiff Arena documentation <https://spiff-arena.readthedocs.io/en/latest/>`_. If you are not familiar with BPMN, you
should start there. If you are looking for a full-fledged BPMN application, you can start and end there. This
documentation will focus on the library itself and is geared towards developers who are building their own
applications.
Please visit `SpiffWorkflow.org <https://www.spiffworkflow.org>`_ for
additional articles, videos, and tutorials about SpiffWorkflow and its
@@ -54,8 +56,7 @@ Contents
.. toctree::
:maxdepth: 2
bpmn/overview
bpmn/intro
concepts
modules
bpmn/index
core/index

54
doc/modules.rst Normal file
View File

@@ -0,0 +1,54 @@
Modules
=======
SpiffWorkflow consists several modules. The modules included in this package and reasons for its structure are historical.
The library originally consisted of only the core library specs; it was later extended to provide BPMN support (in a way
that did not entirely fit in with the operation of the original library); then this BPMN support was further refined to
provide support for custom extensions to the BPMN spec. Recent redevlopment has mainly focused on BPMN, with minimal
changes to the core library. Therefore there are fairly significant differences between the BPMN and non-BPMN parts of the
library. We're working towards making these consistent!
The Core Library
----------------
The core library provides basic implementations of a large set of task spec types and basic workflow execution
capabilities.
- Specs implementations are in :code:`specs`
- Workflow implementation is in :code:`workflow.py`
- Task implementation is in :code:`task.py`, with utilities for iteration and filtering in :code:`util.task.py`
It is documented in :doc:`core/index`.
Generic BPMN Implementation
---------------------------
This module extends the core implementation to support the parsing and execution of BPMN diagrams.
- The base specs of the core library are extended to implement generic BPMN attributes and behavior in
:code:`bpmn.specs`. Task specs are extended in two ways: to provide generic behavior common to all BPMN tasks
(:code:`bpmn.specs.mixins.bpmn_spec_mixin.BpmnSpecMixin`) and type-specific behavior (other task specs in
:code:`bpmn.specs.mixins`). The reason is to allow either category of properties to be extended separately.
- The workflow implementation in the BPMN package handles subworkflows in an entirely different way from the core
library. It also allows for filtering on and iterating over tasks with certain BPMN attributes.
- A workflow has a scripting environment, in which task specific code will be executed (in the core library, this
would be accomplished through an execute task that spawns a subprocess or a custom task spec).
- The serializer has been completely replaced.
This module is documented in :doc:`bpmn/index`.
Spiff BPMN Extensions
---------------------
This module extends the generic BPMN implementation with support for custom extensions used by
`Spiff Arena <https://spiff-arena.readthedocs.io/en/latest/>`_.
Camunda BPMN Extensions
-----------------------
This module extends the generic BPMN implementation with limited support for Camunda extensions.
.. warning::
The Camunda package is not under development. We'll accept contributions from Camunda users, but we are not
actively maintaining this package.