[programmer/prog-overview.tex] \section{Introduction} The {\em \dabc~ Programmer Manual} describes the aspects of the Data Acquisition Backbone Core framework that are necessary for programming user extensions. To begin with, this overview chapter explains the software objects and their collaboration, the intended mechanisms for controls and configuration, the dependencies of packages and libraries, and gives a short reference of the most important classes. The following chapters contain full explanations of the \dabc~ interface and service classes, describe the set-up with parameters, and give a reference of the Java GUI plug-in possibilities. Finally, some implementation examples are treated in detail to illustrate these issues: the adaption of the GSI legacy DAQ system \mbs~ within \dabc; the application of a distributed event builder network (Bnet); the data import via UDP from a readout controller board (ROC); and the use of a PCI express board (\ABB~). \section{Role and functionality of the objects} \subsection{Modules} \label{prog_overview_modules} All processing code runs in module objects. There are two general types of modules: \\ the \class{dabc::ModuleSync} and the \class{dabc::ModuleAsync}. \subsubsection{Class \class{dabc::ModuleSync}} \index{Core classes !dabc::ModuleSync} Each synchronous module is executed by a dedicated working thread. The thread executes a method \func{MainLoop()} with arbitrary code, which {\sl may block} the thread. In blocking calls of the framework (resource or port wait), optionally command callbacks may be executed implicitly ("non strictly blocking mode"). In the "strictly blocking mode", the blocking calls do nothing but wait. A {\sl timeout} may be set for all blocking calls; this can optionally throw an exception when the time is up. On timeout with exception, either the \func{MainLoop()} is left and the exception is then handled in the framework thread; or the \func{MainLoop()} itself catches and handles the exception. On state machine commands (e.g. \comm{Halt} or \comm{Suspend}, see section \ref{prog_fsm}), the blocking calls are also left by exception, thus putting the mainloop thread into a stopped state. \subsubsection{Class \class{dabc::ModuleAsync}} \index{Core classes !dabc::ModuleAsync} Several asynchronous modules may be run by a {\sl shared working thread}. The thread processes an {\sl event queue} and executes appropriate {\sl callback functions} of the module that is the receiver of the event. Events are fired for data input or output, command execution, and if a requested resource (e.g. memory buffer) is available. \strong{The callback functions must never block the working thread}. Instead, the callback must \strong{return} if further processing requires to wait for a requested resource. Thus each callback function must check the available resources explicitly whenever it is entered. \subsection{Commands} \index{Core classes !dabc::Command} A module may register \class{dabc::Command} objects in the constructor and may define command actions by overwriting a virtual command callback method \func{ExecuteCommand}. \subsection{Parameters} A module may register \class{dabc::Parameter} \index{Core classes !dabc::Parameter} objects. Parameters are accessible by name; their values can be monitored and optionally changed by the controls system. Initial parameter values can be set from xml configuration files. \subsection{Manager} The modules are organized and controlled by one manager object of class \class{dabc::Manager}\index{Core classes !dabc::Manager}; this singleton instance is persistent independent of the application's state. One can always access the manager via \func{dabc::mgr()} function. The manager is an \strong{object manager} that owns and keeps all registered basic objects into a folder structure. Moreover, the manager defines the \strong{interface to the control system}. This covers registering, sending, and receiving of commands; registering, updating, unregistering of parameters; error logging and global error handling. The virtual interface methods must be implemented in subclass of \class{dabc::Manager} that knows the specific controls framework. The manager receives and \strong{dispatches commands} to the destination modules where they are queued and eventually executed by the modules threads (see section \ref{prog_overview_modules}). The manager has an independent manager thread, used for manager commands execution, parameters timeout processing and so on. \subsection{Memory and buffers} Data in memory is referred by \class{dabc::Buffer} \index{Core classes !dabc::Buffer} objects. Allocated memory areas are kept in\\ \class{dabc::MemoryPool} \index{Core classes !dabc::MemoryPool} objects. In general case \class{dabc::Buffer} contains a list of references to scattered memory fragments from memory pool. Typically a buffer references exactly one segment. Buffer may have an empty list of references. In addition, the buffer can be supplied with a custom header. The auxiliary class \class{dabc::Pointer} \index{Core classes !dabc::Pointer} offers methods to transparently treat the scattered fragments from the user point of view (concept of "virtual contiguous buffer"). Moreover, the user may also get direct access to each of the fragments. The buffers are provided by one or several memory pools which preallocate reasonable memory from the operating system. A memory pool may keep several sets, each set for a different configurable memory size. A modules communicates with a memory pool via a \class{dabc::PoolHandle} \index{Core classes !dabc::PoolHandle} object. A new buffer may be requested from a memory pool by size. Depending on the module type and mode, this request may either block until an appropriate buffer is available, or it may return an error value if it can not be fulfilled. The delivered buffer has at least the requested size, but may be larger. A buffer as delivered by the memory pool is contiguos. Several buffers may refer to the same fragment of memory. Therefore, the memory as owned by the memory pool has a reference counter which is incremented for each buffer that refers to any of the contained fragments. When a user frees a buffer object, the reference counters of the referred memory blocks are decremented. If a reference counter becomes zero, the memory is marked as "free" in the memory pool. \subsection{Ports} Buffers are entering and leaving a module through \class{dabc::Port} \index{Core classes !dabc::Port} objects. Each port has a buffer queue of configurable length. A module may have several input, output, or bidirectional ports. The ports are owned by the module. Depending on the module type, there are different possibilities to work with the ports in the processing functions of the module. These are described in section \ref{plugin_module_sync} for \class{dabc::ModuleSync} and section \ref{plugin_module_async} for \class{dabc::ModuleAsync} respectively. \subsection{Transport} Outside the modules the ports are connected to \class{dabc::Transport} \index{Core classes !dabc::Transport} objects. On each node, a transport may either transfer buffers between the ports of different modules (local data transport), or it may connect the module port to a data source or sink (e.~g.~ file i/o, network connection, hardware readout). In the latter case, it is also possible to connect ports of two modules on different nodes by means of a transport instance of the same kind on each node (e.~g.~ {\em InfiniBand verbs} transport connecting a sender module on node A with a receiver module on node B via a {\em verbs} device connection). \subsection{Device} A transport belongs to a \class{dabc::Device} \index{Core classes !dabc::Device} object of a corresponding type that manages it. Such a device may have one or several transports. The threads that run the transport functionality are created by the device. If the \class{dabc::Transport} implementation shall be able to block (e.~g.~ on socket receive), there can be only one transport for this thread. A \class{dabc::Device} instance usually represents an I/O component (e.~g.~ network card); there may be more than one \class{dabc::Device} instances of the same type in an application scope. The device objects are owned by the manager singleton; transport objects are owned and managed by their corresponding device. A device is persistent independent of the connection state of the transport. In contrast, a transport is created during \func{connect()} or \func{open()} and deleted during \func{disconnect()} or \func{close()}, respectively. A device may register parameters and define commands. This is the same functionality as available for modules. \subsection{Application} The \class{dabc::Application} \index{Core classes !dabc::Application} is a singleton object that represents the running application of the DAQ node (i.~e.~ one per system process). It provides the main configuration parameters and defines the runtime actions in the different control system states (see section \ref{prog_fsm}). In contrast to the \class{dabc::Manager} implementation that defines a framework control system (e.g. DIM, EPICS), the subclass of \class{dabc::Application} defines the experiment specific behaviour of the DAQ. \section{Controls and configuration} \subsection{Finite state machine} \label{prog_fsm} The running state of the DAQ system is ruled by a {\sl Finite State Machine} \cite{Wikipedia-Statemachine} on each node of the cluster. The manager provides an interface to switch the application state by the external control system. This may be done by calling state change methods of the manager, or by submitting state change commands to the manager. The finite state machine itself is not necessarily part of the manager, but may be provided by an external control system. In this case, the manager defines the states, but does not check if a state transition is allowed. However, the \dabc~ core system offers a native state machine to be used in the controls implementation; it can be activated in the constructor of the \class{dabc::Manager} subclass by method \func{InitSM()}. Some of the application states may be propagated to the active components (modules, device objects), e.g. the \keyw{Running} or \keyw{Ready} state which correspond to the activity of the thread. Other states like \keyw{Halted} or \keyw{Failure} do not match a component state; e.g. in \keyw{Halted} state, all modules are deleted and thus do not have an internal state. The granularity of the control system state machine is not finer than the node application. \figpdf{statemachine}{The finite state machine as defined by the manager.}{htb}{0}{.8} There are 5 generic states to treat all set-ups: \index{Finite state machine ! states} \begin{compactdesc} \item[Halted] : The application is not configured and not running. There are no modules, transports, and devices existing. \item[Configured] : The application is mostly configured, but not running. Modules and devices are created. Local port connections are done. Remote transport connections may be not all fully connected, since some connections require active negotiations between different nodes. Thus, the final connecting is done between \keyw{Configured} and \keyw{Ready}. \item[Ready] : The application is fully configured, but not running (modules are stopped). \item[Running] : The application is fully configured and running. \item[Failure] : This state is reached when there is an error in a state transition function. Note that a run error during the \keyw{Running} state would not lead to \keyw{Failure}, but rather to stop the run in a usual way (to \keyw{Ready}). \end{compactdesc} The state transitions between the 5 generic states correspond to commands of the control system for each node application: \index{Finite state machine ! transition commands} \begin{compactdesc} \item[DoConfigure] : between \keyw{Halted} and \keyw{Configured}. The application plug-in creates application specific devices, modules and memory pools. Application typically establishes all local port connections. \item[DoEnable] : between \keyw{Configured} and \keyw{Ready}. The application plug-in may establish the necessary connections between remote ports. The framework checks if all required connections are ready. \item[DoStart] : between \keyw{Ready} and \keyw{Running}. The framework automatically starts all modules, transport and device actions. \item[DoStop] : between \keyw{Running} and \keyw{Ready}. The framework automaticall stops all modules, transport and device actions, i.e. the code is suspended to \keyw{wait} at the next appropriate \keyw{waiting point} (e.g. begin of \func{MainLoop()}, wait for a requested resource). Note: queued buffers are not flushed or discarded on \keyw{Stop} ! \item[DoHalt] : switches states \keyw{Ready} , \keyw{Running} , \keyw{Configured}, or \keyw{Failure} to \keyw{Halted}. The framework automatically deletes all registered objects (transport, device, module) in the correct order. However, the user may explicitly specify on creation time that an object shall be persistent (e.g. a device may be kept until the end of the process once it had been created). \end{compactdesc} \subsection{Commands} The control system may send (user defined) commands to each component (module , device, application). Execution of these commands is independent of the state machine transitions. \subsection{Parameters for configuration and monitoring} The {\sl Configuration} is done using parameter objects. The manager provides an interface to register parameters to the configuration/control system. On application startup time, the configuration system may set the parameters from a configuration file (e.g. XML configuration files). During the application lifetime, the control system may change values of the parameters by command. However, since the set up is changed on \keyw{DoConfigure} time only, it may be forbidden to change true configuration parameters except when the application is \keyw{Halted}. Otherwise, there would be the possibility of a mismatch between the monitored parameter values and the really running set up. However, the control system may change local parameter objects by command in any state to modify minor system properties independent of the configuration set up (e.g. switching on debug output, change details of processing parameters). The current parameters may be stored back to the XML file. Apart from the configuration, the control system may use local parameter objects for {\sl Monitoring} the components. When monitoring parameters change, the control system is updated by interface methods of the manager and may refresh the GUI representation. Chapter \ref{prog_setup} will explain the usage of parameters for configuration in detail. \section{Package and library organisation} The complete system consists of different packages. Each package is represented by a subproject of the source code with own namespace. There may be one or more shared libraries for each package. Main packages are as follows: \figpdf{DABCcomponents}{Schematic view of the distributed \dabc~ components (coloured) and user specific extensions (white)}{htb}{0}{.9} \subsection{Core system} The \strong{Core system} package uses namespace \class{dabc::}. It defines all base classes and interfaces, and implements basic functionalities for object organization, memory management, thread control, and event communication. Section \ref{prog_core_classes} gives a brief overview of the \strong{Core system} classes. \subsection{Control and configuration system} Depends on the \strong{Core system}. Defines functionality of state machine, command transport, parameter monitoring and modification. Implements the connection of configuration parameters with a database (i.e. a file in the trivial case). Interface to the \strong{Core system} is implemented by subclass of \class{dabc::Manager}. Note that default implementations of state machine and a configuration file parser are already provided by the \strong{Core system}. \subsection{Plugin packages} Plugin packages may provide special implementations of the core interface classes: \\ \class{dabc::Device}, \class{dabc::Transport}, \class{dabc::Module}, or \class{dabc::Application}. Usually, these classes are made available to the system by means of a corresponding \class{dabc::Factory} that is automatically registered in the \class{dabc::Manager} when loading the plugin library. When installed centrally, the \strong{Plugin packages} are kept in subfolders of the \verba{\$DABCSYS/plugins} directory. Alternatively, the \strong{Plugin packages} may be installed in a user directory and linked against the \strong{Core system} installation. \subsubsection{Bnet package} This package uses namespace \class{bnet::}. It depends on the \strong{Core system} and implements modules to cover a generic event builder network. It defines interfaces (virtual methods) of the special Bnet modules to implement user specific code in subclasses. The \strong{Bnet package} provides a factory to create specific Bnet modules by class name. It also provides application classes to define generic functionalities for worker nodes (\class{bnet::WorkerApplication}) and controller nodes (\class{bnet::ClusterApplication}). These may be used as base classes in further \strong{Application packages}. Section \ref{prog_bnet_classes} gives a brief overview of the \strong{Bnet package} classes; chapter \ref{prog_exabnet} describes an example using the Bnet plugins. \subsubsection{Transport packages} Depend on the \strong{Core system}, and may depend on external libraries or hardware drivers. Implement \class{dabc::Device} and \class{dabc::Transport} classes for specific data transfer mechanism, e.g. \strong{verbs} or \strong{tcp/ip socket}. May also implement \class{dabc::Device} and \class{dabc::Transport} classes for special data input or output. Each transport package provides a factory to create a specific device by class name. However, the most common transport implementations are put directly to the \strong{Core system}, e.g. local memory, or socket transport; the corresponding factory is part of the \strong{Core system} then. \subsection{Application packages} They depend on the \strong{Core system}, and may depend on several \strong{transport packages}, on the \strong{Bnet package}, or other plugin packages. They may also depend on other application packages. \strong{Application packages} provide the actual implementation of the core interface class \class{dabc::Application} that defines the set-up and behaviour of the DAQ application in different execution states. This may be a subclass of specific existing application (e.g. subclass of \class{bnet::WorkerApplication}). Additionally, they may provide experiment specific \class{dabc::Module} classes. When installed centrally, the \strong{Application packages} are kept in subfolders of the \\ \small{\verba{\$DABCSYS/applications}} directory. Alternatively, an \strong{Application package} may be installed in a user directory and linked against the \strong{Core system} installation and the required \strong{Plugin packages}. \subsection{Distribution contents} The DABC distribution contains the following packages: \begin{compactdesc} \item [Core system] : This is plain C++ code and independent of any external framework. \item [Bnet plugin] : Depends on the core system only. \item [Transport plugins] : Network transport for \textit{tcp/ip} sockets and \textit{InfiniBand} verbs. Additionally, transports for GSI \textit{Multi Branch System} \mbs\ connections (socket, filesystem) is provided. Optionally, example transport packages may be installed that illustrate the readout of a \textit{PCIe} board, or data taking via \textit{UDP} from an external readout controller (ROC) board. \item [Control and configuration system]: The general implementation is depending on the DIM framework only. DIM is used as main transport layer for commands and parameter monitoring. On top of DIM, a generic record format for parameters is defined. Each registered command exports a self describing command descriptor parameter as DIM service. Configuration parameters are set from XML setup files and are available as DIM services. \item [GUI] A generic controls GUI using the DIM record and command descriptors is implemented with Java. It may be extendable with user defined components. \item[Application packages] : some example applications, such as: \begin{compactitem}[$\circ$] \item Simple \mbs~ event building \item Bnet with switched \mbs~ event building \item Bnet with random generated events \end{compactitem} \end{compactdesc} \section{Main Classes} \subsection{Core system} \label{prog_core_classes} The most important classes of the \dabc~ core system are described in the following. \figpdf{mainclasses}{Simplified UML diagram of the most important \dabc\ classes for active components. Framework base classes are coloured in green. Some implementation examples are shown with other corresponding colours: sockets, {\em verbs}, Bnet, PCI, and \mbs\ . See text for details.}{htb}{0}{1.1} %\begin{compactdesc} \begin{description} \item[\class{dabc::Object}] : \index{Core classes !dabc::Object} The base class for all objects to be kept in \dabc~ collections. \item[\class{dabc::Command}] : \index{Core classes !dabc::Command} Represents a command object. A command is identified by its name which it keeps as text string. Additionally, a command object may contain any number of arguments (integer, double, text). These can be set and requested at the command by their names. The available arguments of a special command may be exported to the control system as \class{dabc::CommandDefinition} objects. A command is sent from a \class{dabc::CommandClient} object to a \class{dabc::CommandReceiver} object that executes it in its scope. The result of the command execution may be returned as a reply event to the command client. The manager is the standard command client that distributes the commands to the command receivers (i.e. module , manager, or device). See chapter \ref{prog_services_commands} for more details on the command mechanisms. \item[\class{dabc::Parameter}] : \index{Core classes !dabc::Parameter} Parameter object that may be monitored or changed from control system. Any \\ \class{dabc::Worker} implementation may register its own parameters. Parameter can be used for configuration of object at creation time (via configuration file), monitoring of object properties in GUI or manipulating of object properties at runtime, changing parameter values via controlling interface. Currently supported parameter types are: \begin{compactitem}[$\bullet$] \item \class{dabc::IntParameter} - simple integer value \item \class{dabc::DoubleParameter} - simple double value \item \class{dabc::StrParameter} - simple string value \item \class{dabc::StateParameter} - contains state record, e.~g.~ current state of the finite state machine and associated colour for gui representation \item \class{dabc::InfoParameter} - contains info record, e.~g.~ system message and associated properties for gui representation \item \class{dabc::RateParameter} - contains data rate record and associated properties for GUI representation. May be updated in predefined time intervals. \item \class{dabc::HistogramParameter} - contains histogram record and associated properties for GUI representation. \end{compactitem} \item[\class{dabc::WorkingThread}] : \index{Core classes !dabc::WorkingThread} An object of this class represents a system thread. The working thread may execute one or several jobs; each job is defined by an instance of \class{dabc::Worker}. The working thread waits on an event queue (by means of pthread condition) until an event for any associated working processor is received; then the corresponding event action is executed by calling \func{ProcessEvent()} of the corresponding working processor. \item[\class{dabc::Worker}] : \index{Core classes !dabc::Worker} Represents a runnable job. Each working processor is assigned to one working thread instance; this thread can serve several working processors in parallel. In a special mode a processor can also run its explicit main loop. \class{dabc::Worker} is a subclass of \class{dabc::CommandReceiver}, i.e. a working processor may receive and execute commands in its scope. \item[\class{dabc::Module}] : \index{Core classes !dabc::Module} A processing unit for one "step" of the dataflow. Is subclass of \class{dabc::Worker}, i.~e.~ the module may be run by an own dedicated thread, or a working thread may execute several modules that are assigned to it. A module has ports as connectors for the incoming and outgoing data flow. \item[\class{dabc::ModuleSync}] : \index{Core classes !dabc::ModuleSync} Is subclass of \class{dabc::Module}; defines interface for a synchronous module that is allowed to block. User must implement virtual method \func{MainLoop()} that uses a dedicated working thread to run. Method \func{TakeBuffer()} provides blocking access to a memory pool. Blocking methods \func{dabc::ModuleSync::Send()} and \func{dabc::ModuleSync::Receive()} are used from the \func{MainLoop()} code to send (or receive) buffers over (or from) a ports. \item[\class{dabc::ModuleAsync}] : \index{Core classes !dabc::ModuleAsync} Subclass of \class{dabc::Module}; defines interface for an asynchronous module that must never block the execution. Several \class{dabc::ModuleAsync} objects may be assigned to one working thread. User must either re-implement virtual method \func{ProcessItemEvent()} wich is called whenever {\bf any} event for this module (i.e. this working processor) is processed by the working thread. Or the user may implement callbacks for special events (e.g. \func{ProcessInputEvent()}, \func{ProcessOutputEvent()}, \func{ProcessPoolEvent()},..) that are invoked when the corresponding event is processed by the working thread. The events are dispatched to these callbacks by the \func{ProcessItemEvent()} default implementation then. There are no blocking function available in \class{dabc::ModuleAsync}; but the user \strong{must} avoid any polling loops, waiting for resources - event processing function must be returned as soon as possible. \item[\class{dabc::Port}] : \index{Core classes !dabc::Port} A connection interface between module and transport. From inside the module scope, only the ports are visible to send or receive buffers by reference. Data connections between modules (i.e. transports between the ports of the modules) are set up by the application using methods of \class{dabc::Manager} which specify the full module/port names. For ports on different nodes, commands to establish a connection may be send remotely (via controls layer, e.g. DIM) and handled by the manager of each node. \item[\class{dabc::Transport}] : \index{Core classes !dabc::Transport} A producing or consuming entity for buffers, which it delivers to (or receives from, resp.) a \class{Module} via the \class{Port} interface. As an example, \class{dabc::NetworkTransport} implements the transport between modules on different nodes. \item[\class{dabc::Device}] : \index{Core classes !dabc::Device} Device class used for creation and configuration of transport objects. Is a subclass of \\ \class{dabc::Worker}. The \class{dabc::Transport} and \class{dabc::Device} base classes have various implementations: \begin{compactitem}[$\bullet$] \item \class{dabc::LocalTransport} for memory transport within same process \item \class{dabc::SocketTransport} and \class{dabc::SocketDevice} for tcp/ip sockets \item \class{verbs::Transport} and \class{verbs::Device} for InfiniBand {\bf verbs} connection \item \class{pci::Transport} and \class{pci::BoardDevice} for DMA I/O from PCI or PCIe boards \end{compactitem} \item[\class{dabc::Manager}] : \index{Core classes !dabc::Manager} Is manager of everything in DABC. There is the only instance of manager in the process scope, availible via \func{dabc::mgr()} or \func{dabc::Manager::mgr} functions. It combines different roles: \begin{compactenum} \item It is a manager of all \class{dabc::Object} objects in the process scope. Objects (e.~g.~ modules, devices, parameters) are kept in a folder structure and can be identified by full path name. \item It defines the interface to the controls system (state machine, remote command communication, parameter export); this is to be implemented in a subclass. The manager handles the command and parameter flow locally and remotely: commands submitted to the local manager are directed to the command receiver where they shall be executed. If any parameter is changed, this is recognized by the manager and optionally forwarded to the associated controls system. Current implementations of manager are: \begin{compactitem}[$\bullet$] \item \class{dabc::Manager} provides base manager functionality, can only be used for single-node application without any controlling possibilities. \item \class{dabc::StandaloneManager} provides simple socket controls connection between several node in multi-node cluster, cannot be used with GUI. \item \class{dimc::Manager} Provides DIM \cite{DIM} as transport layer for controlling commands. Additionally, parameters may be registered and updated automatically as DIM services. There is a general purpose Java GUI for this implementation. \end{compactitem} \item It provides interfaces for user specific plug-ins that define the actual set-up: \\ several \class{dabc::Factory} objects to create objects, and one \class{dabc::Application} object to define the state machine transition actions. \end{compactenum} \item[\class{dabc::Factory}] : \index{Core classes !dabc::Factory} Factory plug-in for creation of applications, modules, devices, transports and threads. \item[\class{dabc::Application}] : \index{Core classes !dabc::Application} Defines user actions on transitions of the finite state machine of the manager. Good place for export of application-wide configuration parameters. May define additional commands. %\end{compactdesc} \end{description} \subsection{BNET classes} \label{prog_bnet_classes} The classes of the Bnet package, providing functionalities of the event builder network. \figpdf{bnetclasses}{Simplified UML diagram of the {\em Bnet} classes: green - \dabc\ base classes; dark green - Bnet interface classes; magenta - Bnet implementation classes; yellow - \mbs\ implementation example. See text for details.}{htb}{0}{1.1} %\begin{compactdesc} \begin{description} \item[\class{bnet::ClusterApplication}] : \index{Bnet classes !bnet::ClusterApplication} Subclass of \class{dabc::Application} to run on the cluster controller node of the builder network. \begin{compactenum} \item It implements the master state machine of the Bnet. The controlling GUI usually sends state machine commmands to the controller node only; the Bnet cluster application works as a command fan-out and state observer of all worker nodes. \item It controls the traffic scheduling of the data packets between the worker nodes by means of a data flow controller (class \class{bnet::GlobalDFCModule}). This controller module communicates with the Bnet sender modules on each worker to let them send their packets synchronized with all other workers. \item It may handle failures on the worker nodes automatically, e.~g.~ by reconfiguring the data scheduling paths between the workers. \end{compactenum} \item[\class{bnet::WorkerApplication}] : \index{Bnet classes !bnet::WorkerApplication} Subclass of \class{dabc::Application} to run on the worker nodes of the builder network. \begin{compactenum} \item Implements the local state machine callbacks for each worker with respect to the Bnet functionality. \item It registers parameters to configure the node in the Bnet, and methods to set and check these parameters. \item Defines factory methods \func{CreateReadout()}, \func{CreateCombiner()}, \func{CreateBuilder()}, \func{CreateFilter()}, \func{CreateStorage()} to be implemented in user specific subclass. These methods are used in the worker state machine of the Bnet framework. \end{compactenum} \item[\class{bnet::GeneratorModule}] : \index{Bnet classes !bnet::GeneratorModule} Subclass of \class{dabc::ModuleSync}. Framework class to fill a buffer from the assigned memory pool with generated (i.e. simulated) data. \begin{compactenum} \item Method \func{GeneratePacket(buffer)} is to be implemented in application defined subclass (e.~g.~ \\ \class{bnet::MbsGeneratorModule}) and is called frequently in module's \func{MainLoop()}. \item Each filled buffer is forwarded to the single output port of the module. \end{compactenum} \item[\class{bnet::CombinerModule}] : \index{Bnet classes !bnet::CombinerModule} Subclass of \class{dabc::ModuleSync}. Framework prototype class to format inputs from several readouts to one data frame (e.g. combine an event from subevent readouts on that node). \begin{compactenum} \item It provides memory pools handles and one input port for each readout connection \\ (either \class{bnet::GeneratorModule} or connection to a readout transport). \item Creates output port for combined subevents. \item The formatting functionality is to be implemented in method \func{MainLoop()} of user defined subclass (e.g. \class{bnet::MbsCombinerModule}). \end{compactenum} \item[\class{bnet::SenderModule}] : \index{Bnet classes !bnet::SenderModule} Subclass of \class{dabc::ModuleAsync}. Responsible for sending the subevents data frames to the receiver nodes, according to the network traffic schedule as set by the Bnet cluster plugin. \begin{compactenum} \item It has {\bf one} input port that gets the event packets (or time sorted frames) from the preceding Bnet combiner module. The input data frames are buffered in the Bnet sender module and analyzed which frame is to be sent to what receiver node. This can be done in a non-synchronized "round-robin" fashion, or time-synchronized after a global traffic schedule as evaluated by the Bnet cluster plugin. \item Each receiver node is represented by one output port of the Bnet sender module that is connected via a network transport (tcp socket, {\em InfiniBand verbs}) to an input port of the corresponding Bnet receiver node. \end{compactenum} \item[\class{bnet::ReceiverModule}] : \index{Bnet classes !bnet::ReceiverModule} Subclass of \class{dabc::ModuleAsync}. Receives the data frames from the Bnet sender modules and sorts together packets, belonging to the same events (or time frames, resp.). \begin{compactenum} \item It has {\bf one} input port {\bf for each sender node} in the Bnet. The data frames are buffered in the Bnet receiver module until the corresponding frames of all senders have been received; then received frames are send sequentially to the output port. \item It has {\bf exactly one} output port. This is connected to the \class{bnet::BuilderModule} implementation that performs the actual event building task. \end{compactenum} \item[\class{bnet::BuilderModule}] : \index{Bnet classes !bnet::BuilderModule} Subclass of \class{dabc::ModuleSync}. Framework prototype class to select and build a physics event from the data frames of all Bnet senders as received by the receiver module. \begin{compactenum} \item It has {\bf one} input port connected to the Bnet receiver module. The data frame buffers of all Bnet senders are transferred serially over this port and are then kept as an internal {\bf std::vector} in the Bnet builder module. \item Method \func{DoBuildEvent()} is to be implemented in user defined subclass \\ (e.~g.~ \class{bnet::MbsBuilderModule}) and is called in module's \func{MainLoop()} when a set of corresponding buffers is complete. \item It provides {\bf one} output port that may connect to a Bnet filter module, or a user defined output or storage module, resp. \item The user has to implement the sending of the tagged events to the output port explicitly in his subclass. %{\bf Should be done automatically in \func{MainLoop()} later?} \end{compactenum} \item[\class{bnet::FilterModule}] : \index{Bnet classes !bnet::FilterModule} Subclass of \class{dabc::ModuleSync}. Framework prototype class to filter out the incoming physics events according to the experiment's "trigger conditions". \begin{compactenum} \item Has {\bf one} input port to get buffers with already tagged physics events from the preceding Bnet builder module. \item Has {\bf one} output port to connect a user defined output or storage module, resp. \item Method \func{TestBuffer(buffer)} is to be implemented in user defined subclass (e.~g.~ \class{bnet::MbsFilterModule}) and is called in module's \func{MainLoop()} for each incoming buffer. Method should return \keyw{true} if the event is "good" for further processing. \item Forwards "good" buffers to the output port and discards others. \end{compactenum} %\end{compactdesc} \end{description}