[programmer/prog-exa-roc.tex]

\section{Overview}
\label{prog-exa-roc}
The CBM ReadOut Controller ({\em ROC}) is an FPGA-based board
to configure and read out the
{\em nXYTER} chip \cite{nxyter}, and to transport the acquired data over Ethernet to a PC. 
The software package {\em ROClib} 
provides the basic functionality to work with such {\em ROC}.
 
To support the usage of {\em ROC} in \dabc~, the following classes 
were implemented:
\bbul
\item \class{roc::Device}  device class, wrapper for the \class{SysCoreController}
class of {\em ROClib} 
\item \class{roc::Transport}  corresponding transport, with access to the functionality of \class{SysCoreBoard} class of {\em ROClib} 
\item \class{roc::CombinerModule}  module to combine data from several {\em ROCs} into a single output port 
\item \class{roc::CalibrationModule}  module to calibrate the time scale in {\em ROC} data  
\item \class{roc::ReadoutApplication}  application to perform readout from {\em ROC} boards   
\item \class{roc::Factory}  factory class to organize these plugins 
\ebul


\section{Device and transport}

The {\em ROC} device class \class{roc::Device} inherits from two classes: \class{dabc::Device} and \class{SysCoreControl},
where \class{SysCoreControl} provides simultaneous access to several {\em ROC} boards. 
Usually the instance of a device class corresponds to one physical device or board, 
but here the device object is rather used as central collection of \class{SysCoreBoard} objects, and as thread provider.

Each instance of \class{roc::Transport} has a pointer to a \class{SysCoreBoard} object
which handles data taking from a specific {\em ROC}. 
The implementation of \class{roc::Transport} is based on the \class{dabc::DataTransport} class (see section \ref{prog_plugin_device_datatransport}) which
runs as \class{Worker} with an asynchronous event handling mechanism,
so it does not require an explicit thread.
This feature allows to process several instances of such transports in the same thread.
In the {\em ROC} case, all \class{roc::Transport} instances use the thread of \class{roc::Device}.   
  
Let's have a look how \class{roc::Transport} is working.
When the connected module starts, method \func{StartTransport()} is called,
which will invoke \func{SysCoreBoard::startDaq()} to start data taking. 
After that, the buffer filling loop consists in subsequent calls of \func{Read\_Size()}, \func{Read\_Start()} and \func{Read\_Complete()}
functions, implementing the interface of \class{dabc::DataTransport} 
(see section \ref{prog_plugin_device_datatransport}). 

Method \func{Read\_Size()} defines the size of the next buffer, 
required for data reading. In case of \class{roc::Transport} this size
is fixed and is taken from a configuration parameter:  

\begin{small}
\begin{verbatim}
unsigned roc::Transport::Read_Size()
{
   return fBufferSize;
}
\end{verbatim}
\end{small}
    
When the system has delivered a buffer of the requested size,  
function \func{Read\_Start()} is called to start reading 
of that buffer from the data source:

\begin{small}
\begin{verbatim}
unsigned roc::Transport::Read_Start(dabc::Buffer& buf)
{
   int req = fxBoard->requestData(fReqNumMsgs);
   if (req==2) return dabc::DataInput::di_CallBack;
   if (req==1) return dabc::DataInput::di_Ok;
   return dabc::DataInput::di_Error;
}
\end{verbatim}
\end{small}

The \class{SysCoreBoard} (accessed by pointer {\tt fxBoard})
keeps internally  own buffers of the received UDP messages.
The call \func{SysCoreBoard::requestData()} informs by return value
either that the required number of messages is already received,
or that the caller should wait (i.~e.~ poll this method here until
it returns that all data is ready). 
However, waiting would mean that the working thread is blocked and 
could not run the other transport instances. Therefore, another approach is used: 
the {\em ROClib} will call back the virtual method \func{SysCoreControl::DataCallBack()}  
when the required amount of data is there. This method
is implemented for \class{roc::Transport} to complete filling the current
\class{dabc::Buffer}.

If data already exists in the internal buffers of \class{SysCoreBoard},
the value \keyw{dabc::DataInput::di\_Ok} 
is returned; then the \class{DataTransport} framework will immediately call \func{Read\_Complete()} which finally fills the \dabc~ buffer:

\begin{small}
\begin{verbatim}
unsigned roc::Transport::Read_Complete(dabc::Buffer* buf)
{
   unsigned fullsz = buf->GetDataSize();
   if (!fxBoard->fillData((char*) buf->GetDataLocation(), fullsz)) 
      return dabc::DataInput::di_SkipBuffer;
   if (fullsz==0) 
      return dabc::DataInput::di_SkipBuffer;
   buf->SetTypeId(roc::rbt_RawRocData);
   buf->SetDataSize(fullsz);
   return dabc::DataInput::di_Ok;
}
\end{verbatim}
\end{small}

The return value \keyw{dabc::DataInput::di\_CallBack}
of function \func{Read\_Start()} indicates 
that processing of this transport should be suspended, 
because the requested amount of data is not ready yet.
When all this data has been received by the {\em ROClib},
it will invoke method \func{SysCoreControl::DataCallBack()} 
which is reimplemented in subclass \class{roc::Device},
simply forwarding to
the following method of \class{roc::Transport}: 

\begin{small}
\begin{verbatim}
void roc::Transport::CompleteBufferReading()
{
   unsigned res = Read_Complete(fCurrentBuf);
   Read_CallBack(res);
}
\end{verbatim}
\end{small}

As the required amount of data is ready now,
one only retrieves it to the current buffer with the same 
\func{Read\_Complete()} method, and 
reactivates the processing of this transport instance
by calling \func{Read\_CallBack()}.   


\section{Combiner module}

Class \class{roc::CombinerModule} combines data from several ROC boards in one MBS event. It also performs sorting of data according the timestamp, resolves the "last epoch" bits, and fixes several coding errors (class \class{SysCoreSorter} is
used for this).

The module has following configuration parameters:
\bbul
\item \param{NumRocs}     - number of {\em ROC} boards, connected to combiner [default 1]  
\item \param{BufferSize}  - size of buffer, used to read data from {\em ROCs} [default 16384]
\item \param{NumOutputs}  - number of outputs [default 2]
\ebul

As output \mbs~ events are provided. Each \mbs~ event contains {\em ROC} messages between two {\em sync markers}.
For each {\em ROC} a separate \mbs~ subevent is allocated; 
field \func{iSubcrate} of the subevent header contains the {\em ROC} id.


\section{Calibration module}

Class \class{roc::CalibrationModule} performs the calibration of the time scale for all
{\em ROCs} and merges all messages into a single data stream. 
As output, an \mbs~ event with a single
subevent is produced. 

The module has following configuration parameters:
\bbul
\item \param{NumRocs}     - number of ROC boards, which should be provided in MBS event [default 2]  
\item \param{BufferSize}  - size of buffer, used to produce output data [default 16384]
\item \param{NumOutputs}  - number of outputs [default 2]
\ebul
 

\section{Readout application}

The main aim of \class{roc::ReadoutApplication} class is to configure and run 
the application which combines the data readouts from several {\em ROC}s.
It can store the data into a {\tt \*.lmd} file, and it  
may provide  \mbs~ {\em stream} or {\em transport} servers 
for online monitoring, e.~g.~ with a remote {\em Go4} analysis.
It has following configuration parameters:

\bbul
\item \param{NumRocs}   - number of ROC boards  
\item \param{RocIp0}, \param{RocIp1}, \param{RocIp2}, ... - addresses (IP or nodname) of ROC boards
\item \param{DoCalibr}  - defines calibration mode (see further)
\item \param{BufferSize} - size of buffer
\item \param{NumBuffers} - number of buffers
\item \param{MbsServerKind} - kind of MBS server ("None", "Stream", "Transport")
\item \param{RawFile} - name of {\tt \*.lmd} file to store "raw" combined data (after
\class{CombinerModule})
\item \param{CalibrFile} - name of {\tt \*.lmd} file to store "calibrated" data (after
\class{CalibrationModule})
would exceed this size, a new output file is automatically created with a sequence number appended to the file name.
\ebul

Three calibration modes are supported:
\bbul
\item \param{DoCalibr}=0 - Only the \class{CombinerModule} is instantiated, which produces a kind of {\em ROC} "raw" data 
\item \param{DoCalibr}=1 - Both \class{CombinerModule} and \class{CalibrationModule} are instantiated   
\item \param{DoCalibr}=2 - Only the \class{CalibrationModule} is instantiated. 
This is used to convert "raw" data read from {\tt .lmd} files into the "calibrated" format.
\ebul

In all modes output in form of raw or (and) calibrated data can be stored in 
{\tt .lmd} file(s), defined by \param{RawFile} and \param{CalibdFile} parameters respectively. The last mode is a special case,
since \param{RawFile} does not specify the output, but the input file for the 
calibration module.


\section{Factory}

Factory class \class{roc::Factory} implements several methods to create the 
{\em ROC}-specific application, device and modules, as described in section 
\ref{prog_plugin_factory}.    


\section{Source and compilation}

The source code of all classes can be found in 
{\tt \$DABCSYS/plugins/roc} directory. 
Compiled library {\tt libDabcKnut.so} is in directory
{\tt \$DABCSYS/lib} .
If one needs to modify some code in this library, one should copy the sources to 
a user directory and call "make" in this directory. In this case the library 
is build into
a subdirectory, named like {\tt \$ARCH/lib}, where {\tt \$ARCH} 
is the current CPU architecture (for instance, "i686").

 
\section{Running the {\em ROC} application}

To run the readout application, an approprite XML configuration file 
is required. There are two examples of configuration files in {\tt \$DABCSYS/applications/roc}.

File {\tt Readout.xml} configures the readout from 3 ROCs: 

\begin{verbatim}
<?xml version="1.0"?>
<dabc version="1">
<Context name="Readout">
  <Run>
    <lib value="libDabcMbs.so"/>
    <lib value="libDabcKnut.so"/>
    <logfile value="Readout.log"/>
  </Run>
  <Application class="roc::Readout">
    <DoCalibr value="0"/>
    <NumRocs value="3"/>
    <RocIp0 value="cbmtest01"/>
    <RocIp1 value="cbmtest02"/>
    <RocIp2 value="cbmtest04"/>
    <BufferSize value="65536"/>
    <NumBuffers value="100"/>
    <TransportWindow value="30"/>
    <RawFile value="run090.lmd"/>
  </Application>
 </Context>
</dabc>
\end{verbatim}

Because this is a single-node application, 
it can be started directly from a shell 
by calling the standard \func{dabc\_run}
executable with the configuration file name as argument:
\verba{dabc\_run Readout.xml}. 
This executable will load the specified libraries,
create the application, configure it, and switch the system in 
the \keyw{Running} state.   

File  {\tt Calibr.xml} shows the special case of a configuration to convert 
"raw" data into "calibrated" data without running any real DAQ:


\begin{verbatim}
<?xml version="1.0"?>
<dabc version="1">
<Context name="Calibr">
  <Run>
    <lib value="libDabcMbs.so"/>
    <lib value="libDabcKnut.so"/>
    <logfile value="Calibr.log"/>
  </Run>
  <Application class="roc::Readout">
    <DoCalibr value="2"/>
    <NumRocs value="3"/>
    <BufferSize value="65536"/>
    <NumBuffers value="100"/>
    <RawFile value="/d/cbm06/cbmdata/SEP08/raw/run028/run028*.lmd"/>
    <CalibrFile value="testcal.lmd"/>
  </Application>
 </Context>
</dabc>
\end{verbatim}

Here the "raw" data is read from the files matching the name wildcard pattern 
as defined in the {\tt <RawFile>} tag. Note that this example will 
read subsequently all data of run "028" which possibly was saved
into several files with subsequent numbers appended to their names, due to the 
{\tt <FileSizeLimit>} mechanism as described above.
The "calibrated" data is written as usual to the output file
as specified in the {\tt <CalibrFile>} tag.

 