Show paper.tex syntax highlighted
\NeedsTeXFormat{LaTeX2e}
\documentclass[10pt,a4]{scrartcl}
\usepackage{tabularx,graphicx,a4,listings,wrapfig,subfigure}
\ifx\pdfoutput\undefined
% We're not running pdftex
% european (better) fonts -- does not look good with pdflatex
\usepackage[T1]{fontenc}
\newcommand{\href}[2]{#2\\{\hspace*{5mm}\scriptsize <#1>}\\}
\else
\pdfcompresslevel=9
\def\pdfBorderAttrs{/Border [0 0 0] } % No border around Links
\usepackage{hyperref}
\fi
\title{Equalizer Programming Guide\\
{\footnotesize\mdseries
\htmladdnormallink{http://www.equalizergraphics.com/documents/Developer/ProgrammingGuide.pdf}
{http://www.equalizergraphics.com/documents/ProgrammingGuide.pdf}}
}
\author{Stefan Eilemann\thanks{eile@eyescale.ch}\\[\medskipamount]
% Eyescale Software Sarl
}
\date{
\textbf{INCOMPLETE}\\[\medskipamount]
Version 0.5, \today
}
\newcommand{\tm}{\texttrademark~}
\newcommand{\rc}{\raise 1ex\hbox{{\tiny\textregistered}}~}
\newcommand{\fig}[1]{Figure~\ref{#1}}
% suppress single floating lines on top (widow) and bottom(club)
% 10000 is infinity
% tradeoff: maybe underfull vboxes
\clubpenalty=10000
\widowpenalty=10000
\begin{document}
\maketitle
\vfill
\lstset{language=C++}
\thispagestyle{empty}
\begin{figure}[ht]
\hfill
\includegraphics[width=4cm]{images/logo.pdf}\hfill
\includegraphics[width=4cm]{images/vmml.pdf}\hfill\vspace{-1em}\\
% {\small\htmladdnormallink{www.eyescale.ch}{http://www.eyescale.ch}}
\end{figure}
\vfill
%\abstract{
% Equalizer is a project to develop software to simplify the creation of
% scalable graphics applications and to improve the usability of
% multipipe visualization systems.
%}
\vfill{\center\begin{tabularx}{\textwidth}{|l|l|X|}
\hline
\bf Version & \bf Date & \bf Changes \\
\hline
0.5 & Sep 3, 2007 & added config and partly pipe\\
0.4 & Aug 31, 2007 & added distributed objects\\
0.3 & Aug 26, 2007 & added application and render client\\
0.2 & Aug 20, 2007 & added main function\\
0.1 & Aug 19, 2007 & outlined the basic concepts\\
\hline
\end{tabularx}}
\clearpage
\tableofcontents
\thispagestyle{empty}
\clearpage
\pagenumbering{arabic}
\section{Introduction}
Equalizer provides a framework for the development of parallel OpenGL
applications. Equalizer-based applications can run a single
shared-memory system with multiple graphics cards (GPU's) or on a
distributed graphics cluster. This Programming Guide introduces the
programming interface using the \textsf{eqPly} example shipped with Equalizer.
Any questions related to Equalizer programming and this Programming
Guide should be directed to the \textsf{eq-dev} mailing
list\footnote{see
\htmladdnormallink{http://www.equalizergraphics.com/lists.html}
{http://www.equalizergraphics.com/lists.html}}.
\section{Getting Started}
\subsection{Compiling and running \textsf{eqPly}}
A prerequisite for this Programming Guide is a working \textsf{eqPly}
example. The Quickstart
Guide\footnote{\htmladdnormallink{http://www.equalizergraphics.com/documents/EqualizerGuide.html}{http://www.equalizergraphics.com/documents/EqualizerGuide.html}}
explains how to run it. \textsf{eqPly} can also be executed without a
server, which simplifies the development cycle. In this case it will be
configured to use one window.
\subsection{Equalizer Processes}
\begin{wrapfigure}{r}{.4\textwidth}
\includegraphics[width=.4\textwidth]{images/processes.pdf}
{\caption{\small\label{fProcesses}Equalizer Processes}}
\end{wrapfigure}
All functionality of Equalizer is accessed through the Equalizer client
library, which implements all functionality discussed in this document.
\subsubsection{The Server}
An Equalizer server is responsible for managing one visualization
system\footnote{a shared memory system or graphics cluster}. Currently
it is only useful for running one application at a time, but it will be
extended to support multiple applications concurrently and efficiently
on one system. The server controls and potentially launches the
application's rendering clients.
\subsubsection{The Application}
The application connects to a server, which chooses a configuration for
the application. It provides a render client, to be launched by the
server. The application reacts on events and controls the rendering.
\subsubsection{The Render Client}
The render client implements the rendering part of an application. It is
passive, and receives all its rendering tasks from the server. The tasks
are executed by calling the appropriate task methods (see
\ref{ssTaskMethods}).
The application might be a rendering client, in which case it can also
contribute to the rendering. It can choose not to implement any render
client-related code, in which case it is reduced to be the application's
'master' process without any OpenGL windows.
The rendering client can be the same executable as the application, as
is the case with \textsf{eqPly}. Real-world applications often implement
a separate, light-weight rendering client.
\section{The Programming Interface}
Equalizer uses a C++ programming interface. The API is minimally
invasive, that is, Equalizer imposes only the minimal, natural execution
framework upon the application. It does not impose a scene graph or does
interfere in any way with the application's rendering code.
Methods called by the application always have the form
\textsf{verb[Noun]}, whereas methods called by Equalizer (`Task
Methods') have the form \textsf{nounVerb}.
\subsection{\label{ssTaskMethods}Task Methods}
The application subclasses Equalizer objects and overrides virtual
functions to implement certain functionality, e.g., the application's
OpenGL rendering in \textsf{eq::Chan\-nel::frameDraw}. These task
methods are in concept similar to C function callbacks. The
\textsf{eqPly} section will discuss the most important task methods. A
full list can be found on the
website\footnote{\htmladdnormallink{http://www.equalizergraphics.com/documents/design/taskMethods.html}{http://www.equalizergraphics.com/documents/design/taskMethods.html}}.
\subsection{The Resource Tree}
The rendering resources are represented in a hierarchical tree structure
which corresponds to the physical and logical resources found in a 3D
rendering environment.
\begin{wrapfigure}{r}{.6\textwidth}
\includegraphics[width=.6\textwidth]{images/cave.pdf}
{\caption{\small\label{fConfig}An example configuration}}
\end{wrapfigure}
\fig{fConfig} shows one example configuration for a four-side
CAVE\texttrademark, running on two machines (node) using three graphics
cards (pipe) with one window each to render to the four output channels
connected to the projectors for each of the walls. The compound
description is only used by the server to compute the rendering
tasks. The application is not aware of compounds, and does not need to
concern itself with the parallel rendering logics of a configuration.
For testing and development purposes it is possible to use multiple
instances for one resource, e.g., to run multiple render client nodes on
one computer. For deployment one node and pipe should be used for each
computer and graphics card, respectively.
\subsubsection{Configuration}
The root of the resource tree is the \textsf{eq::Config}, which
represents the current configuration of the application. It currently
only holds the local node, not all nodes of the configuration.
\subsubsection{Node}
An \textsf{eq::Node} is the representation of a single computer in the
system. It is one operating system process of the render client. All
node task methods are executed from the main application thread. Each
configuration might also use an application node, in which case the
application process is also used for rendering.
\subsubsection{Pipe}
The \textsf{eq::Pipe} is the abstraction of a graphics card (GPU). In
the current implementation it is also one operating system thread,
unless the pipe's thread hint is set to \textsf{false}. All pipe and
child window and channel task methods are executed from the pipe thread
for threaded pipes or from the main application thread for non-threaded
pipes\footnote{see also
\htmladdnormallink{http://www.equalizergraphics.com/documents/design/nonthreaded.html}{http://www.equalizergraphics.com/documents/design/nonthreaded.html}}.
Further versions of Equalizer might introduce threaded windows, where
all window-related task methods are executed in a separate operating
system thread.
\subsubsection{Window}
An \textsf{eq::Window} is an drawable and OpenGL context. The drawable
can be an on-screen window or an off-screen PBuffer or
FBO\footnote{off-screen drawables are not yet implemented, but can be
created by the application and used with Equalizer}.
\subsubsection{Channel}
The \textsf{eq::Channel} is the abstraction of an OpenGL viewport within
its parent window. It is the entity executing the actual rendering.
\subsection{Resource Usage}
How the rendering resources are to be used is configured using a
compound tree. Each compound has a channel, which it uses to execute the
rendering tasks. The rendering tasks are computed by the server and send
to the render clients. At no point the application or render clients
have or need knowledge of compounds. The configuration of compounds is
not in the scope of this document\footnote{see
\htmladdnormallink{http://www.equalizergraphics.com/documents/design/compounds.html}{http://www.equalizergraphics.com/documents/design/compounds.html}}.
\section{The \textsf{eqPly} polygonal renderer}
The \textsf{eqPly} example is shipped with the Equalizer distribution
and serves as a simple reference implementation of an Equalizer-based
application. Its focus is not on rendering features or visual quality.
It serves as a test bed for most of the Equalizer features.
In this section the source code of \textsf{eqPly} is discussed in
detail, and relevant design decision and remarks are raised.
All classes in the example are in the \textsf{eqPly} namespace to avoid
type name ambiguities, in particular for the \textsf{Window} class.
\subsection{The main Function}
The main function starts off with parsing the command line into the
\textsf{LocalInitData} data structure, which in part will be distributed
to all render client nodes. For actual command line parsing is done by
the \textsf{LocalInitData} class and will be discussed there:
{\footnotesize\begin{lstlisting}
int main( int argc, char** argv )
{
// 1. parse arguments
eqPly::LocalInitData initData;
initData.parseArguments( argc, argv );
\end{lstlisting}}
The second step is to initialize the Equalizer library. The
initialization function of Equalizer also parses the command line, which
is used to set certain default values based on Equalizer-specific
options\footnote{Equalizer-specific options always start with --eq-},
e.g., the default server location. Furthermore, a node factory is
provided. The \textsf{EQERROR} macro, and its counterparts
\textsf{EQWARN}, \textsf{EQINFO} and \textsf{EQVERB} allow selective
debugging outputs with various logging levels:
{\footnotesize\begin{lstlisting}
// 2. Equalizer initialization
NodeFactory nodeFactory;
if( !eq::init( argc, argv, &nodeFactory ))
{
EQERROR << "Equalizer init failed" << endl;
return EXIT_FAILURE;
}
\end{lstlisting}}%>>
The node factory is used by Equalizer to create the object instances for
the rendering entities. Each of the classes inherits from the same type
provided by Equalizer in the \textsf{eq} namespace. The provided
\textsf{eq::NodeFactory} base class instantiates a 'plain' Equalizer
object, thus making it possible to selectively subclass individual
entity types. For each rendering resource used in the configuration, one
C++ object will be created:
{\footnotesize\begin{lstlisting}
class NodeFactory : public eq::NodeFactory
{
public:
virtual eq::Config* createConfig() { return new eqPly::Config; }
virtual eq::Node* createNode() { return new eqPly::Node; }
virtual eq::Pipe* createPipe() { return new eqPly::Pipe; }
virtual eq::Window* createWindow() { return new eqPly::Window; }
virtual eq::Channel* createChannel() { return new eqPly::Channel; }
};
\end{lstlisting}}
The third step is to create an instance of the application and to
initialize it locally. The application is an \textsf{eq::Client}, which
is an \textsf{eqNet::Node}. The underlying network distribution in
Equalizer is a peer-to-peer network structure of
\textsf{eqNet::Node}s. The application programmer rarely is aware of the
classes in the \textsf{eqNet} namespace, but both the
\textsf{eq::Client} and the server are \textsf{eqNet::Node}s. The local
initialization of nodes creates a local listening socket, so that the
node, and therefore the \textsf{eq::Client} can communicate over the
network with other nodes, such as the server and the rendering clients.
{\footnotesize\begin{lstlisting}
// 3. initialization of local client node
RefPtr< eqPly::Application > client = new eqPly::Application( initData );
if( !client->initLocal( argc, argv ))
{
EQERROR << "Can't init client" << endl;
eq::exit();
return EXIT_FAILURE;
}
\end{lstlisting}}%>>
Finally everything is set up to run the \textsf{eqPly} application:
{\footnotesize\begin{lstlisting}
// 4. run client
const int ret = client->run();
\end{lstlisting}}
After it has finished, the application and Eqply is deinitialized and
the \textsf{main} function returns:
{\footnotesize\begin{lstlisting}
// 5. cleanup and exit
client->exitLocal();
client = 0;
eq::exit();
return ret;
}
\end{lstlisting}}
\subsection{Application}
In the \textbf{eqPly} case, the application is also the render client. It has
three run-time behaviours:
\begin{enumerate}
\item \textbf{Application}: The executable started by the user, which
is the controlling entity in the rendering session.
\item \textbf{Auto-launched render client}: The typical render client,
started by the server. The server starts the executable with special
parameters, which cause \textsf{Client::initLocal} to never
return. During exit, the server terminates the process.
\item \textbf{Resident render client}: Manually pre-started render
client, listening on a specified port for server commands. This mode
is selected using the command-line option \textsf{--eq-client} and
potentially \textsf{--eq-listen} and \textsf{-r}\footnote{see \htmladdnormallink{http://www.equalizergraphics.com/documents/design/residentNodes.html}{http://www.equalizergraphics.com/documents/design/residentNodes.html}}.
\end{enumerate}
\subsubsection{Main Loop}
The application's main loop starts by connecting the application to an
Equalizer server. The command line parameter \textsf{--eq-server}
explicitly specifies a server address. If no server was specified,
\textsf{Client::connectServer} try first to connect to a server on the
local machine using the default port 4242. If that fails, it will create
a server running within the applications process with a default
1-channel configuration\footnote{see
\htmladdnormallink{http://www.equalizergraphics.com/documents/design/standalone.html}{http://www.equalizergraphics.com/documents/design/standalone.html}}:
{\footnotesize\begin{lstlisting}
int Application::run()
{
// 1. connect to server
RefPtr<eq::Server> server = new eq::Server;
if( !connectServer( server ))
{
EQERROR << "Can't open server" << endl;
return EXIT_FAILURE;
}
\end{lstlisting}}%>>
The second step is to ask the server for a configuration. The
\textsf{ConfigParams} are a placeholder for later implementations to
provide additional hints and information to the server for choosing a
configuration. The configuration is created using
\textsf{NodeFactory::createConfig}. Therefore it is of type
\textsf{eqPly::Config}, but the return value is \textsf{eq::Config},
making q cast necessary:
{\footnotesize\begin{lstlisting}
// 2. choose config
eq::ConfigParams configParams;
Config* config = static_cast<Config*>(server->chooseConfig( configParams ));
if( !config )
{
EQERROR << "No matching config on server" << endl;
disconnectServer( server );
return EXIT_FAILURE;
}
\end{lstlisting}}%>>
Finally it is time to initialize the configuration. For statistics, the
time for this operation is measures and printed. During initialization
the server launches and connects all render client nodes, and calls the
appropriate initialization task methods, as explained in later
sections. \textsf{Config::init} does return after all nodes, pipes,
windows and channels are initialized. It returns \textsf{true} only if
all init task methods were successful. The \textsf{EQLOG} macro allows
topic-specific logging. The numeric topic values are specified in the
respective \textsf{log.h} header files:
{\footnotesize\begin{lstlisting}
// 3. init config
eqBase::Clock clock;
config->setInitData( _initData );
if( !config->init( ))
{
EQERROR << "Config initialization failed: "
<< config->getErrorMessage() << endl;
server->releaseConfig( config );
disconnectServer( server );
return EXIT_FAILURE;
}
EQLOG( eq::LOG_CUSTOM ) << "Config init took " << clock.getTimef() << " ms"
<< endl;
\end{lstlisting}}%>>
When the configuration was successfully initialized, the actual main
loop is executed. The main loop runs until the user exits the
configuration or a maximum number of frames, specified as a command-line
argument, has been rendered. The latter is useful for benchmarks. The
\textsf{Clock} is reused for measuring the overall performance. A new
frame is started using \textsf{Config::startFrame} and a frame is
finished using \textsf{Config::finishFrame}.
When the frame is started, the server computes all rendering tasks and
sends them to the appropriate render client nodes. The render client
nodes dispatch the tasks to the correct node or pipe thread, were they
are executed in the order they arrive.
\begin{wrapfigure}{r}{.4\textwidth}
\subfigure[]{\label{fSync}
\includegraphics[width=.18\textwidth]{images/sync}}\hfil
\subfigure[]{\label{fAsync}
\includegraphics[width=.18\textwidth]{images/async}}%
{\caption{\small\label{fSyncAsync}Synchronous and asynchronous execution}}
\end{wrapfigure}
\textsf{Config::finishFrame} synchronizes on the completion of the frame
\textsf{current - latency}. The latency is specified in the
configuration file, and allows several outstanding frames for which the
tasks are already queued in the node and pipe threads for
execution. This enables overlapped execution and minimizes idle
times. The first \textsf{latency Config::finishFrame} return
immediately, since they have no frame to synchronize
upon. Figure~\ref{fSyncAsync} shows the execution of (hypothetical)
rendering tasks without latency~(\ref{fSync}) and with a latency of
one~(\ref{fAsync}).
When the main loop is finished, \textsf{Config::finishAllFrames} catches
up with the latency. It returns after all outstanding frames have been
rendered, and is needed here to provide an accurate measurement of the
framerate:
{\footnotesize\begin{lstlisting}
// 4. run main loop
uint32_t maxFrames = _initData.getMaxFrames();
clock.reset();
while( config->isRunning( ) && maxFrames-- )
{
config->startFrame();
// config->renderData(...);
config->finishFrame();
}
const uint32_t frame = config->finishAllFrames();
const float time = clock.getTimef();
EQLOG( eq::LOG_CUSTOM ) << "Rendering took " << time << " ms (" << frame
<< " frames @ " << ( frame / time * 1000.f)
<< " FPS)" << endl;
\end{lstlisting}}%>>
The remainder of the application code cleans up in the reverse order of
the initialization. The config is exited, released and the connection to
the server is closed:
{\footnotesize\begin{lstlisting}
// 5. exit config
clock.reset();
config->exit();
EQLOG( eq::LOG_CUSTOM ) << "Exit took " << clock.getTimef() << " ms" <<endl;
// 6. cleanup and exit
server->releaseConfig( config );
if( !disconnectServer( server ))
EQERROR << "Client::disconnectServer failed" << endl;
server = 0;
return EXIT_SUCCESS;
}
\end{lstlisting}}%>>
\subsubsection{Render Clients}
In the second and third case, when the executable is used as a render
client, \textsf{Client::initLocal} never returns. Therefore the
application's main loop is never executed. In order to keep the client
resident, the \textsf{eqPly} example overrides the client loop to keep
it running beyond one configuration run:
{\footnotesize\begin{lstlisting}
bool Application::clientLoop()
{
if( !_initData.isResident( )) // execute only one config run
return eq::Client::clientLoop();
// else execute client loops 'forever'
while( true ) // TODO: implement SIGHUP handler to exit?
{
if( !eq::Client::clientLoop( ))
return false;
EQINFO << "One configuration run successfully executed" << endl;
}
return true;
}
\end{lstlisting}}%>>
\subsection{Distributed Objects}
Equalizer provides distributed objects which help implementing data
distribution in a graphics cluster. The master version of a distributed
object is registered with a \textsf{eqNet::Session}, which assigns a
session-unique identifier to the object. Other nodes can map their
instance of the object to this identifier, thus synchronizing the
object's data with the remotely registered object.
Distributed object are created by subclassing from
\textsf{eqNet::Object}. Distributed objects can be static (immutable) or
dynamic. Dynamic objects are versioned.
The \textsf{eqPly} example has a static distributed object to provide
initial data to all rendering nodes, as well as a versioned object to
provide frame-specific data, such as the camera position, to the
rendering methods.
\subsubsection{InitData - a Static Distributed Object}
The \textsf{InitData} class holds a couple of parameters needed during
initialization. These parameters never change during one configuration
run, and are therefore static.
A static distributed object either has to provide a pointer and size to
its data using \textsf{setInstanceData} or it has to implement
\textsf{getInstanceData} and \textsf{applyInstanceData}. The first
approach can be used if all distributed member variables can be outlayed
in one contiguous block of memory. The second approach is used
otherwise.
The \textsf{InitData} class contains a string of variable
length. Therefore it uses the second approach of manually serializing
and deserializing its data. The serialization is done in
\textsf{getInstanceData}. The serialized data is cached using a
(non-distributed) member variable, which is cleared by each setter of
distributed data. Serializing the data is a simple matter of allocating
a large enough memory buffer, and copying the data into the buffer:
{\footnotesize\begin{lstlisting}
const void* InitData::getInstanceData( uint64_t* size )
{
*size = sizeof( uint32_t ) + sizeof( eq::WindowSystem ) +
_filename.length() + 1;
if( _instanceData )
return _instanceData;
_instanceData = new char[ *size ];
reinterpret_cast< uint32_t* >( _instanceData )[0] = _frameDataID;
reinterpret_cast< uint32_t* >( _instanceData )[1] = _windowSystem;
const char* string = _filename.c_str();
memcpy( _instanceData + sizeof( uint64_t ), string, _filename.length()+1 );
return _instanceData;
}
\end{lstlisting}}
After the data is no longer needed, \textsf{releaseInstanceData} is
called by Equalizer to allow the object to free the data. Since the data
is cached for further usage, the release method is not overwritten.
The memory buffer returned by \textsf{getInstanceData} is transferred to
the remote node during object mapping and passed to
\textsf{applyInstanceData} for deserialization. The deserialization
routine reads the data back into its own member variables:
{\footnotesize\begin{lstlisting}
void InitData::applyInstanceData( const void* data, const uint64_t size )
{
EQASSERT( size > ( sizeof( _frameDataID ) + sizeof( _windowSystem )));
_frameDataID = reinterpret_cast< const uint32_t* >( data )[0];
_windowSystem = reinterpret_cast< const eq::WindowSystem* >( data )[1];
_filename = static_cast<const char*>( data ) + sizeof( uint64_t );
EQASSERT( _frameDataID != EQ_ID_INVALID );
EQINFO << "New InitData instance" << endl;
}
\end{lstlisting}}%>>
\subsubsection{FrameData - a Versioned Distributed Object}
Versioned objects have to override \textsf{isStatic} to return false, in
order for Equalizer to know that they should be versioned. Right now,
only the master instance of the object is writable, that is,
\textsf{eqNet::Object::commit} can be called to generate a new
version.
Upon \textsf{commit} the delta data from the previous version is sent to
all mapped slave instances. The data is queued on the remote node, and
is applied when the application calls \textsf{sync} to synchronize the
object to a new version. The \textsf{sync} method might block if a
version has not been committed or is still in transmission.
In addition to the instance data (de)serialization methods needed to map
an object, versioned objects may implement \textsf{pack} and
\textsf{unpack} to serialize or deserialize the changes since the last
version.
If the delta data happens to be layed out contiguously in memory,
\textsf{setDeltaData} might be used. The default implementation of
\textsf{pack} and \textsf{unpack} (de)serialize the delta data or the
instance data if no delta data has been specified.
The frame data is layed out in one anonymous structure in
memory. It also does not track changes since it is relatively small in
size and changes frequently. Therefore, for the instance and delta
date are the same and set in the constructor:
{\footnotesize\begin{lstlisting}
FrameData()
{
reset();
setInstanceData( &data, sizeof( Data ));
EQINFO << "New FrameData " << std::endl;
}
\end{lstlisting}}%>>
\subsection{Config}
The configuration is driving the applications rendering, that is, it is
responsible for updating the data based on received events, requesting
new frames to be rendered and to provide the render clients with the
necessary data.
\subsubsection{Config Initialization and Exit}
\begin{wrapfigure}{r}{.6\textwidth}
\includegraphics[width=.6\textwidth]{images/configInit.pdf}
{\caption{\small\label{fConfigInit}Config Initialization Sequence}}
\end{wrapfigure}
The config initialization happens in parallel, that is, all config
initialization tasks are transmitted by the server at once and their
execution is synchronized later.
The tasks are executed by the node and pipe threads in parallel. The
parent's initialization methods are always executed before any child
initialization method. This is necessary to allow a speedy startup of
the configuration on large-scale graphics clusters. On the other hand,
it means that initialization functions are called even if the parent's
initialization has failed.
The \textsf{eqPly::Config} class holds the master versions of the
initialization and frame data. Both objects are registered with the
\textsf{eq::Config}, which is the \textsf{eqNet::Session} used for
rendering. Equalizer takes care of the session setup and exit in
\textsf{Client::choose\-Config} and \textsf{Client::releaseConfig},
respectively.
The frame data is registered first, since its identifier is transmitted
using the initialization data, which is registered afterwards. The
identifier of the initialization data is transmitted to the render
client nodes using the \textsf{initID} parameter of
\textsf{eq::Config::init}. Equalizer will pass this identifier to all
\textsf{configInit} calls of the respective objects:
{\footnotesize\begin{lstlisting}
bool Config::init()
{
// init distributed objects
_frameData.data.color = _initData.useColor();
registerObject( &_frameData );
_initData.setFrameDataID( _frameData.getID( ));
registerObject( &_initData );
// init config
_running = eq::Config::init( _initData.getID( ));
if( !_running )
return false;
\end{lstlisting}}
If the configuration was initialized correctly, the configuration tries
to set up a tracking device for head tracking. Equalizer does not
provide extensive support for tracking device, as this is an orthogonal
problem to parallel rendering, and has been solved already by a number
of implementations\footnote{VRCO trackd, OpenVPRN, etc.}. The example
code in \textsf{eqPly} is one reference implementation for the
integration of such a tracking library:
{\footnotesize\begin{lstlisting}
// init tracker
if( !_initData.getTrackerPort().empty( ))
{
if( !_tracker.init( _initData.getTrackerPort() ))
EQWARN << "Failed to initialise tracker" << endl;
else
{
// Set up position of tracking system in world space
// Note: this depends on the physical installation.
vmml::Matrix4f m( vmml::Matrix4f::IDENTITY );
m.scale( 1.f, 1.f, -1.f );
//m.x = .5;
_tracker.setWorldToEmitter( m );
m = vmml::Matrix4f::IDENTITY;
m.rotateZ( -M_PI_2 );
_tracker.setSensorToObject( m );
EQLOG( eq::LOG_CUSTOM ) << "Tracker initialised" << endl;
}
}
return true;
}
\end{lstlisting}}%>>
The exit of the configuration stops the render clients by calling
\textsf{eq::Config::exit}, and then deregisters the initialization and
frame data objects with the session:
{\footnotesize\begin{lstlisting}
bool Config::exit()
{
_running = false;
const bool ret = eq::Config::exit();
_initData.setFrameDataID( EQ_ID_INVALID );
deregisterObject( &_initData );
deregisterObject( &_frameData );
return ret;
}
\end{lstlisting}}
\subsubsection{Frame Control}
The rendering frames are issued by the application. The \textsf{Config}
only overrides \textsf{startFrame} in order to update the its data
before forwarding the start frame request to the \textsf{eq::Config}.
If a tracker is used, the current head position and orientation is
retrieved and given to Equalizer, which uses the head matrix together
with the wall or projection description to compute the view
frustra\footnote{see
\htmladdnormallink{http://www.equalizergraphics.com/documents/design/immersive.html}{http://www.equalizergraphics.com/documents/design/immersive.html}}.
The camera position is updated and the frame data is commited, which
generates a new version. This version is passed to the rendering
callbacks and will be used to synchronize the frame data to the state
belonging to the current frame:
{\footnotesize\begin{lstlisting}
uint32_t Config::startFrame()
{
// update head position
if( _tracker.isRunning() )
{
_tracker.update();
const vmml::Matrix4f& headMatrix = _tracker.getMatrix();
setHeadMatrix( headMatrix );
}
// update database
_frameData.data.rotation.preRotateX( -0.001f * _spinX );
_frameData.data.rotation.preRotateY( -0.001f * _spinY );
const uint32_t version = _frameData.commit();
return eq::Config::startFrame( version );
}
\end{lstlisting}}
\subsubsection{Event Handling}
Events are send by the render clients to the application using
{eq::Config::sendEvent}. At the end of the frame,
\textsf{Config::finishFrame} calls \textsf{Config::handleEvents} to do
the event handling. The default implementation processes all pending
events by calling \textsf{Config::handleEvent} for each of them.
For event-driven execution, the application can override
\textsf{Config::handleEvents} to blockingly receive events using
\textsf{Config::nextEvent} until a new frame has to be rendered.
The \textsf{eqPly} example continuously renders new frames. It
implements \textsf{Config::hand\-le\-Event} to provide the various reactions
to user input:
{\footnotesize\begin{lstlisting}
bool Config::handleEvent( const eq::ConfigEvent* event )
{
switch( event->type )
{
case eq::ConfigEvent::WINDOW_CLOSE:
_running = false;
return true;
[...]
default:
break;
}
return eq::Config::handleEvent( event );
}
\end{lstlisting}}
\subsection{Node}
Foreach active render client, one \textsf{eq::Node} instance is
created on the appropriate machine. Nodes are only instantiated on their
render client processes, i.e., each process should have only one
instance of the \textsf{eq::Node} class. The application process might
also have a node class, which is handled in exactly the same way as the
render client nodes.
During node initialization the initialization data is mapped to
a local instance using the passed identifier from
\textsf{Config::init}. The model is loaded based on the filename in the
initialization data. No pipe, window or channel tasks methods are
executed before \textsf{Node::configInit} has returned.
{\footnotesize\begin{lstlisting}
bool Node::configInit( const uint32_t initID )
{
eq::Config* config = getConfig();
const bool mapped = config->mapObject( &_initData, initID );
EQASSERT( mapped );
const string& filename = _initData.getFilename();
EQINFO << "Loading model " << filename << endl;
_model = PlyFileIO::read( filename.c_str( ));
if( !_model)
EQWARN << "Can't load model: " << filename << endl;
return eq::Node::configInit( initID );
}
\end{lstlisting}}%>>
The node config exit deletes the loaded model and unmaps the
initialization data:
{\footnotesize\begin{lstlisting}
bool Node::configExit()
{
delete _model;
_model = NULL;
eq::Config* config = getConfig();
config->unmapObject( &_initData );
return eq::Node::configExit();
}
\end{lstlisting}}
\subsubsection{Frame Control}
\begin{wrapfigure}{r}{.6\textwidth}
\includegraphics[width=.6\textwidth]{images/mainloop.pdf}
{\caption{\small\label{fFrameSync}Synchronization of frame tasks}}
\end{wrapfigure}
The application has extended control over the task synchronization
during a frame. Upon \textsf{Config::startFrame}, Equalizer invokes the
\textsf{frameStart} task methods of the various entities. The entity
unlock all its children by calling \textsf{startFrame}, e.g., the
\textsf{Node::frameStart} has to call \textsf{Node::startFrame} in order
to unlock the pipe threads. Note that certain \textsf{startFrame} calls
are currently empty, as the synchronization is implicit due to the
threading model.
Likewise, \textsf{Config::finishFrame} causes the invokation of the
\textsf{frameFinish} task methods. These task methods unlock their
parents by calling \textsf{finishFrame}.
The explicit synchronization of child or parent resources allows the
application to optimize the processing, by doing certain, independent
operations when the child or parent resources are already unlocked.
\fig{fFrameSync} outlines the synchronization for the application, node
and pipe classes. The window and channel synchronization is similar and
omitted for simplicity. The \textsf{eqPly} example does not override
\textsf{Node::frameStart} or \textsf{frameFinish}, but it is absolutely
vital for the execution that \textsf{Node::startFrame} or
\textsf{Node::finishFrame} are called, respectively. The default
implementation of the node task methods does take care of that.
\subsection{Pipe}
All task methods for a pipe and its children are executed in a separate
thread. This approach optimizes usage of the GPU resource, since all
tasks are executed serially and do not compete for the usage, i.e.,
context switches are minimized. Later versions of Equalizer might
introduce threaded windows to allow the parallel and independent
execution of rendering tasks on a single pipe.
\subsubsection{Initialization and Exit}
Pipe threads are not explicitely synchronized with each other, that is,
pipes might be rendering different frames at one given time. Therefore
frame-specific data has to be allocated for each pipe thread, which in
the \textsf{eqPly} example is the frame data. The frame data is a member
variable of the \textsf{eqPly::Pipe}, and is mapped to the identifier
provided by the initialization data. The initialization in
\textsf{eq::Pipe} does GPU-specific initialization, e.g., the display
connection is opened when glX/X11 is used:
{\footnotesize\begin{lstlisting}
bool Pipe::configInit( const uint32_t initID )
{
const Node* node = static_cast<Node*>( getNode( ));
const InitData& initData = node->getInitData();
const uint32_t frameDataID = initData.getFrameDataID();
eq::Config* config = getConfig();
const bool mapped = config->mapObject( &_frameData, frameDataID );
EQASSERT( mapped );
return eq::Pipe::configInit( initID );
}
\end{lstlisting}}
The config exit is again symmetric to the config initialization. The
frame data is unmapped and GPU-specific data is de-initialized by
\textsf{eq::Config::exit}:
{\footnotesize\begin{lstlisting}
bool Pipe::configExit()
{
eq::Config* config = getConfig();
config->unmapObject( &_frameData );
return eq::Pipe::configExit();
}
\end{lstlisting}}
\subsubsection{Window System}
Equalizer supports multiple window system interfaces, at the moment
glX/X11, WGL and AGL/Carbon. Some operating systems, and therefore some
Equalizer versions, support multiple window systems\footnote{see
\htmladdnormallink{http://www.equalizergraphics.com/compatibility.html}{http://www.equalizergraphics.com/compatibility.html}}.
Each pipe might use a different window system for rendering, which is
determined before \textsf{Pipe::configInit} by
\textsf{Pipe::selectWindowSystem}. The default implementation of
\textsf{selectWindowSystem} loops over all window systems and returns
the first supported window system, determined using
\textsf{supportsWindowSystem}.
The \textsf{eqPly} examples allows selecting the window system using a
command line option. Therefore the implementation of
\textsf{selectWindowSystem} is overwritten, and return the specified
window system, if it is valid:
{\footnotesize\begin{lstlisting}
eq::WindowSystem Pipe::selectWindowSystem() const
{
const Node* node = static_cast<Node*>( getNode( ));
const InitData& initData = node->getInitData();
const eq::WindowSystem ws = initData.getWindowSystem();
if( ws == eq::WINDOW_SYSTEM_NONE )
return eq::Pipe::selectWindowSystem();
if( !supportsWindowSystem( ws ))
{
EQWARN << "Window system " << ws
<< " not supported, using default window system" << endl;
return eq::Pipe::selectWindowSystem();
}
return ws;
}
\end{lstlisting}}%>>
\subsubsection{Frame Control}
TBD
{\footnotesize\begin{lstlisting}
void Pipe::frameStart( const uint32_t frameID, const uint32_t frameNumber )
{
_frameData.sync( frameID );
startFrame( frameNumber );
}
\end{lstlisting}}
\if 0
{\footnotesize\begin{lstlisting}
\end{lstlisting}}
\fi
\end{document}
See more files for this project here