user-guide.tex from GridBlocks at Krugle

Show user-guide.tex syntax highlighted

%\documentclass[10pt]{article} 
%\usepackage{times} \usepackage{url} \usepackage{here} 
%\usepackage{graphicx}
%\documentstyle[times,art10,twocolumn,latex8]{article}
\documentclass[a4paper,english, titlepage, 10pt]{article}
\usepackage[english]{babel}
\usepackage[latin1]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{times} 
\usepackage{url} 
\usepackage{here} 
\usepackage{graphicx}

%\oddsidemargin=0.10in    %left margin is 1 inch on right-hand pages
%\evensidemargin=0.10in   %same for left-hand pages in twosided document
%\textwidth=6.3in
\topmargin=-0.8in
\textheight=9.82in

%---------------------------------------------------------
\pagestyle{empty}
%---------------------------------------------------------

\begin{document}

\vspace*{2in}
\begin{center}

\begin{figure}[htb]
  \begin{center}
    \includegraphics[width=8.0cm]{gblogo_600}
    \label{cover_logo}
  \end{center}
\end{figure}

\section*{DISK version 1.0 - User Guide}
\end{center}
\clearpage

%---------------------------------------------------------------
\section{What GB-DISK Is Used For}
%---------------------------------------------------------------

GridBlocks project has made it easy for a user to store and access his
personal files location independently. The user can access the storage
service without need for any special client software. Web interfaces
makes it possible to use the storage from devices varying from desktop
computers to mobile phones.

DISK is designed so that it can be installed with greatest ease. By
installing DISK to his home or office computer the user can gain access to
his files from remote location. DISK installation also enables user to
make his personal files securely available amongst his friends and family.


%---------------------------------------------------------------
\section{GB-DISK Introduction}
%---------------------------------------------------------------

While the price of the disk space has come down, the reliable storage
solutions have remained expensive. The reliable storage is usually
constructed from expensive high-end hardware such as SCSI disks in
widely used RAID-5 storages. Moreover, need for maintenance cost as
staff has to replace broken disks in short notice adds costs of
storage.  With GridBlocks DISK, very reliable high-performance storage
can be created by leveraging an open source Java software and
commodity hardware.

GB-DISK can aggregate storage resources to unified pool of highly
reliable disk-space. The software is to be used by system
administrators and the end users see the whole system as a simple web
folder or as a portal interface. Adding and removing storage to the
pool and maintaining and adjusting reliability of stored information
is invisible to the end users.  We call the mechanisms by which
resources can be added, removed and shared on the fly without
operational interruption "Grid functionality".

GB-DISK uses erasure codes to achieve high availability with
theoretically minimal storage overhead.  By using erasure codes we
replace need for costly high quality storage devices. The amount of
parity information can be freely chosen to meet the storage robustness
requirements.  Computing nodes are clustered to increase throughput of
parity calculations.

%---------------------------------------------------------------
\section{System Overview}
%---------------------------------------------------------------

\begin{figure*}[!htb]
\centering
\includegraphics[width=4.0in]{arch2.pdf}
\caption{Functional parts in GB-DISK software.}
\label{arch}
\end{figure*}


The GB-DISK architecture is illustrated in Figure \ref{arch}. We have
grouped the functionality of the GB-DISK into the following core
entities:

\begin{itemize}

\item {\bf Front-end} (FE) hosts are access points to the storage
  system running the {\it Coordinator} that orchestrates the storage
  work-flow.  The Coordinator controls the file striping and is
  responsible for keeping the storage information up-to-date in the
  metadata service. The Coordinator manages the leaving and joining of
  storage elements. Additionally, the Coordinator provides a simple
  API for third party applications and user interfaces.

\item {\bf Metadata service} stores information about files including
  erasure coding parameters, stripe integrity and respective storage
  locations.  The service is implemented using JBossCache project
  offering transparent and automatic replication of data objects over
  a reliable multicast protocol. The resulting design leads to a
  scalable, highly-available and fault tolerant metadata
  service. Currently, metadata service is bundled within front-end
  nodes to provide local access to the replicated metadata.
  
\item {\bf Storage elements} (SE) contribute storage space for storing
  the stripes. No intelligence resides in storage element side and the
  performance impact to hosts is limited to network and disk I/O. The
  SEs can be easily deployed into many kind of environments varying
  from dedicated cluster nodes to unreliable workstations.  In
  addition to GB-DISK storage element software, existing Apache HTTP,
  WebDAV and GridFTP servers can be used to aggregate storage
  space. However, automatical resource discovery works only withing
  GB-DISK servers.
  
\end{itemize}

The minimal installation of GB-DISK requires a single front-end server
and one storage element.  These services can be run in the same
host. In more realistic deployments, multiple front-ends are run to
provide load balancing and high availability of the metadata and
erasure coding.  After striping and coding the file fragments are
stored into $n$ separate storage elements for fault tolerance and
performance.

A user can connect to the storage system using command line utilities
from his or her workstation or using a web interface which runs in the
front-end. The web interface can be used through HTTP browser or any
WebDAV folder utility.  The simple interface and automatized working
of the coordinator enables easy integration as a back-end of different
storage services using only HTTP protocol.

\subsection{Usage Scenario}

From end-user's point of view the only visible part of GB-DISK is a
web folder where new files can be uploaded and stored files
fetched. The entire distribution of files is hidden behind the
scenes. However, internally an example file upload to GB-DISK proceeds
as follows.

\begin{itemize}
\item Client uploads a file into front-end.
\item The FE stripes the file and uses erasure codes to provide redundancy. Integrity is 
ensured by using cryptographic hashes.
\item The FE distributes stripes to multiple storage-elements (SE) and stores 
distribution meta-data to infosystem.
\item Metadata stays shared in the infosystem. 
\end{itemize}

An example file download from GB-DISK proceeds as follows.

\begin{itemize}
\item Client requests a file from the FE.
\item The FE looks up metadata from infosystem and starts to download files from many SEs. It 
is sufficient to download K out of N fraqments from the fastest available SEs.
\item When enough stripes have arrived to the FE the file is decoded.
\item The FE responses to the file request by sending the file to the client.. 
\end{itemize}

\section{Basic concepts}

\subsection{Storage structure}

By default, GB-DISK has storage volumes for public and private
data. Depending on storage configuration, non authenticated users
cannot access the public section at all, they can only read files or
they can also modify them. Authenticated users are always able to read
and modify public files.

Private folder under root directory is reserved for registered
users. Each user has a home directory for private use and all groups
have directory for their group members only. Any other data under
private folder is available for all authenticated members.

\subsection{User accounts}

Users cannot create or modify user account themselves, everything has
to be done by system administrator.

Authentication is done by navigating to \url
{http://server.url/gb-disk/private} folder and entering user name and
password combination. Once authenticated, the user can write into the
public root folder or stay in the private section visible only for
registered users.

%---------------------------------------------------------------
\section{How to Use GB-DISK}   
\label{usage}
%---------------------------------------------------------------


This sections describes the usage of GB-DISK from end-users point of
view. If you want to learn how to store and access files through
already installed DISK service read this Section. If you want to
install your own file server check out the Admin guide
instead.

\subsection{Different Ways to Use the service}

GB-DISK installation provides three different ways to contact the
service. The first and recommended way is to use the WebDAV
interface. WebDAV stands for
"Web-based Distributed Authoring and Versioning": A set of extensions
to the HTTP protocol allows users to collaboratively edit and manage
files on remote web servers.
With DAV interface you can open your shared folder so that
it appers to be just an ordinary file folder in your computer. We
provide instructions for DAV usage in Windows environment. If you have
a limited environment (e.g. mobile device ) or you prefer web
interface a browser can be used to access the files.  Finally for
advanced users a command line tool is offered.

\subsubsection{WebDAV on Microsoft Windows}

Microsoft Windows XP has a support for {\it web folders}. With web
folders you can make a distant folder (in this case DISK folder) to
appear as any other folder in your computer.


To start using DISK in computers running Windows you should first open
a new Internet Explorer window. Then, click {\bf Open} from {\bf File}
menu (or ALT+O) which brings up a {\tt open address} dialog:

\begin{figure*}[H]
\centering
\includegraphics[width=5.0in]{webdav_open.pdf}
\end{figure*}

Type the front-end server address, or try our Opengrid Test-bed at
\url{http://opengrid.cern.ch:8888/gb-disk}.  Remember to check the
box {\bf Open as Web Folder}. Then press the {\bf OK} button to
connect to the server and to list available files and folders.

\begin{figure*}[H]
\centering
\includegraphics[width=5.0in]{webdav_show.pdf}
\end{figure*}


WebDAV folders can also be made available through {\bf My Network
Places}.  Select the {\bf Add a network place} from left-hand side of
the window.

\begin{figure*}[H]
\centering
\includegraphics[width=5.0in]{webdav_mnp.pdf}
\end{figure*}

Continue using {\bf Add Network Place Wizard} and pick up {\bf Choose
another network location} from the list of service provides. Then
input the address of the server e.g.
\url{http://opengrid.cern.ch:8888/gb-disk}.

\begin{figure*}[H]
\centering
\includegraphics[width=5.0in]{webdav_mount.pdf}
\end{figure*}

\begin{figure*}[H]
\centering
\includegraphics[width=5.0in]{webdav_mount_address.pdf}
\end{figure*}

Finally, give a convenient name to the service and complete the
wizard. A new item should be listed on the {\bf My Network Places}.

\begin{figure*}[H]
\centering
\includegraphics[width=5.0in]{webdav_mounted.pdf}
\end{figure*}

\subsubsection{Web browser interface}

If you prefer to use a simple and easy to use interface through your
favourite web browser we have made it easy. Just write the address of
the service to the addressbar of your browser and you can start
uploading and downloading files. You may also try our test-bed at
\url{http://opengrid.cern.ch:8888/gb-disk}.
 
\vspace*{1.0cm}
\begin{tabular}{p{4.0cm} p{7.0cm}}
\hline

{\bf Downloading Files} & You can download any file listed in the
front-page by simply clicking the name of the file.\\ \hline

{\bf Uploading Files} & In the middle of page is a function for
uploading a file. By using the {\bf Browse} button you can browse
through your filesystem and select the file to be uploaded. After
selecting the file press {\bf Upload} to send the selected file to the
service. \\\hline

\end{tabular}
\vspace*{0.5cm}

\subsubsection{Command-line interface}

The command-line WebDAV client can be used to send and receive files
from command prompt or script languages. The usage is simple, however,
not as intuitive as with graphical WebDAV clients.

Using the command-line utility requires gb-disk.jar from the
Sourceforge download page and Java Runtime Environment (JRE) version
5. The command-line client supports following options:

\begin{verbatim} 
   java -jar gb-disk.jar [GET/DELETE/EXISTS] [URL into remote file/directory]
   java -jar gb-disk.jar [PUT/GET/DELETE/EXISTS] [URL into remote directory] [file name(s)]
   java -jar gb-disk.jar [LIST] [URL into directory]
\end{verbatim} 

For example, simple operations can be performed with direct URL to 
a file or directory on local front-end:

\begin{verbatim} 
   java -jar gb-disk.jar GET http://localhost:8080/gb-disk/file1
   java -jar gb-disk.jar GET http://localhost:8080/gb-disk/dir1/
   java -jar gb-disk.jar EXISTS http://localhost:8080/gb-disk/file1
   java -jar gb-disk.jar DELETE http://localhost:8080/gb-disk/file1
   java -jar gb-disk.jar LIST http://localhost:8080/gb-disk/dir1/
\end{verbatim} 

Multiple files can be handled by adding them after the URL to a directory:

\begin{verbatim} 
   java -jar gb-disk.jar GET http://localhost:8080/gb-disk file1 file2
   java -jar gb-disk.jar PUT http://localhost:8080/gb-disk file1 file2
   java -jar gb-disk.jar EXISTS http://localhost:8080/gb-disk file1 file2
   java -jar gb-disk.jar DELETE http://localhost:8080/gb-disk file1 file2
\end{verbatim} 

\end{document}
See more files for this project here