With the advent of Web 2.0, collaborative computing and user content generation has seen tremendous growth on real world networks in recent times. Shared drawing board is an important component for any suite of applications meant for collaborative computing. We designed and developed a shared white board application using the Real-Time Transport Protocol (RTP) as transport protocol. The drawing objects on the white board are stored in SVG (Scalable Vector Graphics) format. This allows standardization of transport medium as well as the data format. This is also aimed at achieving platform independence as any application which supports RTP protocol, has SVG editor and shares the message protocol can interact with our shared white board sessions.
We designed and developed a shared white board application. There are three major components in our shared white board application: the SVG editor, RTP sender/receiver, and the communication protocol used to transmit data. The remaining sections of the report will discuss all these components in detail.
Scalable Vector Graphics (SVG) is a language for describing two-dimensional graphics and graphical applications in XML. Its has two parts: an XML-based file format and a programming API for graphical applications. It includes shapes, text and embedded raster graphics. Its applications include web graphics, animation, user interfaces, graphics interchange, mobile applications and high-quality design. W3C has embedded an SVG editor in its test suite called Amaya. We used the SVG editor provided by Amaya in our application.
Real-Time Transport Protocol (RTP) is the Internet-standard protocol for transport of real-time data. RTP has been designed in such a way that it caters to all kind of real time data not just audio and video. It is used for interactive media services like Internet telephony. RTP consists of a data and a control part. The control part is called RTCP. RTP is mostly used over UDP/IP which is unreliable. In a white board application data flow is real-time and discontinuous. This required RTP to be used over TCP/IP to provide reliable communication. We modified the oRTP library to implement TCP framing.
Our communication protocol is based on multiple point-to-point connections and a client/server model. A server hosts a white board session. The client connects to shared white board session of server through a point-to-point TCP/IP connection. Whenever a client makes changes to the white board canvas, those changes are communicated to other clients by the server.
Shared white board applications have been developed by many research groups in the past. Some of the significant shared drawing tools are ClearBoard [4], GroupSketch [5], VideoDraw [7], MediaBoard [10] and the Java multicast-based whiteboard [11]. The first four use multiple point-to-point connections to transmit information to multiple remote participants. MediaBoard developed at Berkeley was part of the MBone [8] framework. MediaBoard was designed to scale well with increasing number of participants because it was based on multicast. Most attempts to develop shared white board have centered around two methodologies - multicast messaging and server based design.
In [11], the authors used parts of Scalable, Reliable Multicast (SRM) [2] and RTP to develop a shared white board application. The LBL whiteboard application, wb, is a remote collaboration application which uses multicast. wb uses a Scalable, Reliable Multicast protocol called SRM [2] in order to avoid excess network packet loss. The most important feature of SRM is a set of protocol mechanisms that reduce the number of duplicate retransmission requests and replies which in turn reduces network bandwidth usage.
To initialize the session MediaBoard uses the Session Announcement Protocol (SAP) to multicast advertisements that contain the session address.
Amaya is a web editor as well as web browser provided by WWW Consortium as proof of concept for its technologies. Amaya can be used to create and update documents directly on web. It integrates browsing with editing. It has grown into a framework which integrates many W3C technologies like XHTML, MathML, SVG and CSS style sheets.
Amaya is open source software project written in C. It is available for multiple platforms like Windows, Unix and MacOS X. Amaya is a continuously evolving application. Recently, W3C released version 9.54 of the software.
Figure 1: SVG Editor from Amaya
SVG editor of Amaya implements the following SVG elements:
The implementation of shared white board supports all of above except open spline and closed spline. Figure 1 gives a snapshot of SVG editor used in our shared white board. Figure 2 shows sharing of images in shared white board.
Figure 2: Share Pictures in White board.
Undo Mechanism: Amaya has an in-built undo mechanism in which each operation performed by the user is stored in undo queue as an operation. If user chooses to undo her previous operation, then the undo queue is used. In our shared white board, this undo queue is used to extract information about previous operations performed by the user.
A user can perform operations on two types of objects, namely element and attribute A user can add, delete and modify an element or attribute.
Following data-structure was designed to capture an operation:
typedef struct _SingleOperation {
struct _SingleOperation *next;
OperationType opType; // 1 add, 2 delete, 3 modify
OperandType operandType; // 1 element, 2 attribute
union {
struct { // for attribute
int attrType;
int attrNum;
int defAttr;
char attrValue[LgMaxAttrText];
char *elementXPATH;
} s1;struct {
int elType;
char *createdElXPATH;
char *deletedElXPATH;
char *pSiblingXPATH;
char *parentXPATH;
char *text;
}s2; // for element
} u;
} SingleOperation;
4.1 Application specific data format: The the application uses a message format to represent the changes in canvas. This message format is a combination of meta data about the operation and XPath of the element on which the operation has been performed.
A single burst of continuous packets will describe one operation. An operation can consist of several sub-operations. For example creating a new circle consists of four sub-operations. The first sub-operation creates a new SVG circle element. The second sub-operation defines x-coordinate attribute of the circle's center. The third sub-operation defines y-coordinate of the circle's center. And the fourth sub-operation defines the radius of the circle.
A sub-operation is defined as a tuple:
sub-operation = { operation-type, operand-type, operation-specific-data };
operation-type: A sub-operation can be of three types:
operand-type: A sub-operation can be performed on two types of operands:
operation-specific-data: When the operand type is Element, then operation specific data is following tuple:
{ element, xpath-of-created-element, xpath-of-deleted-element, xpath-of-previous-sibling, xpath-of-parent }
when operand is an attribute operation specific data is following tuple:
{ attribute-type, attribute, attribute-value, parent-element-xpath }
Examples:
A single operation which changes color of a circle
Operation Encoding
Number of sub-operations 1
OperationModify 3
OperandAttribute 2
AttributeTypeText 1
Attribute-Fill 26
Attribute-value #FF0000
Length of parent-xpath 17
parent-xpath /svg[1]/circle[1]
An operation that creates a new circle is:
Operation Encoding
Number of sub-operations 4
OperationAdd 1
OperandElement 1
Element-Circle 28
Length of created-element-xpath 17
Created-element-xpath /svg[1]/circle[1]
Previous-sibling-xpath-length 15
Previous-sibling-xpath /svg[1]/line[2]
Parent-xpath-length 7
Parent-xpath /svg[1]
Operation-Add 1
Operand-Attribute 2
Attribute-type-numeric 1
Attribute-x-coordinate 121
Attribute-value 487px
Element-xpath-length 17
Element-xpath /svg[1]/circle[1]
Operation-Add 1
Operand-Attribute 2
Attribute-type-numeric 1
Attribute-y-coordinate 120
Attribute-value 390px
Element-xpath-length 17
Element-xpath /svg[1]/circle[1]
Operation-Add 1
Operand-Attribute 2
Attribute-type-numeric 1
Attribute-Radius 122
Attribute-value 50px
Element-xpath-length 17
Element-xpath /svg[1]/circle[1]
4.2 Protocol message format: A generic message format was designed based on XPath language. In this message format each operation is a collection of XPaths. Each XPath is separated by semi-colon.
Examples:
An operation which changes fill color of a circle is as follows
/svg[1]/circle[1][@fill="#FF0000"]
An operation to add a new circle is as follows
/svg[1]/circle[1];
/svg[1]/circle[1][@cx="487px"];
/svg[1]/circle[1][@cy="390px"];
/svg[1]/circle[1][@r="50px"];
When this message format is used, the application should ensure that every new SVG element that is created has same siblings in server and clients.
Image File Data: If user creates an image object then image files are also sent over RTP. In order to distinguish between SVG operation data and file data begin-file and end-of-file delimiters are used at beginning and end of the file respectively.
oRTP is a library that implements the Real-time Transport protocol and is written in C. It is easy to use, provides a packet scheduler for sending and receiving packets on time. It implements adaptive jitter compensation by default. oRTP was originally designed for Linphone, an open source SIP softphone.
TCP framing: RTP is used as transport protocol over UDP/IP in Linphone. In an application like shared white board the data is discontinuous. Due to discontinuous nature of data RTP has to be used over TCP/IP because UDP/IP will result in high packet loss. To support RTP over TCP framing was implemented. In order to implement framing in TCP, a field called 'length' was added to RTP header which gives the size of an RTP packet.
typedef struct rtp_header
{
uint16_t dummy;
uint16_t length;
#ifdef ORTP_BIGENDIAN
uint16_t version:2;
uint16_t padbit:1;
uint16_t extbit:1;
uint16_t cc:4;
uint16_t markbit:1;
uint16_t paytype:7;
#else
uint16_t cc:4;
uint16_t extbit:1;
uint16_t padbit:1;
uint16_t version:2;
uint16_t paytype:7;
uint16_t markbit:1;
#endif
uint16_t seq_number;
uint32_t timestamp;
uint32_t ssrc;
uint32_t csrc[16];
} rtp_header_t;
Jitter Compensation: For voice data, the oRTP library buffers packets for certain period of time instead of delivering it to the application. This is called jitter compensation. Jitter compensation is required for voice data but it is not required for discontinuous data like shared white board application. So jitter compensation was set to zero for the purposes of shared white board.
For communication protocol two possible designs were considered:
The current version of our shared white board implements point-to-point protocol using TCP sockets.
Figure 2: Shared White Board Architecture
Figure 2 explains software architecture of the shared white board application. A user gives input to the user interface of Amaya. SVG engine of Amaya works on the SVG document. For each operation performed by the user operation is also stored in undo queue. The undo mechanism of Amaya triggers a function in whiteboard module. This function identifies the changes made to the canvas and converts them to the message format.
The communication module has three major components: RTP sender, RTP receiver and the communication protocol
The communication module exposes API to interact with Amaya engine and the network. Communication module loads RTP library dynamically.
Program documentation is available on following link. Please also refer to the possible enhancements for a list of possible enhancements to the shared white board.
SVG editor was taken from Amaya code base. oRTP is open source library. We implemented TCP framing in oRTP. The rest of the project was implemented by Gaurav Gupta.