Experiences with building a continuous media application on Real-Time Mach

Experiences with building a Continuous Media Application on Real-Time Mach y Hiroshi Tezuka Tatsuo Nakajima [email protected] [email protected] http://mmmc.jaist.ac.jp:8000/ Japan Advanced Institute of Science and Technology 15 Asahidai, Tatsunokuchi, Ishikawa, 923-12, JAPAN Abstract Several continuous media applications such as MPEG Real-time system and continuous media system re- Audio Video Player[1], Continuous Media Player[14], searchers have worked in areas that are considered by VuSystem[3], and multimedia conference tools on each to be unrelated to the other's domain although con- MBONE[6] are developed on Unix. These systems pro- tinuous media system researchers agree with the eec- vide media control techniques for smoothing continuous tiveness of real-time technologies on continuous media media streams even when an amount of allocated re- systems. sources such as CPU and memory used by applications Media control techniques for improving the smooth- may be changed during their executions. ness of continuous media such as jitter control and On the other hand, real-time system researchers focus inter-stream synchronization schemes are developed for on resource management techniques[8, 16] for ensuring smoothing continuous media streams running on tra- timing constraints of continuous media streams. How- ditional timesharing operating systems. On the other ever, these researches do not discuss whether real-time hand, real-time system researchers show the eectiveness technologies have in uence on media control techniques of real-time technologies for satisfying timing constraints for smoothing continuous media streams. In [13], we of the continuous media. However, there is no report to showed that real-time technologies are useful for stabi- showing the eectiveness of real-time technologies for the lizing a dynamic QOS control, but the paper did not media control techniques. describe the eect of real-time technologies for media This paper describes experiences with building a con- control techniques. tinuous media application on Real-Time Mach. First, we present the structure of our distributed video player, In order to evaluate the eectiveness of real-time op- called QtPlay, and its media control techniques for erating systems for continuous media applications, we smoothing continuous media streams, then we show the implemented a distributed movie player on Real-Time eectiveness of real-time technologies for the media con- Mach[17, 9, 10]. The distributed movie player, called trol techniques. QtPlay, supports multiple clients which retrieve multi- ple movies from movie les on disks, and provides VCR 1 Introduction capability for controlling the movies. Also, it uses a real- time network system, called NPS, and a storage system, Distributed continuous media applications require op- called CRAS, for transmitting continuous media on net- erating system support to provide exible real-time han- works and retrieving it from disks. In this paper, rst, dling of audio and video streams. Continuous media we present the structure of QtPlay and its media con- applications should playback audio and video data in a trol techniques such as a jitter control scheme, an inter- timely fashion even when these data are delivered from stream synchronization scheme, and an overload detec- dierent machines. Also, audio and video streams should tion mechanism, then we show the eectiveness of real- be scaled down independently according to network and time technologies for the media control techniques. CPU load. Traditional operating systems such as Unix may not The remainder of this paper is structured as follows. guarantee timing constraints of continuous media appli- In Section 2, we present operating system support for cations, since each application shares processor resource continuous media on Real-Time Mach, and we describe fairly. Real-time operating systems are considered to be an overview of QtPlay in Section 3. Section 4 presents useful for building continuous media applications since the structure of QtPlay. Especially, we describe the role their real-time resource managements can satisfy their of threads used for processing audio samples and video timing constraints. frames. In Section 5, we describe media control tech- y He niques used in QtPlay. In Section 6, we show the evalu- now belongs to Real World Computing Partnership ation of QtPlay, and show the eectiveness of QtPlay on Tsukuba Mitsui Building 16F, 1-6-1 Takezono, Tsukuba, Ibaraki, Real-Time Mach. Section 7 describes experiences with building QtPlay on Real-Time Mach. 305, JAPAN. 2 System Support for Continuous Media makes it safe to use critical regions in multi-threaded on Real-Time Mach servers. The facilities supported by real-time IPC[9] make it In this section, we describe a brief overview of Real- possible for a receiver thread to inherit the priority of Time Mach, NPS and CRAS, which are used to imple- a sender thread, or to inherit a specied priority by the ment our distributed video player. The detailed informa- sender that is dierent from the sender's. tion of these systems are found in [9, 10, 11, 12, 17, 18]. Currently, Lites[5], which is a Unix server providing bi- 2.2 NPS: Network Protocol Server nary compatibility of FreeBSD, is running on Real-Time Mach. Thus, many FreeBSD binaries can be run on Real- NPS is a network system which is suitable for support- Time Mach without recompiling. ing real-time communication. NPS can avoid the priority inversion problem in the network server, and propagate 2.1 Overview of Real-Time Mach the priority of a sender to a receive using the Prioritized IP protocol. This section presents an overview of NPS. The microkernel-based operating system architecture is widely recognized as a promising approach for provid- Priority inversion should be avoided for building real- ing better modularity and extensibility. A microkernel time network systems. NPS adopts RT-IPC and real- provides basic resource management functions such as time synchronization provided by Real-Time Mach ker- processor scheduling, memory object management, IPC nel for avoiding unbounded priority inversions. In net- facilities, and low-level I/O support. Traditional func- work systems, fast responses are very important, NPS tions of operating systems such as le services and net- adopts the static prioritized worker model[10] for reduc- work services are all implemented in server tasks which ing the worst case duration of priority inversion. When run as user-level application programs[4]. Several oper- a new request comes to the network server, the network ating system personalities such as Unix, MS-DOS, Win- server selects a thread which has the same priority as dows, VMS, and OS/2 have been implemented on micro- the priority of the request. If the thread is not idle, the kernels. request is blocked until the thread will become idle. Real-Time Mach is an extension of Mach for support- NPS has two sets of workers: one is for processing ing real-time applications. One of its goals is to extend requests from applications and the other is for process- the system functions for real-time applications without ing packets from the network. Outgoing packets are pro- changing the original interface of Mach. For re ecting cessed by output worker threads, and the thread inherits the resource requirements of applications, we can spec- the priority of an application. Incoming packets are pro- ify attributes for objects provided by the kernel such as cessed by input workers. The priorities of input workers threads, processor sets, mutex variables, and IPC ports. are assigned according to the priority in headers of pack- A thread attribute allows us to specify a timing attribute ets. Also, there is one manager thread which receives including the period and the deadline of a thread, and Ethernet packets from a network. Since Ethernet pack- a processor set attribute is used to change a processor ets do not support the notion of priorities, the thread is scheduling policy[17]. Also, a mutex attribute allows us executed at the highest priority in the system. to change a synchronization protocol[10], and a port at- In Real-Time Mach, we can install multiple net- tribute is used for changing a queueing policy in IPC[9]. work systems for supporting a suitable network system In Real-Time Mach, users can specify timing con- for each application. For example, applications using straints explicitly. The clock allows the kernel to export NPS can co-exist with Unix network applications using high resolution timing hardware to them, which are im- TCP/IP on Real-Time Mach. portant for the measurement of performance, the calcu- 2.3 CRAS: Constant Rate Access Server lation of CPU usage, and ne-grain timestamping. There is a timer that provides synchronization action whenever CRAS is a storage server which can retrieve video and it expires. audio data at a constant rate. Continuous media data Resource management modules in Real-Time Mach are usually accessed sequentially and the data which are such as processor scheduling, synchronization, and IPC used in future can predict easily. In traditional storage support multiple policies for satisfying various require- systems, the read-ahead policy is used for improving se- ments of applications. These policies are specied by quential access performance, but it does not ensure all processor set attributes, mutex attributes, and port at- data are gotten in memory when the data are neces- tributes which are set by users. These implementa- sary. CRAS schedules pre-fetches of all data which are tions adopt the policy/mechanism separation principle required to access in future for ensuring all required data for supporting distinct multiple policies. can be fetched before the data are requested. The management of priority inversions in IPC CRAS periodically schedules pre-fetches of media and synchronization is integrated by using a com- data which are necessary at the next period, and ensures mon mechanism called the integrated real-time resource the data are fetched in memory until the end of the cur- manager[10]. The basic idea of the real-time resource rent period. CRAS calls the period interval time. The manager is to adopt the same abstraction and the interval time is dened by the tradeo between the max- same functions in both real-time IPC and real-time syn- imum number of streams supported by CRAS and the chronization, and to implement the priority inheritance minimum delay in streams. CRAS calculates data sizes protocol[15] within the abstraction. The integration which are required to read media data within the interval time for respective streams. For increasing throughput, to the audio server to playback them. Also, it decom- the maximum read size from a disk when requesting to presses video frames, and sends them to the X11 server, the kernel in CRAS is 256k bytes if the data are stored after it reassembles audio and video packets from NPS continuously, and CRAS sorts all the read requests to into frames. NPS is used for transmitting timing critical make the seek time and rotational latency required to audio and video data. Each packet includes a times- access blocks minimum. If the size of contiguous blocks tamp which indicates the playback time of audio and is less than 256k bytes, CRAS reads many small blocks video. Qtclient suspends to playback reassembled media instead of reading a big block, which decreases the total data until its timestamp equals to the current time. The bandwidth. mechanism enables both streams to be synchronized and CRAS schedules pre-fetches of media data using the respective streams to be smoothed. rate of media data not the rate specied by users since QtPlay provides a traditional VCR capability for con- it allows users to change the rate of reading media data troling movies. The user interface has buttons for play- easily. We changed two parts of Real-Time Mach for ing, stopping, rewinding, and fast forwarding movies. supporting CRAS. The rst part is a queue for waiting Also, it allows a user to change a frame rate dynamically, for accessing disk in device drivers. We use two queues as and to seek a movie while displaying digested frames us- a waiting queue, one for normal activities, and another is ing a slide bar. for real-time activities. The second modication is added a new interface for reading a large block from kernel. The current version of CRAS adopts the QuickTime Disk movie format to store audio and video data. The control information such as a rate of media and the duration of CRAS qtserver NPS each block are stored in another le. These information is used to pre-fetch media data and discard obsolete me- dia data in CRAS. The pre-fetching rate is determined by the rate of media data specied in the control le. Network The timestamp of each block is calculated from the sum of durations assigned to all previous media blocks. The CRT timestamp is used for reading data by clients and dis- X server carding obsolete data. Current CRAS adopts the Unix le format to store les. Thus, Unix applications can access the same le NPS qtclient accessed by applications using CRAS. The advantage of SP the approach is that if processing continuous media is Audio server not time critical, the Unix le system can be used for storing or editing les instead of using CRAS. 3 Overview of QtPlay Figure 1: Distributed Movie Player, QtPlay QtPlay is a distributed video player for showing the eectiveness of Real-Time Mach for building continuous A dynamic QOS control is one of the most important media applications. The application consists of two pro- issues in continuous media applications. The QOS lev- grams, qtclient and qtserver, as shown in Figure 1. Qt- els of the applications should be maintained even if the client will request qtserver to send audio and video data. continuous media applications are running concurrently. Qtserver retrieves audio samples and video frames from Also, the QOS levels should be changed according to the movie les on disks using CRAS. Thus, several qtclients number of applications and their respective importances can retrieve dierent movie streams concurrently. when system is overloaded. If there are several active Interleaved audio and video streams such as MPEG streams, importances are used which stream should be system streams are not exible since the streams must degraded rst. In a word, the stream with the least im- be delivered from the same machine. Also, the quality portance value, whose current QOS value is more than of these streams cannot be scaled down independently. its minimum QOS value, is degraded when a new ses- In QtPlay, video and audio streams are separated since sion is created or system becomes overloaded. QtPlay QtPlay needs to scale down the quality of media inde- provides a dynamic QOS control scheme for accepting pendently using media scaling techniques[2] according to more streams, since an amount of resources used by cur- CPU and network load. rent applications may be decreased for accepting a new Qtclient can specify a maximum and a minimum application[11]. In the current implementation, QtPlay frame rate, the importance of a stream, and the name changes the QOS level of a movie dynamically by detect- of a movie le. QtPlay may playback a movie at a dif- ing the violation of timing constraints of movies instead ferent rate from the rate stored in a le. The appli- of using missed deadline notication described in [11]. cation adopts the QuickTime movie format for storing Each stream of QtPlay decreases its frame rate when video and audio in a disk. Qtclient sends audio samples obsolete video frames and audio samples are detected. Also, it increases the frame rate when obsolete frames 4. Sends the current frame to qtclient through NPS. and samples are not detected for a constant time. A oset in the step 3 is used in the jitter control 4 Structure of QtPlay scheme described in Section 5. Qtclient has eights threads, two control threads, three QtPlay assumes to use the FP+RR scheduling policy threads for processing audio, and three threads for video. provided by Real-Time Mach. In the scheduling pol- The rst control thread is a system control thread, which icy, a priority is represented an integer ranged from 0 to sends commands for changing frame rates and adjust- 31, where 0 is the highest priority. A thread assigned a ing clocks between two machines. The second control priority higher than 12 is called a real-time thread. Real- thread, a user interface thread, processes events from time threads cannot be blocked unless they block them- users through QtPlay GUI, and controls starting and selves by some events such as I/O events, or preempted stopping streams. The threads for processing audio and by higher priority real-time threads. Other threads are video have the same structure, and created as real-time called non real-time threads. When there is no runnable threads. Here, we focus on threads for video again. The real-time thread, the scheduling behavior is equivalent video receive thread is an aperiodic thread which has a to the timesharing scheduling policy. Similar scheduling loop. Each loop is started when a new packet is received policies are adopted in many commercial real-time oper- from NPS. When packets can be assembled into a frame, ating systems. The scheduling policy can satisfy timing it is put in a queue between the video receive thread constraints of real-time threads. Also, response time of and the video decoder thread, and resumes the video non real-time threads is much improved than under the decoder thread. The video decoder thread decompresses traditional xed priority scheduling. the frame and put it into a next queue between the video decoder thread and the video display thread. The video decoder thread does not resume the video display thread created per movie non real time thread immediately after a frame is put in the queue. The video System Control real time thread display thread is resumed when the current time will be greater than the playback time specied by a timestamp Movie Control System Control User Interface in each frame. X server Since the loss of audio data is more severe than the CRAS Video Fetch Video Receive Video Decode Video Display loss of video data, the priorities of threads for processing audio are higher than the priorities of any threads for Audio Fetch Audio Receive Audio Decode Audio Play Audio Server processing video in qtclient, Audio data are smaller than video, then, the eect to disturb a video stream by an qtserver qtclient audio stream is not a serious problem. In each stream, a decoder thread has a lower priority, and a display thread has a higher priority. For presenting media smoothly, Figure 2: Structure of QtPlay the assignment strategy of priorities is required, since the possibility that many frames may become obsolete Each stream in QtPlay has several threads for pro- is increased if frame decompression time is longer than cessing audio and video data(Figure 2). When a new qt- expected time. The queues between the decoder threads client is started, three threads are created in qtserver and and the display threads usually includes many frames, eight threads are created in qtclient. The three threads since the dierence between the current time and the in qtserver are movie control, video fetch and audio fetch playback time is usually large when a frame put in the thread. The movie control thread waits for a command queues. If the decode threads and the display threads from qtclient for changing frame rates of streams, and have the same priority, many frames may be obsolete if synchronizing audio and video streams. The video fetch the decoder threads are not preempted by the display and audio fetch thread have the same structure, and are threads. However, if the priority of the display thread created as real-time threads. Here, we focus on the only is higher, processing frames in the queues by the display video fetch thread. CRAS ensures that video frames are threads can preempt the decoder thread even when the available by qtserver before the timestamps of respec- decoder threads need to take a long time for decompress- tive frames. Qtserver uses an aperiodic thread for the ing a frame. The priority assignment strategy can also video fetch thread. The thread has a loop processing the decrease the number of packet losses in NPS's receive following steps. buer and the number of obsolete frames. 1. Suspends itself until the current time will be greater than the timestamp of the next frame. 5 Media Control Techniques of Contin- uous Media in QtPlay 2. Skips some frames using DDA algorithm, if the frame rate of the video stream is degraded. Continuous media applications need media control techniques such as jitter control schemes for smoothing 3. Calculates playback time by adding a constant oset their continuous media streams. In this section, we de- to the timestamp of the current frame. scribe these media control techniques used in QtPlay. 5.1 Jitter Control is faster than a machine executing qtclient, a media frame arrives to the machine executing qtclient earlier than the Continuous media applications should playback video time which qtclient expects. In this case, data buers in frames and audio samples timely for guaranteeing the qtclient are over owed and the frames are dropped. On quality of media. QtPlay assigns a timestamp for each the other hand, if the clock speed of qtserver is slower media frame using high resolution timers provided by than the clock of qtclient, media frames arrive later to Real-Time Mach for suppressing timing jitters caused by qtclient than the playback times of media frames, and the varieties of network delay and decompression time. the frames will be discarded. For solving the problem, a Qtserver stores the current time in CT, and calcu- mechanism to compensate a clock skew or synchronizing lates the time used for fetching a media frame by CT - clocks of qtclient and qtserver is necessary. QtPlay uses a start time, where the start time indicates the time when clock skew compensation for dealing with the dierence a stream is started. When sending a frame, a times- between machines because it is dicult to synchronize tamp is assigned for each video frame and audio sample. high resolution clocks of several machines running Real- The timestamp of each media frame indicates the time Time Mach without disturbing other real-time applica- when playbacking itself. The timestamp is calculated by tions. CT + oset. The oset is a constant value dened in qt- When qtclient detects that the dierence between the server, which is bigger than the maximum network delay actual arrival time of a media frame and an expected + the maximum decompression time. Video/audio dis- time calculated from the timestamp of the media frame play threads in qtclient wait until the current time equals is greater than a pre-dened threshold value, it sends to the time specied in the timestamp of a media frame. a timing adjust message to qtserver. Qtserver which re- The mechanism is also used to synchronize two streams ceives this message puts back or forward the time to send in QtPlay if the oset values for both streams use the a media frame to qtclient. same value. An advantage of our clock skew compensation scheme 5.2 Real-Time FIFO using a feedback message is that it is easy to implement because the scheme does not require to change the imple- The jitter control scheme in QtPlay uses a data struc- mentation of real-time clocks in Real-Time Mach. How- ture, called Real-Time FIFO. The data structure inte- ever, a clock synchronization mechanism is more general grates buering and the jitter suppression mechanism than a clock skew compensation. into one library for constructing QtPlay clearly. Real-Time FIFO consists of four parts. The rst part is data FIFO which buers media frames. The second part is data handler which inserts and removes the media frames into/from the data FIFO. The third part is drop Timing Controller handler which is called when an obsolete media frame whose timestamp is greater than the current time is dis- From Prev. Stage Data To Next Stage carded. The last part is timing controller. When the Handler timestamp of a media frame in Real-Time FIFO equals to Data FIFO the current time, the timing controller invokes the data Drop handler to remove and process media data. Before the Handler data handler is invoked, the timestamp of a media frame in Real-Time FIFO is checked. If the time is greater than the deadline time1, Real-Time FIFO discards the frame Discard automatically, and invokes a drop handler to notify an application that the frame is dropped. QtPlay uses this drop notication for changing a QOS value of a video Figure 3: Internal structure of Real-Time FIFO stream by degrading its frame rate. 5.3 Clock Skew Compensation The jitter control scheme described in Section 5.2 can 6 Evaluation synchronize multiple continuous media streams when all In this section, we show the results which demonstrate clocks of machines are synchronized. However, it is im- the eectiveness of our approach. In the evaluations, we portant to take clock skews into consideration for making used Gateway2000 P5-100 IBM AT compatible comput- it easy to use algorithms for smoothing media streams. ers which have 100 MHz Intel Pentium processors, 32 An important clock skew in QtPlay is the dierence of MB of memory, 2 GB of SCSI disk, and a 10 Mbps Eth- clock speeds between machines executing qtclient and qt- ernet interface. In all evaluations, a movie which is three server. If the clock speed of a machine executing qtserver minute length was used. During the evaluation, we also 1 A thread execute another non real-time application, xgas simulta- which calls the data handler becomes runnable when the timestamp of a media frame equals to the current time. How- neously with qtclient. ever, the thread may not run immediately. The data handler 6.1 Clock Skew Compensation should be called before the deadline time.  sec x 10-3 30.00 No Feedback sec x 10-3 With Feedback No Jitter ctrl. 10.00 25.00 With Jitter ctrl. 8.00 20.00 6.00 4.00 15.00 2.00 10.00 0.00 5.00 -2.00 -4.00 0.00 -6.00 -5.00 -8.00 sec 0.00 50.00 100.00 150.00 -10.00 sec 0.00 50.00 100.00 150.00 Figure 4: Clock Skew Compensation Figure 5: Jitter control The rst evaluation demonstrates the eect of our clock skew compensation scheme using feedback mes- sages. Figure 4 shows the dierence between the arrival sec x 10-3 time of each video frame to qtclient and its timestamp 10.00 Playback time under the FP+RR scheduling policy. Without the clock 5.00 skew compensation scheme, the arrival time of video frames is gradually delayed. This is because that there 0.00 is 0.01 % of clock skew between the server and the client -5.00 machine which we use. If a movie length is longer than -10.00 60 minutes, the arrival time of the last frame will be -15.00 delayed about 0.4 secs. Also, buers of qtclient will be -20.00 over owed. On the other hand, our clock skew compen- sation scheme can compensate the clock skew, and avoid -25.00 to over ow the buers. -30.00 0.00 50.00 100.00 150.00 sec 6.2 Jitter control In the second evaluation, we show the eect of the Figure 6: Playback time under Timesharing policy jitter control scheme using in QtPlay. Figure 5 shows the dierence between the actual playback time of each video frame and its timestamp. In the rst test, we show the result without the jitter control scheme. In this case, FP+RR scheduling policy. Many video frames are lost qtclient playbacks a frame as soon as it is received. In under the timesharing scheduling policy, but no frame is the second test, we show the result with the jitter control lost under the FP+RR scheduling policy. scheme. The tests are executed under the FP+RR policy The result indicates the importance of real-time with the clock compensation scheme. scheduling to smooth media streams such as jitter con- The playback time of video frames is varied over 10 trol schemes. ms if the jitter control is not used, but no timing jitter over than 1 ms cannot be found with the jitter control 7 Discussion as shown in Figure 5. In this section, we discuss three experiments with 6.3 Eectiveness of Real-Time Scheduling building QtPlay on Real-Time Mach. In the rst ex- The last evaluation shows the importance of real-time perience, we describe why real-time technologies are im- scheduling to smooth continuous media streams. As de- portant for media control techniques. The second and scribed in Section 6.1 and 6.2, the FP+RR scheduling the third experience are related to the primitives pro- policy is used for satisfying timing constraint of media vided by Real-Time Mach. Both experiments describe frames. Figure 6 shows the playback time of video frames the weakness of the current thread model of Real-Time under the timesharing scheduling policy. Although the Mach. the jitter control scheme is used in this evaluation, the 7.1 Media Control of Continuous Media playback time of video frames shown in Figure 6 are not stabilized. Real-time technologies ensure to playback each con- Figure 7 shows the number of lost(deadline missed) tinuous media data before its deadline. However, the video frames per a second under the timesharing and the technologies do not ensure that its jitter is small since  7.2 Periodic Thread vs. Aperiodic Thread frame/sec Real-Time Mach provides the periodic thread abstrac- 20.00 Time Sharing tion for implementing periodic activities easily. The pe- riodic thread abstraction is useful to make a program us- FP + RR 18.00 16.00 ing periodic activities, and a mechanism to notify missed 14.00 deadlines makes a recover from timing error easy. How- 12.00 ever, if media streams are scaled down, their frame rates 10.00 are not constant rate data stream as described in Section 8.00 4. 6.00 Reducing frame rate is achieved by skipping several 4.00 frames from an original video stream. The rst imple- 2.00 mentation of QtPlay changes the duration of media when 0.00 sec the frame rate is changed. The approach enables an ap- 0.00 50.00 100.00 150.00 plication to use a periodic thread since a media stream is alway transformed to a periodic stream even when sev- eral frames are skipped. However, the scheme is dicult Figure 7: Lost Vide Frames to process variable rate media streams. Examples of such variable bit rate media is a captured video stream, and stored streams using a non real-time storage system. The current implementation uses aperiodic threads the data may be arrived at any time before its deadline. which have loops. Each loop includes the following steps: Jitter control schemes make continuous media streams (1) waits until the current time to be greater than the smooth by keeping the data waiting until the current time specied in a timestamp. (2) sends a frame to next time will equal to the playback time specied in the stage. The approach is less structured, but can process data. The waiting time is controlled by the oset value both periodic streams and aperiodic streams. which is added in qtserver. If the oset value is bounded, It is the reason to require such implementation that the smoothness of a continuous media stream may be the current thread model is not suitable for continuous broken. When several applications are executed concur- media applications. The experiment shows that a new rently on traditional timesharing operating systems, the abstraction which can deal with quasi-periodic activities oset value cannot be determined in advance since an should be developed. amount of jitter cannot be bounded. In a conference 7.3 Scheduling and Priority Assignment system, an oset can be changed during intervals of si- lence in the audio stream[6] but the technique cannot be QtPlay uses the FP+RR scheduling, and assigns pri- used in video player applications. orities according to the importance of applications not Real-time technologies enable to make jitters small timing parameters such as periods and deadlines. The and bounded so that it is easy to determine a suitable approach loses the elegant results achieved by real-time oset value for removing jitters. The approach is also scheduling theory. The reason is that the loss of audio data is more severe suitable for interactive applications which require shorter than the loss of video data, then the priority of threads waiting time before playbacking media data. for processing audio should be higher than the priority Real-time technologies are also useful for making the of video. It is dicult to represent such constraint in behavior of continuous media streams stable. Under tra- traditional real-time scheduling algorithms. Also, tradi- ditional timesharing operating systems, it is dicult to tional real-time scheduling algorithms assume that activ- distinguish large jitters and the violation of inter-stream ities are executed periodically. As described in Section synchronization. Thus, they cause to send many incor- 7.2, continuous media streams are not represented by rect feedback messages to servers for recovering the vi- periodic activities. olation of synchronization. The situation degrades the However, we found that mechanisms provided for smoothness of a continuous media stream. avoiding the priority inversion problem such as real-time IPC and real-time synchronization are very useful to re- Our dynamic QOS control scheme relies on a feedback duce the blocking time of low priority threads, and makes control to be adaptable according to the current network the quality of media better under the FP+RR schedul- and CPU load. Real-time technologies are also useful ing. for making the behavior of the dynamic QOS control The priority assignment strategy for each stream in stable. If a dynamic QOS control occurs frequently, too qtclient is very ad-hoc as shown described in Section 4. much change of the quality of media causes signicant Especially, we assign dierent priorities for threads con- degradation of the visual quality. sisting of a stream. The reason to require the ad-hoc Real-time technologies can reduce the uctuation of scheme is that the current thread model lacks the mech- jitters and allocated required CPU capacity for an appli- anisms to stop the execution of a thread when its execu- cation. Thus, it ensures that the quality of media is not tion time exceeds over the worst execution time. Also a degraded unpredictably. thread should be stopped when its deadline is missed.  The experiment shows that a more sophisticated [3] C.L.Compton, and D.L.Tennenhouse, \Collaborative Load Shed- scheduling algorithm incorporating the notion of impor- ding for Media-Based Application", In Proceedings of Interna- tional Conference on Multimedia Computing and Systems, 1994. tance makes us to use systematic priority assignment. Also, more sophisticated mechanisms for handling ex- [4] D.Golub, R.Dean, A.Forin, and R.Rashid, "Unix as an application program", In proceedings of the 1990 Summer Usenix Conference, ceptions such as overruns and deadline misses can avoid 1990. the ad-hoc priority assignment. [5] J.Helander, \Unix under Mach: The Lites Server", Helsinki Uni- versity of Technology, Master's Thesis, 1994. 8 Conclusion [6] V.Jacobson, \Multimedia Conferencing on the Internet", SIG- COMM'94 Tutorial, 1994. In this paper, we described experiences with building [7] K.Jeay, and D.Bennett, \A Rate-Based Execution Abstraction a distributed video player, QtPlay. QtPlay is not a toy for Multimedia Computing", In Proceesings of 5th International application. It supports multiple clients, and VCR ca- Workshop on Network and Operating System Support for Digital Audio and Video, 1995. pabilities for respective clients. QtPlay ensures timing constraints of movies by using the facilities provided by [8] D.I.Katcher, K.A. Kettler, and J.K.Strosnider, \Real-Time Op- erating Systems for Multimedia Processing", In Proceedings 5th Real-Time Mach. Workshop on Hot Topics in Operating Systems. 1995. Real-time resource management techniques provided by Real-Time Mach are useful for stabilizing media con- [9] T. Kitayama, T. Nakajima and H. Tokuda, "RT-IPC: An IPC Extension for Real-Time Mach", In Proceedings of the 2nd Micro- trol techniques used in QtPlay. In this paper, we showed kernels and Other kernel Architectures, USENIX, 1993 that real-time technologies are eective for media con- [10] T.Nakajima, T.Kitayama, H.Arakawa,and H.Tokuda, "Integrated trol techniques such as jitter control and inter-stream Management of Priority Inversion in Real-Time Mach", In Pro- synchronization schemes. ceedings of the Real-Time System Symposium, 1993 The experiences with implementing Qtplay shows that [11] T.Nakajima, \NPS:User-Level Real-Time Network Engine on periodic threads and the rate-monotonic scheduling pro- Real-Time Mach\, In Proceedings of First International Workshop on Real-Time Computing System and Applications, 1994. vided by Real-Time Mach were not adopted in QtPlay since they are too naive for implementing actual con- [12] T.Nakajima and H.Tokuda, \Design and Implementation of a User-Level Real-Time Network Engine", IS-RR-94-14S, Research tinuous media applications. Also, we need careful im- Report, Japan Advanced Institute of Science and Technology, plementation techniques such as the thread structure of 1994. applications and the assignment of priorities for these [13] T.Nakajima and H.Tezuka, \A Continuous Media Application threads for satisfying timing constraints. The problem supporting Dynamic QOS Control on Real-Time Mach", In Pro- occurs due to the weakness of the current thread model. ceedings of the ACM Multimedia '94, 1994. We are working on a new thread model using rate-based [14] L.A.Rowe, and B.C.Smith, \A Continuous Media Player, In Pro- concepts which are similar to the proposal described in ceesings of 3th International Workshop on Network and Operating System Support for Digital Audio and Video, 1992. [7]. The experiments show that programming continuous [15] L.Sha, R.Rajkumar, and J.P.Lehoczky, \Priority Inheritance Pro- tocols: An Approach to Real-Time Synchronization", IEEE Trans- media applications is not easy even on real-time oper- actions on Computers, Vol.39, No.9, 1990. ating systems. We think that a sophisticated toolkit [16] J.A.Stankovic, \Continuous and Multimedia OS Support in Real- for building continuous media applications makes to im- Time Control Applications", In Proceedings 5th Workshop on Hot plement them easy. Currently, we are also working on Topics in Operating Systems. 1995. building a toolkit for continuous media applications. The [17] H.Tokuda, T.Nakajima and P.Rao. \Real-Time Mach: Towards toolkit enables programmers to create continuous media a Predictable Real-Time System", In Proceedings of the USENIX applications by connecting several modules. The toolkit 1st Mach Symposium, 1990. will avoid to consider the assignment of priorities and [18] H.Tezuka, T.Nakajima, \Design and Implementation of a Contin- thread structures of applications. uous Media Storage System on Real-Time Mach", IS-RR-94-15S, Research Report, Japan Advanced Institute of Science and Tech- nology, 1994. Acknowledgements We would like to thank the members of the MMMC Project in JAIST for their valuable comments and inputs to the development of Real-Time Mach and integrating Lites with Real-Time Mach. We are also grateful to Prof. Hideyuki Tokuda for very helpful discussions. References [1] S.Cen, C.Pu, R.Staehli, C.Cowan, and J.Walpole, \A Distributed Real-Time MPEG Video Audio Player", In Proceedings of 5th In- ternational Workshop on Network and Operating System Support for Digital Audio and Video, 1995. [2] L.Delgrossi, C.Halstrick, D.Hehmann, R.Herrtwich, O.Krone, J.Sandvoss, C.Vogt, \Media Scaling for Audiovisual Communica- tion with the Heidelberg Transport System", In Proceedings of the ACM Multimedia'93, 1993.