soft real-time systems - nus computingcs3249/lecture/soft rts.pdf · cs3249 soft real-time systems...
Post on 19-Oct-2020
42 Views
Preview:
TRANSCRIPT
-
CS3249 Soft Real-Time Systems 1CS3249
Soft Real-Time Systems 11CS3249 Soft Real-Time Systems
Soft Real-Time Systems
Leow Wee KhengCS3249 User Interface Development
-
CS3249 Soft Real-Time Systems 2CS3249 2Soft Real-Time Systems
Real-Time Systems
Real-time systems must react to inputs “quickly”.
-
CS3249 Soft Real-Time Systems 3CS3249 Soft Real-Time Systems 3CS3249Soft Real-Time Systems 3Soft Real-Time Systems 3CS3249
aircraft autopilot
-
CS3249 Soft Real-Time Systems 4CS3249 Soft Real-Time Systems 4CS3249Soft Real-Time Systems 4Soft Real-Time Systems 4CS3249
GPS navigation system
-
CS3249 Soft Real-Time Systems 5CS3249 Soft Real-Time Systems 5CS3249Soft Real-Time Systems 5Soft Real-Time Systems 5CS3249
car engine control unit
-
CS3249 Soft Real-Time Systems 6CS3249 Soft Real-Time Systems 6CS3249Soft Real-Time Systems 6Soft Real-Time Systems 6CS3249
If your car breaks take 10 sec to respond...
-
CS3249 Soft Real-Time Systems 7CS3249 Soft Real-Time Systems 7CS3249Soft Real-Time Systems 7Soft Real-Time Systems 7CS3249
Real-time systems are
closer than you think!
-
CS3249 Soft Real-Time Systems 8CS3249 Soft Real-Time Systems 8CS3249Soft Real-Time Systems 8Soft Real-Time Systems 8CS3249
video player
-
CS3249 Soft Real-Time Systems 9CS3249 Soft Real-Time Systems 9CS3249Soft Real-Time Systems 9Soft Real-Time Systems 9CS3249
video surveillance
-
CS3249 Soft Real-Time Systems 10CS3249 Soft Real-Time Systems 10CS3249Soft Real-Time Systems 10Soft Real-Time Systems 10CS3249
virtual reality
-
CS3249 Soft Real-Time Systems 11CS3249 Soft Real-Time Systems 11CS3249Soft Real-Time Systems 11Soft Real-Time Systems 11CS3249
interactive system with kinect
-
CS3249 Soft Real-Time Systems 12CS3249 Soft Real-Time Systems 12CS3249Soft Real-Time Systems 12Soft Real-Time Systems 12CS3249
my fancy computer system
-
CS3249 Soft Real-Time Systems 13CS3249 13Soft Real-Time Systems
Types of Real-Time Systems
Hard real-time systems Timing is critical. Failure to meet deadline can result in disaster. Take CG2271 Real-Time Operating Systems.
Soft real-time systems Timing is less critical. Failure to meet deadline results in degraded performance.
-
CS3249 Soft Real-Time Systems 14CS3249 14Soft Real-Time Systems
Activation Modes
Time-triggered Task is activated by timer. Periodic: If activation is at fixed interval. Aperiodic: If activation is not at fixed interval.
Event-triggered Task is activated by event, e.g., mouse click. Usually aperiodic.
-
CS3249 Soft Real-Time Systems 15CS3249 15Soft Real-Time Systems
Processor Load
For simplicity, let's consider periodic tasks e.g., displaying video frames at regular interval.
Ti : period or interval task i
Ci : computation time of task i
Processor utilisation
Total processor utilisation
U=∑i=1
n
U i
i
ii TCU
-
CS3249 Soft Real-Time Systems 16CS3249 Soft Real-Time Systems 16CS3249Soft Real-Time Systems 16Soft Real-Time Systems 16CS3249
Processor Load
Overload Not enough time to service all demands. Total processor utilisation U > 1.
Overrun A task misses its deadline. May or may not cause overload.
Remedy Detect overload or overrun and manage the problem.
-
CS3249 Soft Real-Time Systems 17CS3249 Soft Real-Time Systems 17CS3249Soft Real-Time Systems 17Soft Real-Time Systems 17CS3249
Decoding time of a sequence from Star Wars.
Worst case is rare butmuch longer than average.
-
CS3249 Soft Real-Time Systems 18CS3249 Soft Real-Time Systems 18CS3249Soft Real-Time Systems 18Soft Real-Time Systems 18CS3249
load
average
time
No overload butinefficient use of CPU resources
-
CS3249 Soft Real-Time Systems 19CS3249 Soft Real-Time Systems 19CS3249Soft Real-Time Systems 19Soft Real-Time Systems 19CS3249
load
average
time
Efficient use of CPU resourceswith transient overload
-
CS3249 Soft Real-Time Systems 20CS3249 20Soft Real-Time Systems
Methods for Handling Overload
Adaptive quality of service Change to low-quality service when load is high.
Multithreading Run tasks in multiple threads.
Task skipping Skip (less important) tasks.
Others
-
CS3249 Soft Real-Time Systems 21CS3249 21Soft Real-Time Systems
Adaptive Quality of Service
Change quality of service when needed. Provide high-quality service when CPU load is low. Switch to low-quality service when CPU load is high.
during rotation (high CPU load) after rotation (low CPU load)
-
CS3249 Soft Real-Time Systems 22CS3249 22Soft Real-Time Systems
Case Study: Video Capture and Processing
image widget for image widget forcaptured video frame processed frame
consoleExample 1: Single Thread
-
CS3249 Soft Real-Time Systems 23CS3249 Soft Real-Time Systems 23CS3249Soft Real-Time Systems 23Soft Real-Time Systems 23CS3249
Console
inputimage widget
outputimage widget
Webcam ImageProcessor
OpenCVvideo capture
library
OpenCVimage processing
library
setimage
framecaptured
frameprocessed
queryframe
processimage
processframetimer
event
frame ratecomputed
frame ratecomputed
-
CS3249 Soft Real-Time Systems 24CS3249 Soft Real-Time Systems 24CS3249Soft Real-Time Systems 24Soft Real-Time Systems 24CS3249
// Webcam.h
class Webcam: public QObject{ Q_OBJECT
public: Webcam(int width, int height, ImageProc *p); ~Webcam(); bool setDevice(int device); void setFrameRate(int fps); void start(); void stop();
signals: void frameCaptured(QImage frame); protected: void timerEvent(QTimerEvent *event);
-
CS3249 Soft Real-Time Systems 25CS3249 Soft Real-Time Systems 25CS3249Soft Real-Time Systems 25Soft Real-Time Systems 25CS3249
private: ImageProc *imageProc; int width, height, frameRate; CvCapture *capture; IplImage *frame; int timerId;};
-
CS3249 Soft Real-Time Systems 26CS3249 Soft Real-Time Systems 26CS3249Soft Real-Time Systems 26Soft Real-Time Systems 26CS3249
// Webcam.cpp
bool Webcam::setDevice(int dev){ if (capture) { cvReleaseCapture(&capture); // reset capture = NULL; } capture = cvCaptureFromCAM(dev); if (!capture) return false; cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH, width); cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, height); return true;}
-
CS3249 Soft Real-Time Systems 27CS3249 Soft Real-Time Systems 27CS3249Soft Real-Time Systems 27Soft Real-Time Systems 27CS3249
void Webcam::setFrameRate(int fps){ frameRate = fps;}
void Webcam::start(){ timerId = startTimer(1000 / frameRate);}
void Webcam::stop(){ killTimer(timerId); timerId = 0;}
-
CS3249 Soft Real-Time Systems 28CS3249 Soft Real-Time Systems 28CS3249Soft Real-Time Systems 28Soft Real-Time Systems 28CS3249
void Webcam::timerEvent(QTimerEvent *event){ if (event->timerId() == timerId) { frame = cvQueryFrame(capture); QImage image = IplImage2QImage(frame); emit frameCaptured(image);
// Direct function call to imageProc. imageProc->processFrame(frame); } else QObject::timerEvent(event);}
-
CS3249 Soft Real-Time Systems 29CS3249 Soft Real-Time Systems 29CS3249Soft Real-Time Systems 29Soft Real-Time Systems 29CS3249
// ImageProc.h
class ImageProc: public QObject{ Q_OBJECT public: void setProcessType(int type); void processFrame(IplImage *frame); signals: void frameProcessed(QImage image); private: int procType; IplImage *process(IplImage *frame);};
-
CS3249 Soft Real-Time Systems 30CS3249 Soft Real-Time Systems 30CS3249Soft Real-Time Systems 30Soft Real-Time Systems 30CS3249
// ImageProc.cpp
void ImageProc::setProcessType(int type){ if (type >= 1 && type
-
CS3249 Soft Real-Time Systems 31CS3249 Soft Real-Time Systems 31CS3249Soft Real-Time Systems 31Soft Real-Time Systems 31CS3249
IplImage *ImageProc::process(IplImage *source){ IplImage *target = cvCreateImage( cvSize(source->width, source->height), source->depth, source->nChannels); switch(procType) { case 1: // do nothing return cvCloneImage(source); case 2: // demo fast algo: fast smoothing cvSmooth(source, target, CV_GAUSSIAN, 5); return target;
case 3: // demo slow algo: filter with large kernel ... return target; }}
-
CS3249 Soft Real-Time Systems 32CS3249 Soft Real-Time Systems 32CS3249Soft Real-Time Systems 32Soft Real-Time Systems 32CS3249
// Console.cpp
Console::Console(Webcam *w, ImageProc *p){ webcam = w; connect(webcam, SIGNAL(frameCaptured(QImage)), this, SLOT(getCameraImage(QImage))); imageProc = p; connect(imageProc, SIGNAL(frameProcessed(QImage)), this, SLOT(getProcessedImage(QImage))); inputPanel = new ImageWidget(WIDTH, HEIGHT); connect(inputPanel, SIGNAL(frameRateComputed(float)), this, SLOT(showInputFps(float))); outputPanel = new ImageWidget(WIDTH, HEIGHT); connect(outputPanel, SIGNAL(frameRateComputed(float)), this, SLOT(showOutputFps(float))); ...}
-
CS3249 Soft Real-Time Systems 33CS3249 Soft Real-Time Systems 33CS3249Soft Real-Time Systems 33Soft Real-Time Systems 33CS3249
Test Results
Process Type 1, 2: fast Input frame rate = output frame rate = 25 fps (desired rate). All CPUs run at about 25% load or less.
Type 1, 2 Type 3
-
CS3249 Soft Real-Time Systems 34CS3249 Soft Real-Time Systems 34CS3249Soft Real-Time Systems 34Soft Real-Time Systems 34CS3249
Test Results
Process Type 3: slow Input frame rate = output frame rate = 10 fps < desired rate. Video displays lack behind user actions. Sometimes, 1 CPU is fully loaded. Why input frame rate = output frame rate?
-
CS3249 Soft Real-Time Systems 35CS3249 35Soft Real-Time Systems
Case Study: Video Capture and Processing
Example 2: Multiple Threads
Console, image widgets Main thread: thread 1.
Webcam Secondary thread: thread 2.
Image processor Secondary thread: thread 3.
Buffer Shared buffer between webcam and image processor. Maintains mutex lock.
-
CS3249 Soft Real-Time Systems 36CS3249 Soft Real-Time Systems 36CS3249Soft Real-Time Systems 36Soft Real-Time Systems 36CS3249
Console
inputimage widget
outputimage widget
Webcam ImageProcessor
OpenCVvideo capture
library
OpenCVimage processing
library
setimage
framecaptured
frameprocessed
queryframe
processimage
timerevent
frame ratecomputed
frame ratecomputed
Bufferadd del loop ortimer
event
status
-
CS3249 Soft Real-Time Systems 37CS3249 Soft Real-Time Systems 37CS3249Soft Real-Time Systems 37Soft Real-Time Systems 37CS3249
// Webcam.h
class Webcam: public QThread{ Q_OBJECT public: Webcam(int width, int height, Buffer *b); ~Webcam(); bool setDevice(int device); void setFrameRate(int fps); void start(); void stop();
signals: void frameCaptured(QImage frame); protected: void timerEvent(QTimerEvent *event);
-
CS3249 Soft Real-Time Systems 38CS3249 Soft Real-Time Systems 38CS3249Soft Real-Time Systems 38Soft Real-Time Systems 38CS3249
private: Buffer *buffer; int width, height, frameRate; CvCapture *capture; IplImage *frame; volatile int timerId;};
-
CS3249 Soft Real-Time Systems 39CS3249 Soft Real-Time Systems 39CS3249Soft Real-Time Systems 39Soft Real-Time Systems 39CS3249
// Webcam.cpp
void Webcam::start(){ timerId = startTimer(1000 / frameRate); buffer->clear();}
void Webcam::stop(){ killTimer(timerId); timerId = 0;}
-
CS3249 Soft Real-Time Systems 40CS3249 Soft Real-Time Systems 40CS3249Soft Real-Time Systems 40Soft Real-Time Systems 40CS3249
void Webcam::timerEvent(QTimerEvent *event){ if (event->timerId() == timerId) { frame = cvQueryFrame(capture); QImage image = IplImage2QImage(frame); emit frameCaptured(image);
// Doesn't call imageProc directly. buffer->addToFront(frame); } else QObject::timerEvent(event);}
-
CS3249 Soft Real-Time Systems 41CS3249 Soft Real-Time Systems 41CS3249Soft Real-Time Systems 41Soft Real-Time Systems 41CS3249
Image Processing Types
Type Process Speed Mechanism
1 copy fast Explicit loop
2 copy fast Timer event
3 fastsmoothing fast Explicit loop
4 fast smoothing fast Timer event
5 slowfiltering slowExplicit loop
Don't skip frames
6 slowfiltering slowExplicit loopSkip frames
-
CS3249 Soft Real-Time Systems 42CS3249 Soft Real-Time Systems 42CS3249Soft Real-Time Systems 42Soft Real-Time Systems 42CS3249
// ImageProc.h
class ImageProc: public QThread{ Q_OBJECT public: ImageProc(Buffer *b); void setProcessType(int type); void start(); void stop(); protected: void run(); void timerEvent(QTimerEvent *event); signals: void frameProcessed(QImage image);
-
CS3249 Soft Real-Time Systems 43CS3249 Soft Real-Time Systems 43CS3249Soft Real-Time Systems 43Soft Real-Time Systems 43CS3249
private: Buffer *buffer; int frameRate; volatile int procType; volatile bool stopped; volatile int timerId; void processFrame(); IplImage *process(IplImage *frame);};
-
CS3249 Soft Real-Time Systems 44CS3249 Soft Real-Time Systems 44CS3249Soft Real-Time Systems 44Soft Real-Time Systems 44CS3249
// ImageProc.cpp
void ImageProc::start(){ if (procType == 2 || procType == 4) // Use timer event. timerId = startTimer(1000 / frameRate); else { stopped = false; // Don't use timer event. QThread::start(); // Will call run(). }}
void ImageProc::stop(){ stopped = true; killTimer(timerId); timerId = 0;}
-
CS3249 Soft Real-Time Systems 45CS3249 Soft Real-Time Systems 45CS3249Soft Real-Time Systems 45Soft Real-Time Systems 45CS3249
void ImageProc::run(){ while (!stopped) // Run explicit loop. if (!(buffer->isEmpty())) processFrame();}
void ImageProc::timerEvent(QTimerEvent *event){ if (event->timerId() == timerId) processFrame(); else QObject::timerEvent(event);}
-
CS3249 Soft Real-Time Systems 46CS3249 Soft Real-Time Systems 46CS3249Soft Real-Time Systems 46Soft Real-Time Systems 46CS3249
void ImageProc::processFrame(){ IplImage *frame = NULL; if (procType != 6) frame = buffer->delAtBack(); // Don't skip frames. else frame = buffer->skipToFront(); // Skip frames.
if (frame) { IplImage *pframe = process(frame); QImage image = IplImage2QImage(pframe); emit frameProcessed(image); cvReleaseImage(&frame); cvReleaseImage(&pframe); }}
-
CS3249 Soft Real-Time Systems 47CS3249 Soft Real-Time Systems 47CS3249Soft Real-Time Systems 47Soft Real-Time Systems 47CS3249
// Buffer.h
class Buffer: public QObject{ Q_OBJECT public: Buffer() {front = back = 0;} // Init to empty buffer. bool isEmpty(); void addToFront(IplImage *frame); IplImage *delAtBack(); IplImage *skipToFront(); void clear();
signals: void status(int value);
-
CS3249 Soft Real-Time Systems 48CS3249 Soft Real-Time Systems 48CS3249Soft Real-Time Systems 48Soft Real-Time Systems 48CS3249
protected: enum {size = 100}; IplImage *data[size]; int front; // Points to front of buffer. int back; // Points to back of buffer. QMutex mutex;};
-
CS3249 Soft Real-Time Systems 49CS3249 Soft Real-Time Systems 49CS3249Soft Real-Time Systems 49Soft Real-Time Systems 49CS3249
// Buffer.cpp
bool Buffer::isEmpty(){ return (back == front);}
-
CS3249 Soft Real-Time Systems 50CS3249 Soft Real-Time Systems 50CS3249Soft Real-Time Systems 50Soft Real-Time Systems 50CS3249
void Buffer::addToFront(IplImage *frame){ mutex.lock(); if (front != back - 1 and !(front == size - 1 and back == 0)) // Not full { ++front; // Increment front. if (front == size) front = 0; // Wrap around.
IplImage *clone = cvCloneImage(frame); data[front] = clone; emit status(0); // Buffer not full. } else emit status(1); // Buffer full. mutex.unlock();}
-
CS3249 Soft Real-Time Systems 51CS3249 Soft Real-Time Systems 51CS3249Soft Real-Time Systems 51Soft Real-Time Systems 51CS3249
IplImage *Buffer::delAtBack(){ IplImage *frame = NULL; mutex.lock(); if (back != front) // Not empty. { ++back; // Increment back. if (back == size) back = 0; // Wrap around.
frame = data[back]; // Get data. emit status(0); // Buffer not empty. } else emit status(-1); // Buffer empty mutex.unlock();
return frame;}
-
CS3249 Soft Real-Time Systems 52CS3249 Soft Real-Time Systems 52CS3249Soft Real-Time Systems 52Soft Real-Time Systems 52CS3249
Similarly for IplImage *Buffer::skipToFront()
void Buffer::clear()
-
CS3249 Soft Real-Time Systems 53CS3249 Soft Real-Time Systems 53CS3249Soft Real-Time Systems 53Soft Real-Time Systems 53CS3249
Test Results
For process type 1 – 4: fast Input frame rate = output frame rate = 25 fps (desired rate).
Type 1, 3 Type 2, 4
-
CS3249 Soft Real-Time Systems 54CS3249 Soft Real-Time Systems 54CS3249Soft Real-Time Systems 54Soft Real-Time Systems 54CS3249
Test Results
Process Type 1, 3: Most of the time, 1 CPU is at (almost) full load.
Process Type 2, 4: All CPUs are at 25% load or less.
What causes the difference?
-
CS3249 Soft Real-Time Systems 56CS3249 Soft Real-Time Systems 56CS3249Soft Real-Time Systems 56Soft Real-Time Systems 56CS3249
Test Results
Process Type 6: slow Output frame rate < 1/2 input frame rate = 25 fps. By skipping frames, image processor is in sync with webcam. Buffer is not full.
-
CS3249 Soft Real-Time Systems 57CS3249 57Soft Real-Time Systems
Multithreading vs. Efficiency
Multithreading lets multiple threads run concurrently.
But, multithreading ⇒ efficient processing. Two factors:
Processing time of tasks: long or short. Dependency between tasks.
Example: ImageProc depends on Webcam thru' Buffer.
-
CS3249 Soft Real-Time Systems 59CS3249 Soft Real-Time Systems 59CS3249Soft Real-Time Systems 59Soft Real-Time Systems 59CS3249
Multithreading can't speed up slow tasks.
-
CS3249 Soft Real-Time Systems 61CS3249 61Soft Real-Time Systems
Speeding Up Slow Tasks
How to speed up a slow task? Use efficient algorithm.
e.g., O(n log n) is more efficient than O(n2). Reduce overhead in program.
e.g., pass pointers instead of whole objects. Parallel processing.
-
CS3249 Soft Real-Time Systems 62CS3249 Soft Real-Time Systems 62CS3249Soft Real-Time Systems 62Soft Real-Time Systems 62CS3249
Parallel processing: Split task into multiple independent sub-tasks. Run sub-tasks concurrently, e.g., in multiple threads.
1 2
3 4
processingtime isreduced toabout 1/4
-
CS3249 Soft Real-Time Systems 63CS3249 Soft Real-Time Systems 63CS3249Soft Real-Time Systems 63Soft Real-Time Systems 63CS3249
Splitting slow task into independent sub-tasks.
-
CS3249 Soft Real-Time Systems 64CS3249 64Soft Real-Time Systems
Summary
Soft real-time system Overload degrades system performance.
Overload handling methods adaptive quality of service multithreading task skipping
Explicit loop that does nothing wastes CPU resource.
Event activation saves CPU resource, if done properly.
In multithreading, secondary threads communicate with main thread using signals and slots.
-
CS3249 Soft Real-Time Systems 65CS3249 65Soft Real-Time Systems
Further Reading
Soft real-time systems: [Buttazzo2005].
-
CS3249 Soft Real-Time Systems 66CS3249 66Soft Real-Time Systems
Exercise
Write the functions Buffer::skipToFront() and Buffer::clear().
-
CS3249 Soft Real-Time Systems 67CS3249 67Soft Real-Time Systems
Reference
G. Buttazzo, G. Lipari, L. Abeni, M. Caccamo. Soft Real-Time Systems: Predictability vs. Efficiency, Springer, 2005.
top related