The Webkit Bridge Tutorial - Hybrid Client Application

Files:

• webkit/imageanalyzer/imageanalyzer.cpp
• webkit/imageanalyzer/imageanalyzer.h
• webkit/imageanalyzer/mainwindow.cpp
• webkit/imageanalyzer/mainwindow.h
• webkit/imageanalyzer/main.cpp
• webkit/imageanalyzer/imageanalyzer.pro
• webkit/imageanalyzer/resources/imageanalyzer.qrc

In this example, we will show how to write a hybrid application using QtWebKit Bridge, which distinguishes itself from a thin client in that it performs heavy calculations on the client side in C++, like a native application, but presents nothing more than a QWebView for its user interface, displaying web content written in HTML/JavaScript. The application uses QtConcurrent to distribute its work across as many CPU cores as are available from the system, so it can process each image in parallel.

For the full reference documentation of QtWebKit hybrid development, see The QtWebKit Bridge.

Initially, you will see a user interface with an empty list of images. Clicking on some of the images in the lower pane below adds them to the list view above, as shown in the screenshot below.

Now, we can click on Analyze, and each image is analyzed using some computationally intensive C++ function, in parallel and on different cores. Progress is shown while the analysis is proceeding.

and in the end, we will see something like this, where the average RGB values of each image are shown.

The MainWindow is defined in C++, and creates a QNetworkDiskCache and a QWebView, and tells the QWebView to load the starting page, providing us with a user interface for the client.

 MainWin::MainWin(QWidget * parent) : QWebView(parent)
 {
     m_network = new QNetworkAccessManager(this);
     m_cache = new QNetworkDiskCache(this);
     m_cache->setCacheDirectory(QDesktopServices::storageLocation(QDesktopServices::CacheLocation) + "/imageanalyzer");
     m_cache->setMaximumCacheSize(1000000); //set the cache to 10megs
     m_network->setCache(m_cache);
     page()->setNetworkAccessManager(m_network);

     m_analyzer = new ImageAnalyzer(m_cache, this);

     // Signal is emitted before frame loads any web content:
     QObject::connect(page()->mainFrame(), SIGNAL(javaScriptWindowObjectCleared()),
                      this, SLOT(addJSObject()));

     // qrc:// URLs refer to resources. See imagenalayzer.qrc
     QUrl startURL = QUrl("qrc:/index.html");

     // Load web content now!
     setUrl(startURL);
 }

In this example, the sample content is addressed with the qrc:/index.html URL. qrc:/ indicates that the file is stored as a Qt resource (attached to the executable). In a real-world application, the content and images would likely be retrieved from the network rather than from resources.

We wish to initialize an object reference in the JavaScript web page to point to our ImageAnalyzer before any other scripts are run. To do this, we connect the javaScriptWindowObjectCleared() signal to a slot which does the object creation and handoff to JavaScript.

 void MainWin::addJSObject() {
     // Add pAnalyzer to JavaScript Frame as member "imageAnalyzer".
     page()->mainFrame()->addToJavaScriptWindowObject(QString("imageAnalyzer"), m_analyzer);
 }

The ImageAnalyzer object is created and added to a JavaScript object on the web page's mainFrame with addToJavaScriptWindowObject().

The start page is resources/index.html. In one of its <div> regions, we have images, each with an onClick() handler that calls addImage().

 <div id=imagediv style="clear:both; text-align:center">
 <hr/>
     <img src="images/mtRainier.jpg" height=150px onclick='return addImage(this);' />
     <img src="images/bellaCoola.jpg" height=150px onclick='return addImage(this);'/>
     <img src="images/seaShell.jpg" height=150px onclick='return addImage(this);'/>

Clicking an image adds it to an images list.

 function addImage(newimg) {
     var imglist = document.getElementById('imglist');
     var curChildren = imglist.childNodes;
     var newline = document.createElement('option');
     newline.innerHTML = newimg.src.substring(newimg.src.lastIndexOf('/')+1);
     newline.value = newimg.src;
     imglist.appendChild(newline);
 }

The Analyze button at the bottom of the image list is clicked when we want to start the analysis:

     <div style="float:right; width:50%; border-left: solid 1px black">
         <div id="listdiv" align="center">
             <h5>Images to be analyzed:</h5>
             <select multiple size=10 id=imglist style="width:80%"></select>
             <br />
             <input type="button" id="evalbutton" value="Analyze" onclick="analyzeImages()" />
         </div>
     </div>

When the user clicks the Analyze button, analyzeImages() is called, another regular JavaScript method, shown below. Notice it assumes the imageAnalyzer object is already defined and initialized in JavaScript space, but we guaranteed that by connecting our setup slot to the appropriate signal, javaScriptWindowObjectCleared().

 function analyzeImages() {
     connectSlots();
     var imglist = document.getElementsByTagName('option');
     if (imglist.length > 0) {
         stringlist = [];
         for(var i=0; i<imglist.length; i++) {
             stringlist[i]=imglist[i].value;
         }
         if (!imageAnalyzer.busy) {
             remaining = stringlist.length;
             imageAnalyzer.startAnalysis(stringlist);
         } else {
             alert("Processing, please wait until finished.");
         }

The only methods on ImageAnalyzer that we can or do call from JavaScript are those which are exposed through {The Meta-Object System}{Qt's MetaObject} system: property getter/setter methods, public signals and slots, and other Q_INVOKABLE functions.

 class ImageAnalyzer : public QObject
 {
     Q_OBJECT
 public:
     ImageAnalyzer(QNetworkDiskCache * netcache, QObject * parent=0);

     QRgb lastResults();
     float lastRed();
     float lastGreen();
     float lastBlue();
     bool isBusy();
     Q_PROPERTY(bool busy READ isBusy);
     Q_PROPERTY(float red READ lastRed);
     Q_PROPERTY(float green READ lastGreen);
     Q_PROPERTY(float blue READ lastBlue);
     ~ImageAnalyzer();

 public slots:
     /*! initiates analysis of all the urls in the list */
     void startAnalysis(const QStringList & urls);

 signals:
     void finishedAnalysis();
     void updateProgress(int completed, int total);
     ...
 private:
 QNetworkAccessManager* m_network;
 QNetworkDiskCache* m_cache;
 QStringList m_URLQueue;
 QList<QImage> m_imageQueue;
 int m_outstandingFetches;
 QFutureWatcher<QRgb> * m_watcher;

Most of the members are set up in the constructor:

 ImageAnalyzer::ImageAnalyzer(QNetworkDiskCache* netcache, QObject* parent)
     : QObject(parent), m_cache(netcache), m_outstandingFetches(0)
 {
     /*  ImageAnalyzer only wants to receive http responses
         for requests that it makes, so that's why it has its own
         QNetworkAccessManager. */
     m_network = new QNetworkAccessManager(this);
     m_watcher = new QFutureWatcher<QRgb>(this);
     /*  We want to share a cache with the web browser,
         in case it has some images we want: */
     m_network->setCache(m_cache);

     QObject::connect(m_network, SIGNAL(finished(QNetworkReply*)),
                      this, SLOT(handleReply(QNetworkReply*)));
     QObject::connect(m_watcher, SIGNAL(finished()),
                      this, SLOT(doneProcessing()));
     QObject::connect(m_watcher, SIGNAL(progressValueChanged(int)),
                      this, SLOT(progressStatus(int)));
 }

Back on the JavaScript side, we want to connect signals from this object to JavaScript functions on our web page, after the web page is loaded, but before the images are analyzed.

From connectSlots(), we can see how to connect signals from the imageAnalyzer object to regular JavaScript functions, which can also behave like slots. We use this to monitor and display progress from the C++ side.

 function connectSlots()
 {
     if ( !connected ) {
         connected = true;
         imageAnalyzer.finishedAnalysis.connect(this, finished);
         imageAnalyzer.updateProgress.connect(this, updateProg);
     }
 }

The only public slot is startAnalysis(), called to place a list of URLs into the image analyzer's QtConcurrent processing queue from JavaScript space.

 void ImageAnalyzer::startAnalysis(const QStringList & urls)
 {
     m_URLQueue = urls;
     fetchURLs();
 }

The images need to be loaded again now, which is why fetchURLs first checks the cache to see if we can save an extra network get.

 void ImageAnalyzer::fetchURLs()
 {
     while (!m_URLQueue.isEmpty())
     {
         QString url = m_URLQueue.takeFirst();
         QUrl URL = QUrl(url);
         QIODevice * pData = m_cache->data(URL);
         // Is image already loaded in cache?
         if (pData == 0) {
             // HTTP Get image over network.
             m_outstandingFetches++;
             QNetworkRequest request = QNetworkRequest(URL);
             request.setRawHeader("User-Agent", "Nokia - Custom QT app");
             m_network->get(request);
         } else {
             // Get image from cache
             QImage image;
             image.load(pData, 0);
             if (!image.isNull())
                 queueImage(image);
             delete(pData);
         }
     }
 }

For the images that were not in the cache, handleReply() will load them into a QImage when the data is ready.

 void ImageAnalyzer::handleReply(QNetworkReply * pReply)
 {
     m_outstandingFetches--;
     if (pReply->error()) {
         qDebug() << "Error code" << pReply->error();
         qDebug() << "Http code" << pReply->attribute(QNetworkRequest::HttpStatusCodeAttribute);
         return;
     }
     QImage image;
     image.load(pReply, 0);
     pReply->deleteLater();
     if (image.isNull()) {
         qDebug() << "bad image";
         qDebug() << pReply->rawHeaderList();
         foreach(QByteArray element, pReply->rawHeaderList()) {
             qDebug() << element << " = " << pReply->rawHeader(element);
         }
         return;
     }
     queueImage(image);
 }

After the images are loaded, they are queued up in preparation to be sent in a batch for analysis to a QFutureWatcher, which will distribute the processing across multiple threads and cores, depending on how many are available.

 void ImageAnalyzer::queueImage(QImage img)
 {
     if (!img.isNull())
         m_imageQueue << img;

     if (m_outstandingFetches == 0 && m_URLQueue.isEmpty()) {
         m_watcher->setFuture(QtConcurrent::mapped(m_imageQueue, averageRGB));
     }
 }

The function that gets performed on each image is averageRGB(), as specified in argument 2 to the QtConcurrent::mapped() function. Notice it repeats the same calculations 100 times on each pixel to keep the CPU very busy. This is done only for the purposes of the demo so that the analysis takes a noticeable time to complete.

 QRgb averageRGB(const QImage &img)
 {
     int pixelCount = img.width() * img.height();
     int rAvg, gAvg, bAvg;

     // We waste some time here:
     for (int timeWaster=0; timeWaster < 100; timeWaster++) {
         quint64 rTot = 0;
         quint64 gTot = 0;
         quint64 bTot = 0;
         for (int i=0; i < img.width(); i++) {
             for (int j=0; j < img.height(); j++) {
                 QRgb pixel = img.pixel(i,j);
                 rTot += qRed(pixel);
                 gTot += qGreen(pixel);
                 bTot += qBlue(pixel);
             }
         }
         rAvg = (rTot)/(pixelCount);
         gAvg = (gTot)/(pixelCount);
         bAvg = (bTot)/(pixelCount);
     }
     return qRgb(rAvg, gAvg, bAvg);
 }