Cacomania: Develop a Firefox add-on using GhostText as example

Cacomania

Develop a Firefox add-on using GhostText as example

Guido Krömer - 13. November 2014 - Tags: , ,

During the last months it might seem that my blog is dead, but it is not. I spend my most open source time developing GhostText. I already mentioned it here SublimeTextArea, which was the old name this project. But Federico and I decided renaming the project since it does not have to be restricted to Sublime Text. If you have no idea what GhostText is I recommend this video which show the the Chrome extension version.

In this small tutorial I am going to explain how to build a Firefox add-on using the Add-on SDK which connects a text area or JavaScript based code editor like the ACE editor with an editor running a GhostText server. Currently, the server is available for Sublime Text 3 and a prototype for the Atom editor.

Installing the Add-on SDK

Before I start describing how the add-on was build it is necessary to install the Add-on SDK. Normally it's enough to download and extract the SDK. I extracted it into my PhpstormProjects folder which makes switching from the SDK to the add-on's project folder a little bit quicker.

Once the SDK is extracted you should check your system's default python version by running python --version. If the default python version is 2.7 you can proceed to the next step, but if you are using Arch Linux for example, which default python version is three you have to apply a little workaround that is described in the ArchWiki.

This simple file below stored in /usr/local/bin/python let the add-on SDK run with python2 and all other scripts with python3, you have to ensure that it is executable: chmod +x /usr/local/bin/python . The location where the SDK has been extracted has to be aligned .

#!/bin/bash
script=$(readlink -f -- "$1")
case "$script" in (/home/caco/PhpstormProjects/addon-sdk-1.17/*)
    exec python2 "$@"
    ;;
esac

exec python3 "$@"

Building a simple add-on

The best way to check everything works as expected is building a simple add-on, this test add-on changes the current page background color to red when the user clicks on the action button.

A new add-on gets initialized by running cfx init from within the SDK's terminal in an empty folder. By executing source bin/activate in the add-on SDK folder starts the SDK's terminal.

[caco@A8-3870 SimpleFFAddOn]# cd addon-sdk-1.17
[caco@A8-3870 SimpleFFAddOn]# source bin/activate
(addon-sdk-1.17)[caco@A8-3870 SimpleFFAddOn]# mkdir FirefoxAddOnTest
(addon-sdk-1.17)[caco@A8-3870 SimpleFFAddOn]# cd FirefoxAddOnTest
(addon-sdk-1.17)[caco@A8-3870 SimpleFFAddOn]# cfx init
(addon-sdk-1.17)[caco@A8-3870 SimpleFFAddOn]# ls
data  lib  package.json  test

The file main.js in lib/ folder is your extension's entry point. If you are familiar with Chrome extension development the first lines might be very unlikely, each part of the API has to be loaded explicitly by using require. Our test add-on has an action button, therefore the corresponding API require('sdk/ui/button/action') has to be loaded first. A list of all available APIs can be found in the MDN documentation.

The main.js script is really straightforward, a new action button gets defined which has an onClick callback. When the user clicks the button the script content.js gets injected into the browser's active tab. The injected script gets executed immediately. Instead of attaching a file it is possible to inject some JavaScript as string, too. The contentScriptOptions can be used to pass primitive variable like numbers, strings, arrays and plain objects to the content tab's injected script, I used it to pass the background color to the tab.

Loading files shipped with the add-on and stored in the data/ folder is possible using the self API and is used for loading the content.js script which gets past to the tab.

var buttons = require('sdk/ui/button/action');
var tabs = require('sdk/tabs');
var self = require('sdk/self');

buttons.ActionButton({
    id: 'firefox-add-on-test',
    label: 'Firefox add-on test',
    icon: {
        "32": "./icon/32.png"
    },
    onClick: function () {
        tabs.activeTab.attach({
            contentScriptFile: [
                self.data.url('content.js')
            ],
            contentScriptOptions: {
                color: 'red'
            }
        });
    }
});

As already mentioned the content script has to be saved into the data/ folder and is a stupid one liner, as you can see below, setting the background color to the past color string via self.options.

document.body.style.background = self.options.color;

Now the newly created add-on can be tested by calling cfx run from within the SDK's terminal, a new and clean Firefox instance, with the loaded add-on, will start.

Building a simple add on which modifies the current tabs DOM is easy, in the next part I am going describe how the "GhostText For Firefox" add-on was built.

GhostText For Firefox

I think explaining the protocol first makes more sense at this point, before describing how the FireFox add-on it self was developed. So I do a small explanation on the protocol, which is used for the communication between the browser plug-in and the editor plug-in.

The protocol

The idea behind the protocol was being simple rather than being complex an efficient which should speed up the development of new Editor or Browser side plug-ins.

The Editor has to start a HTTP server, running default on Port 4001, which awaits new connections from the Browser. If the GhostText client wants to establish a new connection, a HTTP request on the local running HTTP server starts a new WebSocket server and the HTTP Server returns the protocol version and the port the WebSocket server is listening. The response is a simple JSON document which could look like this one:

{"ProtocolVersion": 1, "WebSocketPort": 43929}

The Browser's GhostText plug-in now connects to the GhostText server's WebSocket listening on the given port and sends an initial text change as JSON. The client message contains a title, the connected element's text, the cursor or selected text range and the page's host name.

The url field is used to let the editor change the syntax by the given host name, cacodaemon.de for example lets Sublime Text change the syntax to HTML.

This example shows the message send to the client from the editor:

{
    "title":      "The page's title",
    "text":       "The text area text…",
    "selections": [{"start": 11, "end": 19}],
    "url":        "example.com"
}

The changes send from the GhostText server to the browser are quite similar, the only difference is the url field which has been replaced by the syntax field which tells the browser the syntax scheme the editor uses. This example below could be an update send to the browser:

{
    "title":      "The page's title",
    "text":       "The text area text…, and some more…",
    "selections": [{"start": 35, "end": 35}],
    "syntax":     "Markdown.tmLanguage"
}

The implementation

I think the best way to explain how this Firefox add-on works is going the way from the action button click to the first received text change from the editor step-by-step.

Like in the small background color change add-on everything begins within the main.js. To be more specific everything begins in the init() method, which gets called immediately when the add-on gets loaded. It has three tasks, creating a page-worker, adding the action button and adds some event listeners for tab specific events. The existence of the ActionButton should be self explaining, but what is a page-worker and for what is it needed?

Firefox disallows non secure WebSocket connections from HTTPS pages and setting up a secure WebSocket server within the editor plug-in would be really hard to implement. Even if implemented this would have not solved the problem with the self signed certificate which is not trusted by the browser by default. Therefore, GhostText goes a way around this problem by sending all messages through a WebSocket from a non secure page. But this leads to another problem, the Firefox add-on SDK has no WebSocket API which can be used, the main.js code is limited to methods which provided by an add-on SDK API. This is where the page-worker comes into play, it creates a new invisible page where all common HTML5 JavaScript APIs are available, including WebSockets.

It means that all text changes from the text area, for example, has to be passed to the main.js script which redirects the messages directly to the background page and all text changes from the GhostText server has to be forwarded from the receiving background page to the main.js script which transfer the changes to the matching tab.

init: function () {
    GhostTextMain.backgroundPage = pageWorker.Page({
        contentScriptFile: [self.data.url('background.js')],
        contentURL: self.data.url('background.html'),
        contentScriptWhen: 'ready',
        onMessage: GhostTextMain.onMessageBackground,
        contentScriptOptions: {
            verbose: GhostTextMain.verbose
        }
    });

    GhostTextMain.button = buttons.ActionButton({
        id: 'ghost-text',
        label: 'GhostText',
        icon: {
            "18": "./icon/18.png", // toolbar icon non HiDPI
            "32": "./icon/32.png", // menu panel icon non HiDPI
            "36": "./icon/36.png", // toolbar icon HiDPI
            "64": "./icon/64.png"  // menu panel icon HiDPI
        },
        onClick: GhostTextMain.handleClick
    });

    tabs.on('close', GhostTextMain.onTabClosedOrReloaded);
    tabs.on('pageshow', GhostTextMain.onTabClosedOrReloaded)
}

Once the action button has been clicked handleClick gets called, it checks if the active tab is connected with the server or not. If it is not connected the content script gets injected into the tab and a message from type select-field will be send to the tab. If the tab is already connected a disconnect gets initiated by sending a close-connection message to the background page, which handles the open WebSockets.

handleClick: function () {
    /* … */
    var activeTab = tabs.activeTab;

    if (!GhostTextMain.connectedTabs[activeTab.id]) { //connect
        GhostTextMain.connectedTabs[activeTab.id] = true;
        GhostTextMain.injectScript(activeTab);
        GhostTextMain.tabsWithScriptInjected[activeTab.id].postMessage({
            type: 'select-field'
        });
    } else { //disconnect
        GhostTextMain.backgroundPage.postMessage({
            type: 'close-connection',
            tabId: activeTab.id
        });
        GhostTextMain.connectedTabs[activeTab.id] = false;
    }
}

The GhostTextMain.injectScript() function attaches the needed scripts to the provided tab and ensures that the scripts gets only injected once per tab. All messages send from a tab gets processed by the GhostTextMain.onMessageTab() function.

The are three script which are attached to the tab, the content.js script itself. input-area.js which is a abstraction layer between text area elements, JS code editors like Code Mirror or the ACE editor and content editable elements and the content script. The GhostText.InputArea script is the same used in the Chrome extension. It was intitally developed for the chrome extension but needed only some minor changes for the Firefox browser. The last script is humane.js a notification system used by GhostText.

For proper message passing through the add-on the main.js tells the content.js script it's tab id using the contentScriptOptions, this id gets used for directing messages from the content script to the right WebSocket and message from the WebSocket to the right tab.

injectScript: function (activeTab) {
    /* … */

    if (GhostTextMain.tabsWithScriptInjected[activeTab.id]) {
        return false;
    }

    GhostTextMain.tabsWithScriptInjected[activeTab.id] = activeTab.attach({
        contentScriptFile: [
            self.data.url('vendor/humane-ghosttext.min.js'),
            self.data.url('input-area.js'),
            self.data.url('content.js')
        ],
        contentScriptOptions: {
            tab: {
                id: activeTab.id,
                title: activeTab.title,
                url: url.URL(activeTab.url)
            },
            css: self.data.url('vendor/humane-ghosttext.css'),
            verbose: GhostTextMain.verbose
        },
        onMessage: GhostTextMain.onMessageTab
    });

    return true;
}

Now we have to switch to the content.js file which awaits a message send by the main.js script from type select-field. So the first thing the content script does is binding the message event to the GhostTextContent.messageHandler function which delegates all incoming messages.

self.on('message', GhostTextContent.messageHandler);

There is nothing special about the messageHandler, if a select-field message arrives the GhostTextContent.selectField() method gets called.

messageHandler: function (message) {
    /* … */

    switch (message.type) {
        case 'select-field':
            GhostTextContent.selectField();
            break;
        case 'disable-field':
            GhostTextContent.disableField();
            break;
        case 'text-change':
            var response = JSON.parse(message.change);
            GhostTextContent.currentInputArea.setText(response.text);
            GhostTextContent.currentInputArea.setSelections(GhostText.InputArea.Selections.fromPlainJS(response.selections));
            break;
        case 'close-connection':
            GhostTextContent.currentInputArea.unbind();
            GhostTextContent.currentInputArea = null;
            GhostTextContent.informUser('Disconnected! \n Report issues');
            break;
        case 'error':
            GhostTextContent.handleError(message);
            break;
        default:
            GhostTextContent.log(['Unknown message of type', message.type, 'given'].join(' '));
    }
}

The selectField method instantiates a InputArea detector which searches for supported input elements. If the user focuses a text area for example the focusEvent gets risen which parameter is an instance of IInputArea. Once the input area has been selected the enableField method binds all input area specific events. This layer of abstraction makes it easy integrating new input areas, like fancy JS code editors for example and encapsulates some complexity from the browser specific add-on/extension… .

The InputArea is written in TypeScript code belongs to the Chrome extension GhostText-for-Chrome repository, the Firefox add-on has only a compiled version of this part of the software. Explaining this part would be interesting but might go beyond the scope of this tutorial.

selectField: function () {
    /* … */

    var detector = new GhostText.InputArea.Detector(GhostText.InputArea.Browser.Firefox);
    detector.focusEvent(function (inputArea) {
        GhostTextContent.currentInputArea = inputArea;
        GhostTextContent.enableField();
    });

    var countElementsFound = detector.detect(document);
    if (countElementsFound === 0) {
        GhostTextContent.informUser('No text area elements on this page');
    } else if (countElementsFound > 1) {
        GhostTextContent.informUser('There are multiple text areas on this page. \n Click on the one you want to use.');
    }
}

Now all events get binded to the corresponding content.js methods, the most important one is the textChangedEvent which task should be clear, after all events has been bound the connect message gets send to the main.js script.

enableField: function () {
    /* … */

    var inputArea = GhostTextContent.currentInputArea;

    inputArea.textChangedEvent(GhostTextContent.reportFieldData);
    inputArea.removeEvent(GhostTextContent.requestServerDisconnection);
    inputArea.unloadEvent(GhostTextContent.requestServerDisconnection);
    inputArea.focusEvent(null); //disable
    inputArea.selectionChangedEvent(null);

    /** @type TextChange */
    var textChange = inputArea.buildChange();

    self.postMessage({
        tabId: self.options.tab.id,
        change: JSON.stringify(textChange),
        type: 'connect'
    });
}

Back in the main.js script all messages from a tab gets redirected to the backgound.js script handling the WebSocket logic, but the connect message has to be handled by the main.js script since a Ajax request on any host is only allowed at this part of the add-on. The GhostTextMain.handleAjax is a little convenience method doing the HTTP request.

The Ajax response gets send to the background.js script that is going to start the WebSocket connection.

onMessageTab: function (message) {
    /* … */

    switch (message.type) {
        case 'connect':
            GhostTextMain.handleAjax('http://localhost:' + prefs.port, function (response) {
                GhostTextMain.log('AJAX ' + JSON.stringify(response));

                GhostTextMain.backgroundPage.postMessage({
                    type: 'connect',
                    tabId: message.tabId,
                    response: response,
                    originMessage: message
                });
            });
            break;

        default:
            GhostTextMain.backgroundPage.postMessage(message);
    }
}

You can see the reason for this method here, in a world of jQuery, Angular or Backbone.js doing a plain Ajax call became a little bit inconvenient. But the comfort of using a JS library for this job does not exist in the main.js context. This is a great drawback in comparison to the Chrome extension development.

handleAjax: function (url, response) {
    /* … */

    var xhr = new XMLHttpRequest();
    xhr.open('GET', url, true /*async*/);
    xhr.send();
    xhr.onreadystatechange = function () {
        if (this.readyState !== 4) {
            return;
        }

        response({
            status: this.status,
            responseText: this.responseText
        });
    };
}

Now we finally reached the backgound.js script, it has a messageHandler like the content.js and main.js which calls the right method based on the given message type. The connect message send by the main.js calls the connect method. This function creates a new WebSocket and connects it to the GhostText Server listening on the Port provided by the HTTP response, send in the main.js. Once the WebSocket onopen event has been fired the content.js script gets informed by passing a message through the main.js to the target tab's content.js script.

The content.js script sends an initial text change back to the background.js script when a message of the type connected has been received.

All messages send through the WebSocket to the add-on gets marked as text-change and will be redirected to the target tab, take a look at the onmessage event below. For redirecting a message to the target tab the tabId is used.

connect: function (message) {
    /* Some error handling code has been intentionally removed for this tutorial. */

    var webSocket = new WebSocket('ws://localhost:' + response.WebSocketPort);

    webSocket.onopen = function () {
        GhostTextBackground.log('webSocket.onopen');

        self.postMessage({
            tabId: message.tabId,
            type: 'connected'
        });
    };

    webSocket.onmessage = function (event) {
        self.postMessage({
            tabId: message.tabId,
            type: 'text-change',
            change: event.data
        });
    };

    webSocket.onclose = function () {
        delete GhostTextBackground.webSockets[message.tabId.toString()];

        self.postMessage({
            tabId: message.tabId,
            type: 'disable-field'
        });
    };

    webSocket.onerror = function () {
        GhostTextBackground.postErrorMessage(message.tabId, 'web-socket');
    };

    GhostTextBackground.webSockets[message.tabId.toString()] = webSocket;
}

I could start bore you explaining the whole way back and how the text replaced the input area content, but I think that I have explained all necessary parts of the add-on and how to do WebScoket and HTTP connection to every server by outmaneuver the Firefox security restrictions.

Conclusion

I have to admit developing a Chrome extension is a little bit more comfortable, especially the debugging features and the fact that no browser restart is needed for reloading the add-on. Using the word debugging is a little bit absurd since the most time a console.log has to be used which output often appears in the terminal and not in the Developer Tools console.

I hope you liked reading this tutorial and the concept behind GhostText, too. The code is available at GitHub, here is the tag used during writing this tutorial: 0.0.9-beta. Here are some more interesting links about GhostText like the Sublime Text 3 plug-in at Package Control the Chrome extension or Firefox add-on it self.

Feel free to leave a comment here or create a new issue at GitHub!