How to test chrome extensions?

Question

Is there a good way to do this? I'm writing an extension that interacts with a website as a content script and saves data using localstorage. Are there any tools, frameworks, etc. that I can use to test this behavior? I realize there are some generic tools for testing javascript, but are those sufficiently power to test an extension? Unit testing is most important, but I'm also interested in other types of testing (such as integration testing).

Answer 1

Yes, the existing frameworks are pretty useful..

In the recent past, I have placed all my tests on a "test" page that was embedded in to the application but not reachable unless physically typed.

For instance, I would have all the tests in a page accessible under chrome-extension://asdasdasdasdad/unittests.html

The tests would have access to localStorage etc. For accessing content scripts, in theory you could test that through embedded IFRAMEs in your test page, however these are more integration level testing, unit tests would require you to abstract that away from real pages so that you don't depend on them, likewise with access to localStorage.

If you want to test pages directly, you can orchestrate your extension to open new tabs (chrome.tab.create({"url" : "someurl"}). For each of the new tabs your content script should run and you can use your testing framework to check that your code has done what it should do.

As for frameworks, JsUnit or the more recent Jasmine should work fine.

Answer 2

Working on several chrome extensions I came up with sinon-chrome project that allows to run unit-tests using mocha, nodejs and phantomjs.

Basically, it creates sinon mocks of all chrome.* API where you can put any predefined json responses.

Next, you load your scripts using node's vm.runInNewContext for background page and phantomjs for render popup / options page.

And finally, you assert that chrome api was called with needed arguments.

Let's take an example:
Assume we have simple chrome extension that displays number of opened tabs in button badge.

background page:

chrome.tabs.query({}, function(tabs) {
  chrome.browserAction.setBadgeText({text: String(tabs.length)});
});

To test it we need:

1. mock chrome.tabs.query to return predefined response, e.g. two tabs.
2. inject our mocked chrome.* api into some environment
3. run our extension code in this environment
4. assert that button badge equals to '2'

The code snippet is following:

const vm = require('vm');
const fs = require('fs');
const chrome = require('sinon-chrome');

// 1. mock `chrome.tabs.query` to return predefined response 
chrome.tabs.query.yields([
  {id: 1, title: 'Tab 1'}, 
  {id: 2, title: 'Tab 2'}
]);

// 2. inject our mocked chrome.* api into some environment
const context = {
  chrome: chrome
};

// 3. run our extension code in this environment
const code = fs.readFileSync('src/background.js');
vm.runInNewContext(code, context);

// 4. assert that button badge equals to '2'
sinon.assert.calledOnce(chrome.browserAction.setBadgeText);
sinon.assert.calledWithMatch(chrome.browserAction.setBadgeText, {
  text: "2"
});

Now we can wrap it into mocha's describe..it functions and run from terminal:

$ mocha

background page
  ? should display opened tabs count in button badge

1 passing (98ms)

You can find full example here.

Additionally, sinon-chrome allows to trigger any chrome event with predefined response, e.g.

chrome.tab.onCreated.trigger({url: 'http://google.com'});

Answer 3

While sinon.js seems to work great, you can also just use plain Jasmine and mock the Chrome callbacks you need. Example:

Mock

chrome = {
  runtime: {
    onMessage : {
      addListener : function() {}
    }
  }
}

Test

describe("JSGuardian", function() {

  describe("BlockCache", function() {

    beforeEach(function() {
      this.blockCache = new BlockCache();
    });

    it("should recognize added urls", function() {
      this.blockCache.add("http://some.url");
      expect(this.blockCache.allow("http://some.url")).toBe(false);
    });
} // ... etc

Just modify the default SpecRunner.html to run your code.

Answer 4

About already existing tool in Chrome:

1. In the chrome developer tool, there is section for Resources For local storage.

   Developer Tools > Resources > Local Storage

   See the changes of localstorage there.

2. You can use console.profile to test performance and watch run time call stack.

3. for fileSystem You can use this URL to check your file is upload-ed or not: filesystem:chrome-extension:///temporary/

If you are use content script and local-storage together without background page/script and without message passing, local-storage will be accessible from that site only. So, to test those pages, you have to inject your test script in those tabs.

Answer 5

I found that I can use Selenium web driver for starting fresh browser instance with preinstalled extension and pyautogui for clicks - because Selenium cannot drive "view" of the extension. After clicks you can make screenshots and compare them with 'expected' ones, expecting 95% of similarity (because on different browsers it is acceptable markup movements to a few pixels).

Answer 6

To confirm a couple previous answers, Jasmine seems to work well with Chrome extensions. I'm using version 3.4.0.

You can use Jasmine spies to easily create test doubles for the various APIs. No need to build your own from scratch. For example:

describe("Test suite", function() {

  it("Test case", function() {

    // Set up spies and fake data.
    spyOn(chrome.browserAction, "setPopup");
    spyOn(chrome.identity, "removeCachedAuthToken");
    fakeToken = "faketoken-faketoken-faketoken";
    fakeWindow = jasmine.createSpyObj("window", ["close"]);

    // Call the function under test.
    logout(fakeWindow, fakeToken);

    // Perform assertions.
    expect(chrome.browserAction.setPopup).toHaveBeenCalledWith({popup: ""});
    expect(chrome.identity.removeCachedAuthToken).toHaveBeenCalledWith({token: fakeToken});
    expect(fakeWindow.close.calls.count()).toEqual(1);

  });

});

Some more details, if it helps:

As mentioned in another answer, I created an HTML page as part of my browser extension that runs my tests. The HTML page includes the Jasmine library, plus my extension's JavaScript code, plus my test suite. The tests are run automatically and the results are formatted for you. No need to build a test runner or a results formatter. Just follow the installation instructions, and use the HTML documented there to create your test runner page, and include your test suite in the page as well.

I don't think you can fetch the Jasmine framework dynamically from another host, so I just included the Jasmine release in my extension. I will omit it and also my test cases when I build my extension for production, of course.

I haven't looked at how to execute my tests at the command line. That would be handy for automated deployment tools.

Answer 7

To test End-to-End you can use puppeteer. Here is snippet I wrote for my extension to check loaded extension title and to verify if extension was enabled in incognito mode.

const path = require("path");
const puppeteer = require("puppeteer");
const assert = require("assert");
const Constants = require("../contants");
const Utils = require("./util");

const extensionID = Constants.EXTENSION_ID;
const extensionPath = path.join(__dirname, "../dist");
const extensionOptionHtml = "option.html";
const extPage = `chrome-extension://${extensionID}/${extensionOptionHtml}`;
let extensionPage = null;
let browser = null;

async function boot() {
  browser = await puppeteer.launch({
    // slowMo: 250,
    headless: false, // extension are allowed only in head-full mode
    args: [
      `--disable-extensions-except=${extensionPath}`,
      `--load-extension=${extensionPath}`,
      "--no-sandbox",
      "--disable-setuid-sandbox"
    ]
  });

  extensionPage = await browser.newPage();
  await extensionPage.goto(extPage);
}

describe("Extension UI Testing", function() {
  this.timeout(20000); // default is 2 seconds and that may not be enough to boot browsers and pages.
  before(async function() {
    await boot();
  });

  describe("option page home", async function() {
    it("check title", async function() {
      const h1 = "Allow extension in Incognito Mode";
      const extH1 = await extensionPage.evaluate(() =>
        document.querySelector("h1").textContent.trim()
      );
      assert.equal(extH1, h1);
    });
    it("show option ui after enabling extension in incognito", async () => {
      await extensionPage.goto(`chrome://extensions/?id=${extensionID}`);
      extensionPage.evaluate(() =>
        document
          .querySelector("body > extensions-manager")
          .shadowRoot.querySelector("#viewManager > extensions-detail-view")
          .shadowRoot.querySelector("#allow-incognito")
          .shadowRoot.querySelector("#crToggle")
          .click()
      );
      await Utils.sleep(2000);
      await extensionPage.goto(
        `chrome-extension://${extensionID}/${extensionOptionHtml}`
      );
      const h3 = "Mark Incognito";
      const headingID = `#${Constants.OPTION_SCRIPT_HOST_ID} > div > div > header > div > h6`;
      await extensionPage.waitFor(headingID);
      console.log({ headingID });
      const extH3 = await extensionPage.evaluate(headingID => {
        return document.querySelector(headingID).textContent.trim();
      }, headingID);
      console.log({ extH3 });
      assert.equal(extH3, h3);
    });
  });

  after(async function() {
    await browser.close();
  });
});