Articles published on the website

• Mobile Analytics SDK: beta release of Piwik iOS SDK

  30 October 2013, by Piwik team

  Mattias Levin, a Mobile developer enthusiast from Sweden, has released the first public beta version of the official Piwik SDK for iOS!

  If you are building apps for iOS or OSX, you will be able to track your App usage with Piwik. Learn more in this blog post.

  Apps & Mobile apps Analytics

  Using Piwik to track your app usage would give interesting statistics usage such as:

  • number of active users (per day, week, month, …) of my mobile or desktop app,
  • how long users spend in the app,
  • track which icons, buttons are clicked (or any other custom event),
  • record device info, operating system,
  • reports on any Custom Variables you that are relevant to your app (see examples below),
  • how often is the app opened? When and how long is the app opened?
  • number of new users, active users, total users,
  • record errors or exception thrown

  Piwik SDK for iOS
  

  The PiwikTracker is an Objective-C framework (for iOS and OSX) designed to send app usage data to a Piwik analytics server. It is realeased under MIT license. Piwik server is a downloadable, Free/Libre (GPLv3 licensed) real time analytics platform.

  Getting started

  1. Create a new website in the Piwik web interface called “My App”. Copy the Website ID and the token_auth.
  2. Download the PiwikTracker SDK.
  3. Add the PiwikTracker files to your project.
  4. Create and configure the PiwikTracker.
  5. Add code in your app to track screen views, events, exceptions, goals and more
  6. Let the dispatch timer dispatch pending events to the Piwik server, or dispatch events manually.

  For more info, check out the Readme.

  Requirements

  The latest PiwikTracker version uses ARC and support iOS6+ and OSX 10.7+.

  • iOS tracker depends on: Core Data, Core Location, Core Graphics, UIKit and AFNetworking.
  • OSX tracker depends on: Core Data, Core Graphics, Cocoa and AFNetworking.

  Demo project

  The workspace contains an iPhone demo app that uses and demonstrates the features available in the SDK.

  

  Feedback needed

  If you use the iOS SDK to track your app, we would like to hear your suggestions, bug reports or general feedback.

  We hope to work with you to improve the SDK and move it out of beta!

  Please report suggestions, bugs, feature requests in the Github Issues at Piwik iOS SDK.

  Happy App Analytics!

• New Piwik 2.0 public beta for testers

  16 October 2013, by matt

  Dear Piwik community,

  We are excited to announce the release of the first public beta version of Piwik 2.0.

  This is software still in development and we really don’t recommend that you run it on a production site — set up a test database just to play with the new version.

  Piwik 2.0 is a major update from Piwik 1.12 and is the result of 5 months of work on the platform!

  What’s changed?

  We focused on upgrading Piwik source code quality and maintainability: upgraded to PHP 5.3 using namespaces, changed templating library to Twig, started using composer, using Less as well as css, improved QA tests, introduced new Screenshots tests, refactored translations to nice JSON format, refactored LOTS of code, added documentation….

  There also performance improvements, in particular the “All Websites Dashboard” is now usable with 20,000+ websites!

  Several bugs were fixed and we added some very-special-and-exciting new features.

  See the list of closed tickets in Piwik 2.0, or learn more about our recent developments in the Development update blog post.

  Piwik 2 is the open platform for your analytics data!

  How to update to Piwik 2.0 beta?

  1. You can tell Piwik to use the latest beta from within the user interface. See this FAQ: [piwik.org]
  2. or alternatively you may download the latest beta from the build server, upload these files on top of your existing piwik files, and visit Piwik to upgrade.

  Beta cycle
  The more you test the beta, the more stable our release candidates and our final release will be. If you think you’ve found a bug, you can post it in this forum post. Or, if you’re comfortable writing a reproducible bug report, file one. The stable Piwik 2.0 release is planned for mid-November!

  Happy testing,

  PS: if you are interested in professional support for Piwik, or custom feature development, contact the Piwik experts.

• [Aug-Sept 2013] Piwik 2.0 Development Update!

  3 October 2013, by Fabian Becker — Development

  This Development Update is the first in a new series of posts we’ll be writing to keep you, our loyal users, informed of our efforts. We hope these updates keep you excited about Piwik’s future, and if you’re a developer, we hope they inspire and challenge you to accomplish more yourself!

  Despite this being our first update, it will probably be one of our biggest. We’ve gotten a lot done as we race towards the Piwik 2.0 release! Just see for yourself:

  What we’ve accomplished

  Theming

  Piwik now supports theming, a feature that was requested often in the past. Because of our switch to the Twig template engine and other major code changes it is now possible to change the way Piwik looks. Additionally, developers can use the dynamic stylesheet language LESS, instead of CSS. Piwik will automatically transform the LESS code into CSS.

  Piwik 2.0 will ship with a new dark theme called PleineLune (french for Full Moon) that makes use of the new theming feature. Another theme with a left-aligned menu was created during the Piwik Meetup in Paris. Both of these themes were created by Thomas Zilliox, a very talented designer and CSS expert.

  

  PHP 5.3 Namespaces

  For Piwik 2.0 we decided to make use of namespaces, a feature introduced in PHP 5.3. The usage of namespaces makes our code more readable and allows us to better modularize the platform. This is in part why we are raising the required minimum PHP version to 5.3 for Piwik 2.0. (Remember to update your server!)

  Translations in JSON

  All translations are now stored in JSON files which makes storing translations in Piwik a lot cleaner that the giant PHP array we previously used.

  Side note: if you’d like to make Piwik available to more languages, please sign up at translations.piwik.org. We’d love to have your help!

  UI Tests

  We now use UI tests to make sure that changes to the code don’t break the UI. UI tests use PhantomJS and CutyCapt and are automatically executed on Travis CI. Whenever an integration test fails the script produces a screenshot diff that shows the difference. Learn more.

  

  AnonymizeIP supports IPv6

  The AnonymizeIP plugin now masks IPv6 addresses. The concept of the config option ‘ip_address_mask_length’ has now changed to reflect the level of masking that should be applied to the IP. With a masking level of 1 Piwik will mask the last octet of an IPv4 address and the last 80 bits of an IPv6 address.

  All Websites Dashboard usable with 20,000+ Websites

  The All Websites Dashboard is now usable even if you track many thousands of websites in your Piwik instance. We rewrote parts of the archiving process in order to make this possible. Making Piwik fast and memory efficient is a constant concern for core developers.

  Plugins can now add new Visualizations

  Piwik Plugins and Themes can now create new visualizations for your report data. They can also specify their own ViewDataTable footer icons or modify existing ones. This will allow plugin developers to create new ways for you to view your data, customize existing reports so they look great in new visualizations and provide extra analytics functionality accessible in each of your reports.

  The new TreemapVisualization plugin makes use of this feature to let you view your reports as treemaps. It serves as an example of this new functionality.

  Piwik Marketplace

  The Piwik Marketplace is a new platform developers can use to publish their plugins and themes so all Piwik users can easily access them. The marketplace is hosted at plugins.piwik.org and is currently in an early development state, but we’re already able to host plugins!

  Developers can easily publish their plugins by adding a commit hook to their Github repositories. Every time you push a new tag, the marketplace will make a new version of your plugin available. The marketplace will provide a centralized platform to search for plugins and also provide statistics on plugin usage.

  Install Plugins and Themes in one click from within Piwik

  Piwik has offered since the beginning the much-loved “one click update” feature. We are bringing the same functionnality to the Marketplace: you will be able to install Plugins and Themes in one click directly within the Piwik interface! Similarly to WordPress or Firefox, Piwik will let you extend the functionnality of your analytics platform.

  Conclusion

  In Piwik 2.0 you will be able to install plugins and themes from the marketplace. And, if you’re so inclined, you will be able to create and host your own plugins/themes on the marketplace so everyone can use them. This is by far the accomplishment we are most excited by… the possibilities it opens up for Piwik’s future are truly unlimited. We hope you share our excitement!

  Au revoir, until next time!

  PS: our mission is to liberate web analytics; thank you for sharing the word about Piwik 2.0!

• Our latest improvement to QA: Screenshot Testing

  2 October 2013, by benaka — Development

  Introduction to QA in Piwik

  Like any piece of good software, Piwik comes with a comprehensive QA suite that includes unit and integration tests. The unit tests make sure core components of Piwik work properly. The integration tests make sure Piwik’s tracking and report aggregation and APIs work properly.

  To complete our QA suite, we’ve recently added a new type of tests: Screenshot tests, that we use to make sure Piwik’s controller and JavaScript code works properly.

  This blog post will explain how they work and describe our experiences setting them up; we hope to show you an example of innovative QA practices in an active open source project.

  Screenshot Tests

  As the name implies, our screenshot tests (1) first capture a screenshot of a URL, then (2) compare the result with an expected image. This lets us test the code in Piwik’s controllers and Piwik’s JavaScript simply by specifying a URL.

  Contrast this with conventional UI tests that test for page content changes. Such tests require writing large amounts of test code that, at most, check for changes in HTML. Our tests, on the otherhand, will be able to show regressions in CSS and JavaScript rendering logic with a bare minimum of testing code.

  Capturing Screenshots

  Screenshots are captured using a 3rd party tool. We tried several tools before settling on PhantomJS. PhantomJS executes a JavaScript file with an environment that allows it to create WebKit powered web views. When capturing a screenshot, we supply PhantomJS with a script that:

  • opens a web page view,
  • loads a URL,
  • waits for all AJAX requests to be completed,
  • waits for all images to be loaded
  • waits for all JavaScript to be run.

  Then it renders the completed page to an PNG file.

  • To see how we use PhantomJS see capture.js.
  • To see how we wait for AJAX requests to complete and images to load see override.js.

  Comparing Screenshots

  Once a screenshot is generated we test for UI regressions by comparing it with an expected image. There is no sort of fuzzy matching involved. We just check that the images consist of the same bytes.

  If a screenshot test fails we use ImageMagick’s compare command line tool to generate an image diff:

  

  In this example above, there was a change that caused the Search box to be hidden in the datatable. This resulted in the whole Data table report being shifted up a few pixels. The differences are visible in red color which gives rapid feedback to the developers what has changed in the last commit.

  Screenshot Tests on Travis

  We experienced trouble generating identical screenshots on different machines, so our tests were not initially automated by Travis. Once we surpassed this hurdle, we created a new github repo to store our UI tests and screenshots and then enabled the travis build for it. We also made sure that every time a commit is pushed to the Piwik repo, our travis build will push a commit to the UI test repo to run the UI tests.

  We decided to create a new repository so the main repository wouldn’t be burdened with the large screenshot files (which git would not handle very well). We also made sure the travis build would upload all the generated screenshots to a server so debugging failures would be easier.

  Problems we experienced

  Getting generated screenshots to render identically on separate machines was quite a challenge. It took months to figure out how to get it right. Here’s what we learned:

  Fonts will render identically on different machines, but different machines can pick the wrong fonts. When we first tried getting these tests to run on Travis, we noticed small differences in the way fonts were rendered on different machines. We thought this was an insurmountable problem that would occur due to the libraries installed on these machines. It turns out, the machines were just picking the wrong fonts. After installing certain fonts during our Travis build, everything started working.

  Different versions of GD can generate slightly different images. GD is used in Piwik to, among other things, generate sparkline images. Different versions of GD will result in slightly different images. They look the same to the naked eye, but some pixels will have slightly different colors. This is, unfortunately, a problem we couldn’t solve. We couldn’t make sure that everyone who runs the tests uses the same version of GD, so instead we disabled sparklines for UI testing.

  What we learned about existing screenshot capturing tools

  We tried several screenshot capturing tools before finding one that would work adequately. Here’s what we learned about them:

  • CutyCapt This is the first screenshot capturing tool we tried. CutyCapt is a C++ program that uses QtWebKit to load and take a screenshot of a page. It can’t be used to capture multiple screenshots in one run and it can’t be used to wait for all AJAX/Images/JavaScript to complete/load (at least not currently).

  • PhantomJS This is the solution we eventually chose. PhantomJS is a headless scriptable browser that currently uses WebKit as its rendering engine.

    For the most part, PhantomJS is the best solution we found. It reliably renders screenshots, allows JavaScript to be injected into pages it loads, and since it essentially just runs JavaScript code that you provide, it can be made to do whatever you want.

  • SlimerJS SlimerJS is a clone of PhantomJS that uses Gecko as the rendering engine. It is meant to function similarly to PhantomJS. Unfortunately, due to some limitations hard-coded in Mozilla’s software, we couldn’t use it.

    For one, SlimerJS is not headless. There is, apparently, no way to do that when embedding Mozilla. You can, however, run it through xvfb, however the fact that it has to create a window means some odd things can happen. When using SlimerJS, we would sometimes end up with images where tooltips would display as if the mouse was hovering over an element. This inconsistency meant we couldn’t use it for our tests.

  One tool we didn’t try was Selenium Webdriver. Although Selenium is traditionally used to create tests that check for HTML content, it can be used to generate screenshots. (Note: PhantomJS supports using a remote WebDriver.)

  Our Future Plans for Screenshot Testing

  At the moment we render a couple dozen screenshots. We test how our PHP code, JavaScript code and CSS makes Piwik’s UI look, but we don’t test how it behaves. This is our next step.

  We want to create Screenshot Unit Tests for each UI control Piwik uses (for example, the Data Table View or the Site Selector). These tests would use the Widgetize plugin to load a control by itself, then execute JavaScript that simulates events and user behavior, and finally take a screenshot. This way we can test how our code handles clicks and hovers and all sorts of other behavior.

  Screenshots Tests will make Piwik more stable and keep us agile and able to release early and often. Thank you for your support & Spreading the word about Piwik!

• Hello world!

  13 September 2013, by clearcode — Uncategorized

  Welcome to Piwik Sites. This is your first post. Edit or delete it, then start blogging!