Should we start collecting software usage data?

Development
81

I want to do a pull request to Slicer/SlicerTelemetry, the repository is empty and i cant fork it, i need to fork it to do the pull request

82

I’ve added a few files. You should be able to fork the repository now.

1 Like
83

I would be nice if this extension arrived on the new stable release :slight_smile:

84

Hello everyone,

I’d like to share my progress on the development of the telemetry extension for Slicer 3D.
Here is what you need to know:

For End Users:

When there is data to be uploaded to the server this dialog will show up:

SendToServerPopUp
SendToServerPopUp466×113 4.44 KB

Including the raw data

seeRawData
seeRawData469×228 6.73 KB

And a visualization of the data collected

dataVisualization
dataVisualization967×680 12.4 KB

The data used in the following images was generated randomly to test the frontend.

mapAndTimeLine
mapAndTimeLine1108×863 80.5 KB

ComponentAndEvent
ComponentAndEvent1079×759 47.1 KB

You can also select which extensions are allowed to collect information on the Telemetry Extension

selectExtensions
selectExtensions476×211 4.55 KB

The default configuration is asked on slicer start just once.

installedExtensionsPermission
installedExtensionsPermission298×125 3.58 KB

For Slicer Developers:

Telemetry Extension Workflow

The Telemetry Extension is designed to log, organize, and upload usage data. Here’s a breakdown of its components and functionality:

  1. Logging Events in Extensions:
  • To log an event from an extension, use the following code snippet:

slicer.app.logUsageEvent(“Component”, “Event”).

  • This emits a usageEventLogged signal.
  • The signal is connected to the onUsageEventLogged function, which processes the event.
  1. Event Logging with onUsageEventLogged(component, event):
  • This method saves incoming events into a .csv file.
  • It verifies whether the event’s source component belongs to one of three categories:
    • enabledExtensions: Telemetry is explicitly enabled.
    • disabledExtensions: Telemetry is explicitly disabled.
    • defaultExtensions: Telemetry depends on a default setting (e.g., whether telemetry is enabled by default for new installations).
  • Only events from components with enabled telemetry (or default-enabled settings) are saved.
  • The method also:
    • Adds timestamps to each event.
    • Groups events by occurrence per day for aggregation and easier analysis.

Extension Installation:

  • When an extension is installed, it is automatically added to the defaultExtensions category.

Data Upload Prompt:

  • A prompt asking for permission to upload data appears only if the following conditions are met:
    • More than 7 days have passed since the last upload.
    • There is data in the .csv file waiting to be uploaded.
  • If the user selects “Do not show again,” the preference is saved, and the prompt will not appear again.

Uploading and Request Handling:

  • Functions like weeklyUsageUpload and handleQtReply use QNetworkAccessManager for asynchronous handling of HTTP requests and responses. This avoids blocking the main application, unlike synchronous libraries such as requests.

Server-Side Implementation

The server is implemented using Flask and performs the following tasks:

  1. Geolocation Mapping:
  • The get_location_from_ip function:
    • Extracts the country, country code, and city based on the IP address in a POST request.
    • Uses the GeoIP2 database for geolocation mapping.
    • This data is appended to the event details (e.g., component, event, date, and aggregated times).
  1. Data Conversion:
  • The convert_alpha2_to_alpha3 function:
    • Converts two-letter country codes (ISO Alpha-2) to three-letter codes (ISO Alpha-3) for compatibility with the frontend.
  1. Data Handling:
  • Event data from POST requests is appended to a data.csv file located in the static folder.
  • This .csv file is used by the frontend for generating statistics.
  1. Frontend Hosting:
  • The Flask application serves the frontend content, also stored in the static folder.

Nginx Server

  • An Nginx server is used as a reverse proxy for the Flask application, offering several advantages:
    • Handles incoming traffic and forwards it to the Flask application.
    • Efficiently manages a high volume of concurrent connections.
    • Provides faster response times and a smoother user experience compared to Flask handling requests directly.

Frontend Visualization

The frontend provides a user-friendly interface for visualizing telemetry data:

  1. Charts with stats.js:
  • Uses D3.js and Crossfilter libraries to:
    • Create interactive charts.
    • Establish linkages between dimensions, allowing charts to filter and update based on user interactions.
  1. World Map:
  • The map, defined in world.geojson:
    • Is an SVG file containing country names and codes.
    • Links to the event data for geographic visualization.

Thank you for taking the time to read about my progress. I welcome any feedback or suggestions from the community as I continue to develop this Slicer 3D extension!

Best regards,
Bernardo Dominguez

3 Likes