Should we start collecting software usage data?

Without the ability to collect user metrics from Slicer that would let me gain knowledge about the user base or gather use-cases data to try to seek funding in the future, I’ve chosen to stop developing BoneReconstructionPlanner (apart from addressing bugs).

Therefore, I’ll be using my next weekends with free-time to make an Slicer analytics (i.e. telemetry) extension.

@jcfr suggested to learn from:

And to try to implement with GoatCounter:
https://www.goatcounter.com/

I’ll post about my advances here

Mauro

1 Like

Thanks for the information and considering working on analytics features. The highest priority is to implement the ability to opt in to telemetry. We cannot have extensions collecting data without explicit user consent.

The opt in could be obtained by displaying a widget in the Welcome dialog (and also make it possible to change the choice later in the application settings, and maybe also allow an extension to make a consent dialog appear). The widget could contain some text that explains why enabling telemetry is important and display some options (probably we could start with a single checkbox to globally enable/disable telemetry, but later there may be more refined options). Since the widget could appear at different places, it would be nice to implement it as a reusable widget (similarly to the qSlicerSettingsUserInformationPanel).

@mau_igna_06 if you want to have telemetry in your module then please work on this consent dialog first. If you are unsure about the implementation then you can start with drafting a text that explain the user why he should consent.

2 Likes

3 posts were split to a new topic: Using mailing list for communicating with users

Example: renderdoc tool

After installing renderdoc, I was presented with the following popup:

The analytics infrastructure description along with the collected data are also publicly reported on the associated analytics page.

References:

4 Likes

Nice example @jcfr

Thanks for sharing

Although I’m not so sure users would go through such a verbose dialog (maybe add hrefs instead?)

I don’t know what others think :slight_smile:

Hi

I was able to do a “Hello World” example:

Feedback is welcomed

GoatCounter looks like a nice project, but I would prefer something more established and longer-term solution, such as an OpenTelemetry server hosted by Kitware. Probably something simpler would be sufficient, too, but it should be open-source, using open standards, and preferably with an option of either self-hosting or inexpensive hosting.

@jcfr Does Kitware use (or plan to use) telemetry for its projects?

2 Likes

With the help of ChatGPT (GPT-3.5)

My humble prompting trial

What’s a dark pattern? What’s a deceptive pattern?

Dark patterns and deceptive patterns are both terms used to describe unethical or manipulative practices used in user interface design and online marketing to trick or deceive users into taking actions they might not want to take. These practices are designed to benefit the business or website at the expense of the user’s best interests. While they may be effective in the short term, they can harm the user experience and damage a company’s reputation in the long run.

Dark Patterns:
Dark patterns refer to user interface design choices that intentionally mislead or deceive users. These patterns are typically used to trick users into making decisions or taking actions that they might not otherwise choose if they had full and clear information. Dark patterns can take various forms, including:
    Misdirection: Steering users towards a particular action by making it more prominent or attractive while downplaying other options.
    Hidden Costs: Concealing additional charges or fees until late in the transaction process.
    Forced Continuity: Making it difficult for users to cancel subscriptions or memberships.
    Trick Questions: Using confusing language or design to manipulate user choices.
    Urgency and Scarcity: Creating a false sense of urgency or scarcity to pressure users into taking immediate action.

Deceptive Patterns:
Deceptive patterns refer to a broader range of practices that go beyond just user interface design and include any deceptive tactics or strategies employed by a company or marketer to manipulate or mislead consumers. This can include not only design elements but also misleading advertisements, false claims, or any practice that involves dishonesty. Deceptive patterns can take many forms, such as:
    False Advertising: Making false or misleading claims about a product or service.
    Bait-and-Switch: Advertising one product but substituting it with a different, often lower-quality, product.
    Phishing: Attempting to trick users into revealing personal information by posing as a legitimate entity.
    Fake Reviews: Posting fabricated positive reviews to artificially boost a product's reputation.
    Disguised Ads: Making advertisements look like regular content to deceive users.

Both dark and deceptive patterns are considered unethical and can damage a company’s reputation when discovered. Many countries have laws and regulations in place to protect consumers from such practices, and companies that engage in these behaviors may face legal consequences or fines. Ethical design and transparent marketing practices are generally recommended to build trust and maintain a positive relationship with customers.

What’s Slicer biomedical research software?

3D Slicer is not specific to biomedical research but is widely used in the field due to its versatility and capabilities. It’s an open-source software platform used for medical image analysis, visualization, and 3D image reconstruction. Biomedical researchers, clinicians, and engineers often use 3D Slicer for a variety of tasks, including but not limited to:

  1. Medical Imaging Research: Researchers can use 3D Slicer to process and analyze medical images from various modalities such as MRI, CT, ultrasound, and more. It allows for segmentation (identifying and outlining specific structures or regions of interest), registration (aligning images from different scans or time points), and advanced visualization.
  2. Surgical Planning: Surgeons use 3D Slicer to plan and simulate procedures. They can visualize patient-specific anatomical structures in 3D, helping them make more informed decisions before surgeries.
  3. Image-Guided Interventions: 3D Slicer supports real-time navigation during medical procedures. It can register preoperative images with the patient’s current anatomy, aiding in guidance and precision during surgeries.
  4. Radiomics and Radiogenomics: Researchers use 3D Slicer to extract quantitative features from medical images, allowing for the analysis of large datasets to identify correlations between image-based metrics and clinical outcomes.
  5. Diffusion Imaging: It’s widely used in the analysis of diffusion-weighted imaging (DWI) and diffusion tensor imaging (DTI) data, which can provide insights into the microstructure of tissues in the brain and other organs.
  6. Custom Module Development: 3D Slicer is highly extensible, and researchers can create custom modules and extensions to tailor it to their specific research needs.
  7. Education and Training: It’s used in medical education and training, providing a platform for students and professionals to learn and practice medical image analysis and 3D visualization.

3D Slicer is a powerful and flexible tool, and its open-source nature means that it has a large and active user community, which contributes to its development and supports its use in various applications, including biomedical research. It is often used in conjunction with other software and tools to enable a wide range of medical image analysis and research applications.

Write a non dark-pattern, non deceptive-pattern telemetry collection statement for 3D Slicer

Certainly, here’s a straightforward and transparent telemetry collection statement for 3D Slicer:

"At 3D Slicer, we are committed to improving our software and providing a better user experience. To achieve this, we collect limited telemetry data to help us understand how our software is being used and identify areas for enhancement. We value your privacy and are dedicated to being transparent about our data collection practices.

What data is collected:

  • Usage statistics: We collect information about the features and tools used within 3D Slicer, helping us understand which functionalities are most valuable to our users.
  • Error reports: In the event of a software error or crash, we may collect diagnostic information to diagnose and address issues more effectively.
  • Basic system information: We may gather data about the operating system and hardware configuration to optimize compatibility and performance.

How we use the data: The collected data is exclusively used for improving 3D Slicer. It helps us identify and prioritize software enhancements, address technical issues promptly, and enhance the user experience.

Data Protection: Your privacy is important to us. The data we collect is anonymized and aggregated to ensure that it cannot be used to identify individual users. We do not collect any personal or sensitive information.

Opting out: Participation in telemetry data collection is entirely optional. You can easily opt out of telemetry data sharing during the installation process or within the software’s settings.

We are dedicated to maintaining transparency and respecting your privacy. Your feedback and usage data play a crucial role in our efforts to make 3D Slicer a more valuable tool for the biomedical research community. Thank you for your support."

This telemetry collection statement is designed to inform users about the purpose of data collection, how it’s used, and how they can opt out if they wish, without resorting to dark patterns or deceptive practices.


As discussed on 2023.10.31 Weekly Meeting, a telemetry collection statement is proposed.

What do you think? :slight_smile:

NOTE: This OSS example of implementation was also mentioned on that meeting.

1 Like

That’s actually pretty good. I’d take out the part about error reports though. I don’t want to collect those automatically since there’s a good chance they would have PII (personally identifying information) or even PHI (protected health information). Better if people only share those when they are sure they are safe.

2 Likes

That seems like a good start. I agree with @pieper that the error reports part should be dropped. In addition, I think the sentence “The collected data is exclusively used for improving 3D Slicer” should be thought about carefully. Might this data also be used to try to obtain funding, for example?

More generally, once the exact mechanism and scope of data collection is decided on, I think it would be better to include some more specifics. For example, perhaps the module names and number of times opened is being collected? If the variety is too great to include in a short statement of consent, there should be a link to a page which goes in depth about what is collected and how.

1 Like

I agree that the Chatgpt output is pretty good, but I would rather use a large open-source software organization’s policy as a basis. For example, the KDE telemetry policy is quite good. I could accept it for Slicer as is:

https://community.kde.org/Policies/Telemetry_Policy

3 Likes

Just to have in mind, this appears in Qt installation:

telemetry-question : Contribute to Qt Development : Help us improve Qt and Qt Creator by allowing tracking of pseudonymous usage data in Qt Creator. The tracking can be disabled at any time. Read the Qt Company data collection privacy statement Legal | Terms & Conditions - Qt. Yes|No

Hi Rudolf,

Have you gotten any other insights on past months?

What have been the most direct benefits of setting up those counters?

First of all, it is interesting and motivating how many runs (>23000 since May 2023, 88371 uses per month, 3156 per week and 450 per day) were done.

People almost double use the segmenter over the analyzer.
I can see that most people use AI techniques for the lungs now and that TotalSegmentator use is not as prominent as I thought, maybe because lungmask is faster and less complicated to install (inbuild).
Few people use airway seg and vessel segmentation, probably because it is just not good enough yet or not needed in their studies. All in all interesting information for development. So here is room for development. I stored IP in the very beginning but now the counting is IP-less. Still, I could make IP-less segmentation statistics over weeks, months, and years.starting in September 2023. There was not one complaint concerning the counters yet. But they are really non-personal technical counters.

2 Likes

Would it make sense to hash the IPs and store only the hashes to see how many runs were from the same user? Then there would be no personal information stored but the benefit preserved.

2 Likes

Such fingerprinting of users based on IP (and potentially based on additional parameters of the computer) requires prior consent and probably other mechanisms as well (allow users to view/erase data that is maintained about them?). I would feel more comfortable adopting telemetry policy (and perhaps tooling) from a large, well-established project (e.g., the KDE) than handcrafting our own.

1 Like

Rudolf, could you get an average of number of uses per download using the download stats?

This link does not work in my chrome - but I have no individual user indications that could indicate the number of program runs per download … sry

It’s a JSON file that may have a size of 500MB since it has a long history of Slicer versions and Slicer extension download count, that maybe the reason it appears to not render anything

Otherwise you could install DeveloperToolsForExtensions on latest Slicer Stable release and check “Extension Download Statistics” module, write “LungCTAnalyzer” and click “Get download statistics”.

With those numbers you can sum the total number of installs of your extension, and the total extension use cases (or of a particular feature). Then divide the latter by the former and you get a “use cases per extension download” number

I would do it if I had the data myself, hope this is interesting for you as well

re: Visual Studio

An here is what is displayed in the context of the NVIDIA Nsight / Visual Studio integration:

image

2 Likes