This will help to show if a plugin is active and give some useful realistic statistical feedback.
That’s a good idea. We have been tracking extension installations internally so that extension authors and quote the figures in grant applications and progress reports, but yes, it would be even better if the figures were available to users as well. @jcfr has been completely revamping the extension server infrastructure so let’s finish that up but once that is done it should be easier for someone to contribute this upgrade.
You can use DeveloperToolsForExtensions extension to query download counts from the current extension server.
Yes you can use DeveloperToolsForExtensions to generate download counts. Here is what the results reveal for SlicerCochlea for example. I removed earlier versions (before 4.8.0) where there were no downloads since it was not available then.
This only provides details about downloads which is not necessarily unique users. People could uninstall/reinstall, download using different machines (work and home), and download using different versions of Slicer.
I believe @brhoom was suggesting these numbers be included in the Extension Manager dialog, which should be quite doable.
Is the motivation to put them in the Extensions Manager for Developer insight? Or user insight?
Probably convenience for both, to build an awareness of what people are finding interesting and useful. Like pypi and npm where they provide download stats and you get a sense of whether a package is new or well established and popular.
Thank you all for your replies.
Both!
Are these numbers realistic? if the cdash testing process is included then these numbers are not realistic.
Yes, the numbers are correct. CDash tests do not download any extensions from the extension server (all the extensions upload and download tests use a local mock server).
I am not sure if they are on the roadmap of the new extension mechanism, but I would like to remind about these two requests (second should have been renamed more like “reminder for available updates”).
@brhoom Are you surprised by the results? Are they higher than you would have expected?
somehow! it seems the reason is clear:
Download statistics show how much people are interested in trying your extension. It shows how well your advertise your extension with its name, icon, description, documentation, posts on discourse, social media, etc.
We don’t have direct usage statistics of Slicer and extensions. It would require implementing data collection, prompt users to opt in, set up the storage backend, and a frontend to access the results. With the new extension manager and Slicer download page, we are one step closer to have a backend that could store the information,but there is still a lot to do.
I should add that, while download stats might have the confounding issues @jamesobutler mentioned, they cannot be happening all the time for all users. So while we may not knowing the exact user count, a continuous download trend indicates a sustained core group of users/usage, which very important for any kind of funding.
If I create a bot that makes web requests to download an extension some amount of times on a schedule, I could in theory greatly influence a result to make it appear that an extension is more popular than it actually is.
You sure can. But an obscure extension with off the roof download statistics will not get you the funding, at least not from major funding institutions. While the download stats are important, it is not the only thing that matters.
But i get your point, it is hard to document real usage. More real time tracking is not reliable solution either. For example my institutions network policy blocks all outgoing ports. So you might have slicer working on every single hospital computer, but wouldn’t get a single hit.
To move this discussion beyond speculation and hypotheses, have a look at the all-time download statistics of all extensions in this excel sheet (compiled in May 2021).
There are some very surprising numbers. The absolute numbers seem to be heavily influenced by the extension’s name and icon, maybe also by description and documentation, because very important and widely used and advertised extensions fall behind compared to some much “better” ones.
However, even though there are big outliers and some surprising numbers, the magnitude of the download count is informative. Also, download trends for an extension is useful information, too.
This is very useful and definitely interesting. SlicerXNAT hasn’t been touched in quite a few years, and was definitely surprising for me to see as one of the top total downloads.
SlicerMorph bundles a number of other extensions, so they might be increasing the their total count. These are:
IGT, SegmentEditorExtraEffects, SurfaceWrapSolidy, DCM2NIIX, auto3DGM, sandbox and rawimageguess.
Interesting that your stats from May 2021 show SlicerCochlea at 260 downloads for Slicer 4.11.20210226, while the stats I generated the other day showed that version as having a little over 2000 downloads. That’s quite the increase from May to July (now).
@brhoom Does this match behavior that you are aware of? Has your increase of this extension gone up since May?
The over 5000 count for Slicer 4.10.2 matches and so does the over 3000 count for post-4.10.2.
Very interesting. In extensions: Would it be allowed (theoretically) to implement a “send usage statistics” checkbox or even include parameter feedback (f.e. thresholds used, training data etc) back to a central server?
