I tried to download the extension download statistics using the stable, and got the error below. This used to work, so I assume this is related to change in extension server.
Going forward, what is the plan on obtaining extension stats?
Switch to module: "ExtensionStats"
Traceback (most recent call last):
File "C:/Users/murat/AppData/Local/NA-MIC/Slicer 4.11.20210226/NA-MIC/Extensions-29738/DeveloperToolsForExtensions/lib/Slicer-4.11/qt-scripted-modules/ExtensionStats.py", line 579, in getExtensionSlicerRevisionAndDownloads
item_rev_downloads[itemid] = [self.getItemById(url, itemid)['download'], slicerrevision]
File "C:/Users/murat/AppData/Local/NA-MIC/Slicer 4.11.20210226/NA-MIC/Extensions-29738/DeveloperToolsForExtensions/lib/Slicer-4.11/qt-scripted-modules/ExtensionStats.py", line 553, in getItemById
return self._call_midas_url(url, data)
File "C:/Users/murat/AppData/Local/NA-MIC/Slicer 4.11.20210226/NA-MIC/Extensions-29738/DeveloperToolsForExtensions/lib/Slicer-4.11/qt-scripted-modules/ExtensionStats.py", line 515, in _call_midas_url
response_data = response_dict['data']
KeyError: 'data'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/murat/AppData/Local/NA-MIC/Slicer 4.11.20210226/NA-MIC/Extensions-29738/DeveloperToolsForExtensions/lib/Slicer-4.11/qt-scripted-modules/ExtensionStats.py", line 153, in onApplyButton
release_downloads = self.logic.getExtensionDownloadStats(extensionName)
File "C:/Users/murat/AppData/Local/NA-MIC/Slicer 4.11.20210226/NA-MIC/Extensions-29738/DeveloperToolsForExtensions/lib/Slicer-4.11/qt-scripted-modules/ExtensionStats.py", line 645, in getExtensionDownloadStats
rev_downloads = self.getExtensionSlicerRevisionAndDownloads(url, extensionName)
File "C:/Users/murat/AppData/Local/NA-MIC/Slicer 4.11.20210226/NA-MIC/Extensions-29738/DeveloperToolsForExtensions/lib/Slicer-4.11/qt-scripted-modules/ExtensionStats.py", line 584, in getExtensionSlicerRevisionAndDownloads
self.setStatus("Retrieving package info {0}/{1} for extension {2} - Error: {3} - ".format(idx+1, len(all_itemids), extensionName, str(e)))
UnboundLocalError: local variable 'e' referenced before assignment
You can get download counts in a single json file from the new extensions server: SPKC
We also plan to develop a tool to get more detailed statistics from web server logs, which will work retrospectively, so no download statistics information will be lost.
Thanks JC. I parsed the output from the link you provided and the numbers are very different from what I used to get out of Extension Download Statistics tool, even though it includes historical revisions (not just the 4.13 revisions) for SlicerMorph.
This is important for us, and I suspect other grant supported projects. Would it be possible to reconcile this with the older statistics?
I’ve updated the ExtensionStats module in SlicerDeveloperToolsForExtensions:
it retrieves download counts from Girder
it loads Midas statistics from a CSV file and adds it to the numbers provided by Girder
The Midas statistics is collected in May 2021, therefore we miss about 3 months of downloads. If @jcfr can retrieve the latest snapshot from the old server and updates the Midas csv file then we should have accurate download counts.
By the way, getting full statistics on all extensions is much faster now. Using the old server it took 6-8 hours, now it takes about 2 seconds. The main difference is that the new server stores the statistics as a single file that can be retrieved using one query, while on Midas we had to submit thousands of queries (one for each revision of each extension).
I works great and really fast, thank you Andras (and JC as well).
There is a minor issue that table copy doesn’t function
Traceback (most recent call last):
File "C:/Users/murat/AppData/Local/NA-MIC/Slicer 4.13.0-2021-10-26/NA-MIC/Extensions-30334/DeveloperToolsForExtensions/lib/Slicer-4.13/qt-scripted-modules/ExtensionStats.py", line 144, in copyTableToClipboard
tableText += table.GetColumn(columnIndex).GetValue(rowIndex)
TypeError: must be str, not int