Support for keyframe based animation

This reminds me of the following project put together by @agirault and @Andinet_Enquobahrie

Edit: Updated link to video publicly available.

Edit 2: More details are available on this page: https://www.slicer.org/wiki/Documentation/Labs/FlyThroughNavigation

2 Likes

@jcfr @muratmaga @hherhold I think we should open a separate thread for the keyframe animation so we can focus on answering @Dave_DeMarle question regarding photorealistic rendering, like @lassoan did here.

1 Like

iVSPlan looks cool :+1:

@muratmaga and I have been talking for a while about the possibility for keyframing support and I had in mind something kind of like what’s shown in the video, but I was also thinking of making it handle potentially any node, not just camera nodes. For example clipping planes and volume rendering transfer functions.

I was thinking of doing something based on the Sequences infrastructure, since that already has multi-node animation abilities. But what’s missing is the way to interpolate various node types between keyframes. I don’t have a specific implementation plan at this point, but I hope to try some experiments and report back with ideas about how to make something with a clean and extensible architecture.

1 Like

I had in mind something kind of like what’s shown in the video, but I was also thinking of making it handle potentially any node, not just camera nodes

I think that would make a lot of sense. Like you brought up, the interpolation is then the question (it was easy to use vtkSplines for the camera position and orientation). My understanding is that the main use case would be for the camera, so maybe it’s worth not over-engineering a fit-all solution but start building around the camera?

I agree about not over-engineering - but still, I don’t think it’s enough to do just the camera. I’d thought about adding a virtual method like vtkMRMLDisplayableNode::Interpolate() so that any displayable could expose it’s own interpolation method but I don’t like that for a number of reasons (mainly because it would make MRML depend on Sequences but also because some animations should set up relations between objects - like the camera’s lookat is tied to another object’s location).

Instead I think the place to start is with some custom scripts that handle some specific animation styles, basically as an extension of the current ScreenCapture module.

For reference, here’s a video that @muratmaga suggested as an inspiration:

Here the things we would need to keyframe are:

  • Camera parameters (or object transform parameters)
  • volume rendering lookup table
  • cutting plane or cropping ROI

These are all fairly easy values to interpolate.

1 Like

We plan to merge Sequences extension features into Slicer core, so interpolation of any node will be available. If interpolation is implemented for a node then Sequence browser can use it quite easily. The main difficulty is to implement a simple yet powerful user interface. What is in Paraview or MITK seems to have taken a lot of work and still very hard to use.

If we need to change only one parameter at a time then we can easily add those as animation modes to screen capture module.

Camera animation is a special case, as you may not want to specify it by saving current camera positions as key frames, but instead by specifying a curve and maybe a look-at point. This can be managed in the endoscopy module. We should probably rename the module to something that reflects that it is a general “curve to transform” module (Camera path might be good, or maybe something even more general). We should update it to use the new curve widget, add option to export to sequence, etc.

What’s the timeframe for merging Sequences to the core?

Sequences should be in its final place in Slicer5. Since Slicer5 is scheduled for next RSNA and we need time for stabilization and testing, Sequences should be moved by the end of summer.

1 Like

I’m obviously biased, but the interface I created for IVSPlan (linked by @jcfr in the top comment) was really intuitive and easy to use.

1 Like

@agirault agreed, it looks nice. Is the code available for reuse?

Now yes!

With the help of @jcfr, we’ve pulled out the code for that module, added a LICENSE and README files, and published it to GitHub:

JC described it in the README of the repository above, but it depends on an old fork of Slicer with some small changes (which also includes a “flying” as an option that could be toggled in the UI, which worked pretty well):

Keep in mind that I wrote this as an intern who discovered Slicer 4 years ago :wink:

Also, here was the full demo I had created (14min+):

Hth

Just noting that I did not intend to say “my interface is nice” out of the blue: I had received this email below this morning from Yoan via 3D Slicer Forum, to which I answered by email:

However, when looking in Discourse, it looks like there is no Yoan involved in the discussion, and my comment appears as an answer to JoseDamasceno (top right) who does not seem involved here either.

In the end, it looks like what I received today by email was a part of @lassoan’s comment from 11 days ago:

… discourse bug? :slight_smile:

I’ve seen in the past spam accounts that will copy and paste another user’s response. Likely to gain higher discourse forum privileges by making a post. I’m guessing the post/account was later deleted.

Yes, it was a spammer. But it accidentally did something useful by reviving the topic. :smile:

3 Likes

Then sorry for the noise! Still weird that it showed as “Yohan” in gmail, and “JoseDamasceno” in discourse… Also, I got an email for your answer @jamesobutler, but not yours @lassoan :man_shrugging:

Hi @agirault - thanks for sharing the example :+1: Just to confirm, this is not expected to compile with current slicer and you aren’t planning to work on it, right? I ask because with @muratmaga and @jcfr we’re trying to get a ballpark idea of the effort involved in getting the keyframing features in place.

Not expected to build against current Slicer.

I anticipate updating code and backporting changes to Slicer should be straightforward.

We will account for this in project with @muratmaga.

Correct @pieper, this worked with Slicer 4.4, and I don’t have funding to update it to latest Slicer. Are you looking into the effort needed to implement this on top of Sequence, or to simply update it as-is to latest Slicer? I presume the later shouldn’t be much work, there weren’t many functionalities missing in Slicer.

Thanks @jcfr and @agirault - yes, at this point we’re still trying to scope out all the options so we know what we have to work with and what to build off of. Yes, I’m thinking that we would want to build on top of Sequences but expose a very high level UI to create the specific types of animations described above (not just camera, but also volume rendering transfer functions and cropping) and a simple timeline editor.

I think the way this takes whatever camera positions you have and makes key frames from them for you is important. That is you move the camera the same way you normally do and then when you hit Add Keyframe the details are filled in for you. You have to hunt a bit to find the similar interface in ParaView’s animation view and most people have to confront the awkward “Enter camera positions manually” interface instead.

In both ParaView and Slicer we should make it so that more/all of the controls (not just the camera) that vary over the animation are set up in this user friendly way and then manually editable after the fact.