Animation mixing/blending

Ggeowal-wondeluxe · 2022年12月5日

I know there have been a number of posts on blending animations, but I haven't been able to find a solution to my issue. Currently I'm trying to blend between a walking and running animation using TrackEntry.alpha e.g.

TrackEntry track1 = skeletonAnimation.state.SetAnimation(0, walkReferenceAsset.Animation, loop);
TrackEntry track2 = skeletonAnimation.state.SetAnimation(1, runReferenceAsset.Animation, loop);
track2.Alpha = 0.5f;

However this method appears to rely on the two animations being the same duration, obviously a walk cycle and a run cycle are going to be different lengths. I've tried changing the MixBlend value on TrackEntry as well, but changing from the default value of Replace only makes things worse.

I've found that I'm able to get a nice blend using SkeletonMecanim and Unity's built-in Blend Tree, however if I use Mecanim I can't use Spine's IK constraints. I require both for my project.

It looks to me like Spine isn't normalising the time between the two animations when it calculates the blend, but I'm not sure. Is there a way I can manually apply the blend?

I've also done a screen recording to demonstrate the issue. On the left is a Mecanim Animator using a blend tree, on the right is a SkeletonAnimation using the TrackEntry.Alpha property. It's easy to see how the SkeletonAnimation breaks down pretty much immediately compared to the Mecanim blend tree.

Ggeowal-wondeluxe · 2022年12月5日

Nate · 2022年12月5日

Alpha gives you a pose between the pose from track1 and the pose from track2. If your animations are out of phase such that one wants the leg in front and the other wants the leg behind, then 50% of the way between those poses is the leg nearly straight up and down. This is the sort of stuttering you see.

When the animations have different durations, their phase will vary over time. However, this could also happen when the animations are the same durations, depending on when the legs are animated in front/behind and also when the animations were started.

I think you could solve this by adjusting TrackEntry timeScale so the animations match. Maybe start by setting the run animation so it plays at the speed of the walk animation. Then increase the run speed based on the TrackEntry alpha, between walk speed (0 alpha) and full run speed (100% alpha). The walk speed would also change, from normal walk speed (0 alpha) to full run speed (100% alpha).

This way the animations always match as you transition from walk to run. The animations don't have to have the same duration, but the movements do need to match when run is played at the speed that matches walk.

You may find this a bit weird because it's different from the more common weighted animation blending. There are some advantages, eg it allows layering animations. You can do weighted animation blending if you prefer. For that you use additive animation blending by setting TrackEntry mixBlend to MixBlend add. It would work like this:

First you need to reset everything from last frame back to some reasonable value. This can be done with Skeleton setToSetupPose or by applying an animation on a lower track than the tracks using MixBlend add.
Next you play walk on a track and run on a different track. Set MixBlend add on both.
Set TrackEntry alpha to the weights you want so they sum to 1.

For example, 0.25 walk and 0.75 run. This adds 25% of the walk pose and 75% of the run pose to the pose established by step 1. By that I mean 25% of the values that are the distance from the setup pose to the pose from the walk animation. You can mix any number of animations this way, eg 0.3, 0.2, 0.5 sums to 1. You can also exceed the sum of 1, but will distort the poses.

If you don't do step 1, then pose values from last frame remain. You add new pose values to that each frame and in just a few frames your skeleton has exploded!

I expect weighted animation blending is what Mechanim is doing. I doubt Mechanim is scaling the animations, it's just harder to notice when they are out of phase. The result can be similar to what I described above, but isn't the same. You can still have the same out of phase problem, but it may be less noticeable.

Ggeowal-wondeluxe · 2022年12月5日

Thanks Nate.

I've implemented the time scale solution and it's working pretty well. I'm currently applying TrackEntry.TimeScale during the SkeletonAnimation.BeforeApply event, is this the best place to do it?

I haven't had any success implementing weighted blending as you've described. My results are even worse than what I started with. Should I play the first MixBlend.Add animation on a track above track 0 (as track 0 always uses MixBlend.First)?

My code is as follows, have I made some mistake in interpreting your outline?

[SerializeField]
[Range(0f, 1f)]
private float blend;

private void Start()
{
   walkTrack = skeletonAnimation.state.SetAnimation(1, walkAsset.Animation, true);
   walkTrack.MixBlend = MixBlend.Add;
   walkTrack.Alpha = 1f - blend;

   runTrack = skeletonAnimation.state.SetAnimation(2, runAsset.Animation, true);
   runTrack.MixBlend = MixBlend.Add;
   runTrack.Alpha = blend;

   skeletonAnimation.BeforeApply += ApplyBlending;
}

private void ApplyBlending(ISkeletonAnimation iSkeletonAnimation)
{
   skeletonAnimation.Skeleton.SetToSetupPose();
   walkTrack.Alpha = 1f - blend;
   runTrack.Alpha = blend;
}

Harald · 2022年12月5日

geowal-wondeluxe wrote
I've found that I'm able to get a nice blend using SkeletonMecanim and Unity's built-in Blend Tree, however if I use Mecanim I can't use Spine's IK constraints. I require both for my project.

For sake of completeness: why can't you use IK Constraints when using SkeletonMecanim? In general you can create your animations as usual using IK constraints, and then access and set the ikConstraint.Mix values as desired from your code. Or did you mean something else?

Nate · 2022年12月5日

BeforeApply should be a fine place. The code looks OK. When blend is 0 do you get walk correctly, and when 1 do you get run correctly?

We've confirmed Mecanim does time scaling like what I've described above, so you'd need to do the same even using additive (weighted) animation blending.

Ggeowal-wondeluxe · 2022年12月8日

For sake of completeness: why can't you use IK Constraints when using SkeletonMecanim? In general you can create your animations as usual using IK constraints, and then access and set the ikConstraint.Mix values as desired from your code. Or did you mean something else?

I'm using IK and transform targets in Unity to modify the animations at run-time. To position/rotate the feet correctly on slopes for example. I was under the impression this wasn't possible using Mecanim.

The code looks OK. When blend is 0 do you get walk correctly, and when 1 do you get run correctly?

The animation runs correctly at 0 and 1, except for when blend is 0, the sprite swapping isn't what I would expect. The hands use a relaxed/open pose for the walk animation, then swap to a fist for the run animation. I would expect to see the relaxed/open sprite when blend is 0, instead it's the fist. This is what I get when using the time scale method as well.

We've confirmed Mecanim does time scaling like what I've described above, so you'd need to do the same even using additive (weighted) animation blending.

Makes sense to me.

So just following up on this.

I've implemented a pretty tidy AnimationBlend class that works in a similar way to 1D Blend Trees in Unity. I can supply any number of TrackEntries, each with a Threshold value, and interpolate between them (adjusting the Alpha property of each TrackEntry). There's also an option to adjust the TimeScale so animations of different lengths stay in sync.

The only problem I'm dealing with now is managing how sprites are swapped during the animation. To me it looks like attachments on higher tracks are always preferenced over attachments on lower tracks, even if the TrackEntry on a higher track has Alpha set to 0. Is there any way to work around this?

I've found TrackEntry.AttachmentThreshold but that only applies when an ainimation is mixing out. A similar property to that but tied to Alpha would be really useful.

Nate · 2022年12月8日

geowal-wondeluxe wrote
I'm using IK and transform targets in Unity to modify the animations at run-time. To position/rotate the feet correctly on slopes for example. I was under the impression this wasn't possible using Mecanim.

SkeletonMecanim allows Mecanim to orchestrate applying Spine animations. The Spine animations are applied to a Spine skeleton using the timeline API. After the animations are applied, before the skeleton is drawn, you can make changes to the skeleton just like you can when using AnimationState instead of SkeletonMecanim. That includes positioning/rotating feet or other IK targets. The "baking" features in spine-unity are the only features that would prevent you from doing such dynamic modifications to the skeleton.

Sorry if this information is a bit late! Using the AnimationState API (eg via the SkeletonAnimation component) has some advantages, such as not needing to set additional keys for mixing to work in some cases.

I understand your need to control attachment timelines based on alpha rather than last applied (highest track). TrackEntry attachmentThreshold seems very close to what you need, but as you found is only affected by mix duration (mixTime / mixDuration), not TrackEntry alpha. I think it could make sense to have the same attachmentThreshold property apply to both. I'll discuss this with our runtimes team.

In the meantime you can try it by modifying AnimationState.cs locally. Find where applyAttachmentTimeline is called with the last parameter true and change true to mix < current.attachmentThreshold. I believe that should do it, though I haven't had time to test it.

Ggeowal-wondeluxe · 2022年12月8日

Sorry if this information is a bit late!

All good! I'm not so deep in yet where I couldn't change approach. It's just not clear to me what the best way to go is. I want to blend between a set of movement animations like WalkSlow, WalkFast, Run, but I will also want to stack animations on top such as Carry. From all of the resources I looked at it seemed like generally SkeletonAnimation was preferred, but maybe that's not the case? Either way I guess I'm having trouble as I'm trying to do things outside of the basic use cases (which I can't really find resources to guide me on).

I think I'm just about where I want to be thanks to your last tip. I'm calculating the attachments parameter for ApplyAttachmentTimeline as follows:

bool attachments = (i == 0 || mix >= current.attachmentThreshold);

I set the value to true for the first track to ensure attachments are always added. I also had to tweak the SetAttachment method so that slot.Attachment is only assigned if the attachments parameter is true. I'm not sure if this will cause other issues or not, but so far it seems things are working how I expect.

Thanks so much for your help!

Nate · 2022年12月8日

geowal-wondeluxe wrote
From all of the resources I looked at it seemed like generally SkeletonAnimation was preferred, but maybe that's not the case?

Using SkeletonAnimation is best for most users. It has the most features and doesn't require extra keys.

geowal-wondeluxe wrote
I set the value to true for the first track to ensure attachments are always added. I also had to tweak the SetAttachment method so that slot.Attachment is only assigned if the attachments parameter is true.

Your bool attachments logic looks good. You shouldn't need to avoid Slot setAttachment though. The attachment still needs to be set for reasons (deform timelines need it). Afterward, if nothing else set an attachment that should be kept, then the attachment is reset to the setup pose. AnimationState is quite complex, as there are a large number of edge cases. Hopefully you never need to dig in there any further. Also, please don't find any bugs! :tmi: