Hi!

I would like to implement a replay system in my game and was wondering how should I approach this problem. I'm using the skeleton animation. Basically, I need to be able to save the state of the animation in order to replay it in my replay system. It looks like I would need to save a LOT of stuff. I think I would need to serialize everything in the animation state + serialize track entries.

Anybody had to do anything close to this ?

I can't tell you how much I appreciate a response that quick and detailed. As a matter of fact, I think I could get away without the need to use backward playback, but I do need to be able to start from an arbitrary location which is why I can't really use an event driven approach and save add animation calls.

The bones rotation and location is pretty interesting too because I dont think the storage will be that much of an issue. I tried doing it and I was not able to apply it during playback. I will try and save Track Entries and the timescale of animation state. I will report back when I have it working and show you the end result and possibly helping others that would like to accomplish this sort of things as well.

It's almost working flawlessly.

Here is a recording I made : https://drive.google.com/file/d/1sHGXfqKlkMgvFYvrgsZWfDppzvlcjTzD/view?usp=sharing

The way the player skates is pretty much perfect. The goaltender when facing south has a real intense glitchy animation. Southwest and Southeast seems ok for some reason. Same for the skater when he celebrates at the end of the goal, all the bones are all over the place and I am not sure why. I'm thinking its probably related to some sort of attribute that I am not saving. The puck and the shadow is weird but it's normal, I have not gotten yet to saving the fake shadow position/rotation.

Here is what and how I serialize :

public class RecordableSkeletonAnimation : ReplayRecordableBehaviour
{

private SkeletonAnimation _skeletonAnimation;
private AnimationState animState = null;

public override void Awake()
{
    SetAnimState();
}

void SetAnimState()
{
    _skeletonAnimation = GetComponent<SkeletonAnimation>();
    if (_skeletonAnimation != null)
    {
        animState = _skeletonAnimation.AnimationState;
    }
}

public override void OnReplaySerialize(ReplayState state)
{
    if (animState == null)
    {
        SetAnimState();
    }

    state.Write(animState.TimeScale);
    state.Write(animState.Tracks.Count);

    animState.Tracks.ForEach(t =>
    {
        state.Write(t.Animation.Name);
        state.Write(t.Alpha);
        state.Write(t.Loop);
        state.Write(t.AnimationStart);
        state.Write(t.AnimationEnd);
        state.Write(t.AnimationLast);
        state.Write(t.AttachmentThreshold);
        state.Write(t.Delay);
        state.Write(t.DrawOrderThreshold);
        state.Write(t.EventThreshold);
        state.Write(t.HoldPrevious);
        state.Write(t.MixDuration);
        state.Write(t.MixTime);
        state.Write(t.TrackEnd);
        state.Write(t.TrackTime);
        state.Write(t.TimeScale);
    });
}

public override void OnReplayUpdate(ReplayTime replayTime)
{
    animState.Apply(_skeletonAnimation.Skeleton);
}

public override void OnReplayPlayPause(bool paused)
{
    animState.TimeScale = 0f;
    animState.Apply(_skeletonAnimation.Skeleton);
}

public override void OnReplayStart()
{
    _skeletonAnimation.enabled = true;
    _skeletonAnimation.UpdateMode = UpdateMode.FullUpdate;
}

public override void OnReplayDeserialize(ReplayState state)
{
    animState.TimeScale = state.ReadFloat();
    animState.Tracks.Clear();
    int numTracks = state.ReadInt32();

    for (var i = 0; i < numTracks; i++)
    {
        TrackEntry track = new TrackEntry();
        track.Animation = animState.Data.SkeletonData.FindAnimation(state.ReadString());
        track.Alpha = state.ReadFloat();
        track.Loop = state.ReadBool();
        track.AnimationStart = state.ReadFloat();
        track.AnimationEnd = state.ReadFloat();
        track.AnimationLast = state.ReadFloat();
        track.AttachmentThreshold = state.ReadFloat();
        track.Delay = state.ReadFloat();
        track.DrawOrderThreshold = state.ReadFloat();
        track.EventThreshold = state.ReadFloat();
        track.HoldPrevious = state.ReadBool();
        track.MixDuration = state.ReadFloat();
        track.MixTime = state.ReadFloat();
        track.TrackEnd = state.ReadFloat();
        track.TrackTime = state.ReadFloat();
        track.TimeScale = state.ReadFloat();
        
        animState.Tracks.Add(track);
    }
}
}

Any idea why and how I could fix the small glitches ?

I hit a wall :sucks:

I don't understand how I can get the regular skater to animate almost seamlessly, while the goaltender have its mesh all deformed. Basically we have animations facing south, south-east and south-west. When I transition from SE to SW it's working, but whenever I add animation to the south the mesh gets completely deformed. Also, I tried adding mixing by saving MixingFrom and MixingTo but it still is not mixing during playback. I really don't get how I can recreate the state for playing it back correctly.

Here is the goaltender deforming : https://drive.google.com/file/d/1l43t_UqpPRTtavIhZbvpGHFMI7gvxwwL/view?usp=sharing

Ok my bad, I thought it was obvious by looking at it. Here is an example where if the player is in the same "direction", its working perfectly. The player skates in south west then transitions to the celebration that is south west too. Works perfectly :

https://drive.google.com/file/d/1cF5nXMsaH1hFJPrYCt7V4J0lgkwvwHN9/view?usp=sharing

The next sequence is broken and the skater is going in north-east but the anim celebration transition to south west. In game it looks weird but we will eventually make a transition anim. In replay, it looks like the mesh is not updating at all and are still stuck with the anim in north east:

https://drive.google.com/file/d/1pgyvOoKSbzTL4DJdYy7Gdr5Ujab9D9ft/view?usp=sharing

And here is my code, I tried adding the mixing, it was not changing anything so I commented out.

using Spine;
using Spine.Unity;
using UltimateReplay;
using AnimationState = Spine.AnimationState;


public class RecordableSkeletonAnimation : ReplayRecordableBehaviour
{

private SkeletonAnimation _skeletonAnimation;
private AnimationState animState;

private TrackEntry currentTrack;
private TrackEntry mixingFrom;
private TrackEntry mixingTo;
private TrackEntry mixingNextFrom;
private TrackEntry mixingNextTo;
private TrackEntry next;

private float timescale;

public override void Awake()
{
    SetAnimState();
}

void SetAnimState()
{
    _skeletonAnimation = GetComponent<SkeletonAnimation>();
    if (_skeletonAnimation != null)
    {
        animState = _skeletonAnimation.AnimationState;
    }
}

private TrackEntry GenerateTrack(ReplayState state)
{
    TrackEntry track = new TrackEntry();
    track.Animation = animState.Data.SkeletonData.FindAnimation(state.ReadString());
    track.Alpha = state.ReadFloat();
    track.Loop = state.ReadBool();
    track.AnimationStart = state.ReadFloat();
    track.AnimationEnd = state.ReadFloat();
    track.AnimationLast = state.ReadFloat();
    track.AttachmentThreshold = state.ReadFloat();
    track.Delay = state.ReadFloat();
    track.DrawOrderThreshold = state.ReadFloat();
    track.EventThreshold = state.ReadFloat();
    track.HoldPrevious = state.ReadBool();
    track.MixDuration = 0.2f;//state.ReadFloat();
    track.MixTime = 0.2f;//state.ReadFloat();
    track.TrackEnd = state.ReadFloat();
    track.TrackTime = state.ReadFloat();
    track.TimeScale = state.ReadFloat();

    return track;
}

private void FillTrack(TrackEntry t, ReplayState state)
{
    state.Write(t.Animation.Name);
    state.Write(t.Alpha);
    state.Write(t.Loop);
    state.Write(t.AnimationStart);
    state.Write(t.AnimationEnd);
    state.Write(t.AnimationLast);
    state.Write(t.AttachmentThreshold);
    state.Write(t.Delay);
    state.Write(t.DrawOrderThreshold);
    state.Write(t.EventThreshold);
    state.Write(t.HoldPrevious);
    //state.Write(t.MixDuration);
    //state.Write(t.MixTime);
    state.Write(t.TrackEnd);
    state.Write(t.TrackTime);
    state.Write(t.TimeScale);
}

public override void OnReplaySerialize(ReplayState state)
{
    if (animState == null)
    {
        SetAnimState();
    }

    state.Write(animState.TimeScale);
    state.Write(animState.Tracks.Count > 0);

    if (animState.Tracks.Count <= 0)
    {
        return;
    }
    var currTrack = animState.Tracks.Items[0];

    FillTrack(currTrack, state);
    bool hasMixingFrom = currTrack.MixingFrom != null;
    bool hasMixingTo = currTrack.MixingTo != null;
    bool hasNext = currTrack.Next != null;
    bool hasNextMixingFrom = currTrack.Next != null && currTrack.Next.MixingFrom != null;
    bool hasNextMixingTo = currTrack.Next != null && currTrack.Next.MixingTo != null;;
    
    state.Write(hasMixingFrom);
    state.Write(hasMixingTo);
    state.Write(hasNext);
    state.Write(hasNextMixingFrom);
    state.Write(hasNextMixingTo);
    
    if (hasMixingFrom)
    {
        //FillTrack(currTrack.MixingFrom, state);
    }
    if (hasMixingTo)
    {
        //FillTrack(currTrack.MixingTo, state);
    }
    if (hasNext)
    {
        FillTrack(currTrack.Next, state);
    }
    if (hasNextMixingFrom)
    {
        //FillTrack(currTrack.Next.MixingFrom, state);
    }
    if (hasNextMixingTo)
    {
        //FillTrack(currTrack.Next.MixingTo, state);
    }
    
}

public override void OnReplayUpdate(ReplayTime replayTime)
{
    animState.Tracks.Clear();
    animState.TimeScale = timescale;
    
    animState.Tracks.Add(currentTrack);
    animState.Apply(_skeletonAnimation.Skeleton);
}



public override void OnReplayPlayPause(bool paused)
{
    if (paused)
    {
        animState.TimeScale = 0f;
        return;
    }
   
    animState.TimeScale = 1f;
}

public override void OnReplayStart()
{        
    _skeletonAnimation.UpdateMode = UpdateMode.FullUpdate;
    _skeletonAnimation.enabled = true;

}

/*private void LateUpdate()
{
    if (IsReplaying)
    {
        _skeletonAnimation.LateUpdate();
    }
}*/

public override void OnReplayDeserialize(ReplayState state)
{

    currentTrack = null;
    next = null;
    mixingFrom = null;
    mixingNextFrom = null;
    mixingNextTo = null;
    
    timescale = state.ReadFloat();
    if (!state.ReadBool())
    {
        return;
    }

    currentTrack = GenerateTrack(state);
    
    bool hasMixingFrom = state.ReadBool();
    bool hasMixingTo = state.ReadBool();
    bool hasNext = state.ReadBool();
    bool hasNextMixingFrom = state.ReadBool();
    bool hasNextMixingTo = state.ReadBool();


    if (hasMixingFrom)
    {
        //mixingFrom = GenerateTrack(state);
        //currentTrack.MixingFrom = mixingFrom;
    }
    
    if (hasMixingTo)
    {
        //mixingTo = GenerateTrack(state);
        //currentTrack.MixingTo = mixingTo;
    }
    
    if (hasNext)
    {
        next = GenerateTrack(state);
    }
    
    if (hasNextMixingFrom)
    {
        //mixingNextFrom = GenerateTrack(state);
        //next.MixingFrom = mixingNextFrom;
    }
    
    if (hasNextMixingTo)
    {
        //mixingNextTo = GenerateTrack(state);
        //next.MixingTo = mixingNextTo;
    }

    currentTrack.Next = next;
    if (hasNext)
    {
        currentTrack.Next.Previous = currentTrack;
    }
}
}

Do you think simply adding mixing would make it work ? I feel like there is some kind of call that is missing so that I can update the skeleton while this transition is happening.

Oh wow, thank you so much! I just added a call to SetToSetupPost() if the animation is different and it's working!!! There is still a couple of small glitches here and there, but you clearly explained why and I'll be able to change my code to a more delta-ing approach which will probably fix everything. There is no interpolation but so far it's pretty pretty close to the real thing and its good enough for our state of development. Thank you so much.

What it looks like now : https://drive.google.com/file/d/1u_-61iYWLbEzA9hC9D_4_pz8Z6DXTjWS/view

Hello again!

I finally tried to do the bone method. Once again, it's almoooost perfect. There is a little glitch here and there, sometimes major sometimes minor. Basically I am saving attachment changed and Getting/Setting them. I do the same with bones data. Finally, I do SkeletonAnimation.Update(), AnimationState.Apply(), Skeleton.UpdateWorldTransform() and then a LateUpdate() as mentionned in an earlier post.

What is happening is that for some reason during playback its like the bones are kind of stuck in the SetupPose. Here is a look in game play :

Here it is during playback:

which is how it looks when I click in the debug menu to SetToSetupPose().

Here is my code :

using System.Collections.Generic;
using Sirenix.Utilities;
using Spine;
using Spine.Unity;
using UltimateReplay;
using UnityEngine;
using AnimationState = Spine.AnimationState;


public class RecordableSkeletonAnimation : ReplayRecordableBehaviour
{

private SkeletonAnimation _skeletonAnimation;
private SkeletonUtility _skeletonUtility;
private AnimationState animState;

private TrackEntry currentTrack;
private TrackEntry next;

private float timescale;

private string previousAnimName;

private List<string> slotName = new List<string>();
private List<string> attachmentName = new List<string>();
private List<int> sortOrder = new List<int>();
private List<float> boneData = new List<float>();


public override void Awake()
{
    SetAnimState();
}

void SetAnimState()
{
    _skeletonAnimation = GetComponent<SkeletonAnimation>();
    if (_skeletonAnimation != null)
    {
        animState = _skeletonAnimation.AnimationState;
    }
}
public override void OnReplaySerialize(ReplayState state)
{
    if (animState == null)
    {
        SetAnimState();
    }
    
    state.Write(_skeletonAnimation.skeleton.Slots.Items.Length);
    _skeletonAnimation.skeleton.Slots.Items.ForEach(slot =>
    {
        state.Write(slot.Data.Name);
        if (slot.Attachment == null)
        {
            state.Write("null");
        }
        else
        {
            state.Write(slot.Attachment.Name);
        }
    });
    
    _skeletonAnimation.skeleton.DrawOrder.Items.ForEach(slotOrder =>
    {
        state.Write(slotOrder.Data.Index);
    });
    _skeletonAnimation.skeleton.Bones.ForEach(bone =>
    {
         state.Write(bone.WorldX);
         state.Write(bone.X);
         state.Write(bone.WorldY);
         state.Write(bone.Y);
         state.Write(bone.Rotation);
         state.Write(bone.ScaleX);
         state.Write(bone.ScaleY);
         state.Write(bone.ShearX);
         state.Write(bone.ShearY);
    });
    
}

public override void OnReplayPlayPause(bool paused)
{
    if (paused)
    {
        animState.TimeScale = 0f;
        return;
    }
    animState.TimeScale = 1f;
}

public override void OnReplayEnd()
{
    _skeletonAnimation.enabled = true;
}

public override void OnReplayStart()
{
    _skeletonAnimation.enabled = false;
    animState.ClearTracks();
}

private void LateUpdate()
{
    if (IsReplaying)
    {
        _skeletonAnimation.LateUpdate();
    }
}

private void Update()
{
    if (IsReplaying)
    {
        int idxBone = 0;
        _skeletonAnimation.skeleton.Bones.ForEach(bone =>
        {
            bone.WorldX = boneData[idxBone++];
            bone.X = boneData[idxBone++];
            bone.WorldY = boneData[idxBone++];
            bone.Y = boneData[idxBone++];
            bone.Rotation = boneData[idxBone++];
            bone.ScaleX = boneData[idxBone++];
            bone.ScaleY = boneData[idxBone++];
            bone.ShearX = boneData[idxBone++];
            bone.ShearY = boneData[idxBone++];
        });
        

        _skeletonAnimation.skeleton.DrawOrder.Sort((slot1, slot2) =>
        {
            var idxSlot1 = sortOrder.FindIndex(slotIdx => slot1.Data.Index == slotIdx);
            var idxSlot2 = sortOrder.FindIndex(slotIdx => slot2.Data.Index == slotIdx);
            return idxSlot1 - idxSlot2;
        });
        for (var i = 0; i < slotName.Count; i++)
        {
            if (attachmentName[i] == "null")
            {
                _skeletonAnimation.skeleton.SetAttachment(slotName[i], null);
                continue;
            }
            _skeletonAnimation.skeleton.SetAttachment(slotName[i], attachmentName[i]);
        }
        _skeletonAnimation.Update(Time.deltaTime);
        _skeletonAnimation.state.Apply(_skeletonAnimation.skeleton);
        _skeletonAnimation.skeleton.UpdateWorldTransform();
    }
}

public override void OnReplayDeserialize(ReplayState state)
{
    
    slotName.Clear();
    attachmentName.Clear();
    sortOrder.Clear();
    boneData.Clear();
    var numSlots = state.ReadInt32();
    for (var i = 0; i < numSlots; i++)
    {
        slotName.Add(state.ReadString());
        attachmentName.Add(state.ReadString());
    }
    for (var i = 0; i < numSlots; i++)
    {
        sortOrder.Add(state.ReadInt32());
    }
    _skeletonAnimation.skeleton.Bones.ForEach(bone =>
    {
        boneData.Add(state.ReadFloat());
        boneData.Add(state.ReadFloat());
        boneData.Add(state.ReadFloat());
        boneData.Add(state.ReadFloat());
        boneData.Add(state.ReadFloat());
        boneData.Add(state.ReadFloat());
        boneData.Add(state.ReadFloat());
        boneData.Add(state.ReadFloat());
        boneData.Add(state.ReadFloat());
    });
}
}

What should I do to make it render the correct bones positions ? I tried saving differents things but always got the same result.

Ok I think I figured it out, I need to save the deform by slots as well! It's starting to be a whole lot of data to save by frame hehe.

That's what I tried first but I had a bit of difficulty serializing everything and there was always a couple of glitches so I changed to the bone method. Let me know if you ever do something about the serialization of the animation state, ill be more than interested.

Any update on the serialization task ? I'm implementing a multiplayer mode, so I tried to switch to mecanim in order to use out of the box network animator, but our setup is a bit too complex and I could not get the mixing to be as smooth than with SkeletonAnimation. So now I'm trying to send as little as possible data on the network to sync all animations, hence why the serialization of animation state would help me a bunch.

Here is the replay in action on the last goal in the video : https://twitter.com/Tape2Tape_Game/status/1486070308494876679

Yes you are completely right. I had the replay mindset which in this case I need the anim to be pretty much identical as what happened during the play. But for online I must indeed only pass the gamestate and reproduce the animations accordingly.

Thank you!