このサイトを正しく表示するには、最新のブラウザを使用し、Javascriptを有効化してください。

Mix and Match Normalmap Packing

Anisoft

What is the correct way create a sprite via mix and match, and pack normals with the character?

I have a character creator which uses mix and match on different unique sprite parts. I got all the mix and match working but what I don't fully understand how to get normal textures to work too.

I was thinking I'd need to have a way to assign both the diffuse and the normal sprite.

I'm trying to keep all my sprites as separate pieces, each with their own normal. Then assign or pack at runtime.

Edit* Is the method involved still the same as mentioned here?

http://esotericsoftware.com/forum/Runtime-Repacking-with-Normal-Map-12701

Harald

Please do not implement anything yourself as described in the above posting!

The proper and easy way is described in the official spine-unity documentation pages here:
spine-unity Runtime Documentation: Combining Skins (see subsection Advanced - Runtime Repacking with Normalmaps below)

Anisoft

What has me confused is that there isn't an input for the actual normalmap sprite. So I'm unsure how it creates the normalmap skin without a reference to my actual normal sprite.

My game generates all the skins at runtime via combining skin parts into skins.

So I had thought I'd need to make a sprite for the normal-map and put it in the part.

Harald

Sorry for the late reply.

How are you combining the single Sprites in your PartSO script? Are you using Attachment.GetRemappedClone()? Please provide more context regarding the code used in your PartSO component.

In general, if you have created a working single Attachment that has a Material with a normalmap assigned, then it will use this assigned normalmap texture when packing a new texture-set via Skin.GetRepackedSkin(). So if the Material has the normalmap assigned beforehand, it will work after packing (when following the documentation section mentioned above and passing the respective optional parameters at GetRepackedSkin()).

Anisoft

I am too sorry for the late reply.

Currently I combine everything inside my CharacterVisual script.

So this is my CharacterVisuals script Im using.

using System;
using Anisoft.Character;
using Anisoft.Game.Entity.Skin;
using Spine;
using Spine.Unity;
using Spine.Unity.AttachmentTools;
using UnityEngine;

namespace Anisoft.Behaviors.Character {
    [RequireComponent(typeof(CharacterB))]
    public class CharacterVisualsB: MonoBehaviour {
        private SkeletonAnimation _skeletonAnimation;
        private CharacterB CharacterRef { get; set; } = null;
        public GameObject visuals;
        public Material sourceMaterial;
        public Material blankMaterial;

    public Skin[] skinParts;
    public Skin customSkin;
    [Header("Do not assign")] public Texture2D runtimeAtlas;
    public Material runtimeMaterial;

    public Texture2D[] additionalOutputTextures = null;
    public int[] additionalTexturePropertyIDsToCopy = new int[] { Shader.PropertyToID("_BumpMap") };
    private bool _refreshTexture = true;
    
    
    public void Setup(CharacterB character) {
        skinParts = new Skin[Enum.GetValues(typeof(SkinType)).Length];

        CharacterRef = character;
        CharacterRef.UpdateVisuals += RefreshVisuals;
        _skeletonAnimation = visuals.GetComponent<SkeletonAnimation>();
    }

    private void RefreshVisuals() {
        RefreshSkin();
        RefreshSkinTexture();
    }

    private void RefreshSkin() {
        var skinPartsSlots= CharacterRef.DataActor.dataSkinPartSlots;
        var skeleton = _skeletonAnimation.Skeleton;
        var templateSkin = skeleton.Data.FindSkin("base").GetClone();
        

        customSkin = customSkin ?? new Skin("Custom Skin");
        customSkin.AddAttachments(templateSkin);

        for (var index = 0; index < skinPartsSlots.Length; index++) {
            var skinPart = skinPartsSlots[index];
             if (skinPart == null || skinPart.Name == (skinParts[index]?.Name)) continue;

            
            var skin = new Skin(skinPart.Name);
            foreach (var subParts in skinPart.subParts) {
                var attachKey = subParts.attachKey;
                var attachSlot = subParts.attachSlot;
                var slotIndex = skeleton.FindSlotIndex(attachSlot);
                var attachSprite = subParts.attachSprite;

                var templateAttachment = templateSkin.GetAttachment(slotIndex, attachKey);
                var newAttachment = templateAttachment.GetRemappedClone(attachSprite, blankMaterial,
                    false, false, true);
                skin.SetAttachment(slotIndex, attachKey, newAttachment);
            }
            customSkin.AddAttachments(skin);
            skinParts[index] = skin;
            _refreshTexture = true;

        }
    }
    private void RefreshSkinTexture() {
        if (!_refreshTexture) return;

            var skeleton = _skeletonAnimation.Skeleton;

        customSkin = customSkin.GetRepackedSkin("custom skin", sourceMaterial, 
            out runtimeMaterial, out runtimeAtlas,2048, 
            additionalTexturePropertyIDsToCopy : additionalTexturePropertyIDsToCopy, additionalOutputTextures : additionalOutputTextures);
        skeleton.SetSkin(customSkin);
        _refreshTexture = false; 
    }
}



}

Which looks like the following in the inspector

What I'm trying to do is build a custom skin + normals from a base skin (using a texture atlas and normal texture atlas)
and then on top apply separate attachments that each have their own texture/normals

One issue I found is that if I assign a material that doesn't have a normal, I get an error about missing textures.
//inspector

//base texture settings

//error

So i tested out just using the same material with the same normals for each (for testing)

And it... sorta worked but resulted in an incorrect normalmap.

I will note that it wouldn't allow me to combine the textures unless they matched formats. So I changed it to the only uncompressed format I could RGBA 32 bit

And it seems like when I check the channels... the GBA have texture but the R doesn't... so it seems like it copied the colors to the wrong channels based on what i did?

So its pretty evident I don't know what I'm doing just yet lol. I'm try to get some sleep and tackle it more later tmw. I hope I've provided some info as to what I'm trying to pull off here.

I tested not combining the skin and it appears the seperate attachment with a placeholder normal worked fine.|

Separate attachments

Combined skin attachments

The combined skin doesn't apply the normal correctly

Looking at the combined texture it seems the rgba aren't correct for me. Its placing the channels of the normal texture in gba instead of rgb.

The added normal is the little square of noise in the middle right.

Harald

Thanks very much for the detailled writeup!

Anisoft wrote
One issue I found is that if I assign a material that doesn't have a normal, I get an error about missing textures.

Unfortunately packing would not work consistently if regions (where textures are null) would be skipped. So it is a requirement that all repacked materials have the all used maps assigned and not be null.

Anisoft wrote
Looking at the combined texture it seems the rgba aren't correct for me. Its placing the channels of the normal texture in gba instead of rgb.

You are right about the repacked normalmap looking strange in terms of RGBA order.
I would assume that the content is not normal xyz input data, but already the prepared reordered data, e.g. _y_x as is commonly used for e.g. DXT5_NM compression and other later compressed formats. Here the Y and A channels are filled with data, as they always receive the highest precision when packed using any texture compression format (since our eye is more sensitive to green than to blue). The third channel (Z, upward) is left out and deduced at runtime in the shader for greater precision. Judging from the look of the single channels data at my local test scene, this is exactly what happens when I press play:

Strangely your output image seems slightly different in channel order, with up and right channels seeming flipped compared to mine.

When I test copying from a source normalmap to a repacked texture, the source format shows RGBA32 in the debugger (with Windows active as target platform). For testing, I have implemented functionality to specify RGB24 at the normalmap, which however fails at the copy operation with error message of incompatible src and dst formats (again listing src as RGBA).

What puzzles me, is that you said that the repacked normal map does not apply correctly and your channels seems flipped in X/Y order - it behaves correctly at my test setup, using the Stretchyman assets. Could you perhaps please send us a minimal reproduction Unity project to contact@esotericsoftware.com, then I can have a look at what's going wrong.

Anisoft

I sent an email with the file.

I do caution it contains a nsfw base character.

It should be a very small and easy to open file.

Harald

Thanks for the reproduction package. I will get back to you as soon as I've figured out what's going wrong.

Anisoft

Okay thank you! I wonder if it has to do with the default material or something. Since if you notice its generating normalmaps automatically using the base material it seems, then it uses the newly one I added.

I actually liked that as it allows me to easily use a base skin/template, but maybe its the reason things are acting up?

Harald

The reason why the hair normal map ended up incorrect is because of different size - the diffuse sprite image and normal map needs to match in size, otherwise different regions are copied upon the repack operation. So be sure to use a normalmap of the same size (I have sent you one as attachment), and set the texture import settings in Unity under Advanced - Non-power of 2 to None instead of To nearest to keep this size.

After this modification, the scene looks correct after repacking, including the hair which was looking incorrect before due to only having copied a smaller part of the too-large normalmap.

What I don't quite understand is why the repacked normal map channel order is different in your project (with your project that is also the case at my PC) as compared to the project I tested repacking normalmaps of Stretchyman in the past. Also the target platforms are the same, which was my first suspect. Anyway, as the lit result scele looks correctly oriented, I would declare this issue as resolved. Please let me know if it looks correct from your view now as well.

Anisoft

Thanks so much for looking into it! Unfortunately after following what you said I'm still getting wrong normals I feel.

left is the original, middle is part packed into 1 texture, right is part but seperate materials

The middle packed one is off compared to the others.

Harald

Now I found your problem. Your input normalmap atlas is incorrect:

Here the head and hair region has green facing left and red facing up, looking as if the normalmap has just been copied and rotated 90 degrees. As a result, the face and the initial hair behave incorrectly.
How did you create these incorrect normalmap regions? In case you manually copied and rotated the regions in an image editor, please always make sure to flip red and green channels accordingly when rotating.

Anisoft

Heya, thanks for pointing that out. I had the sprites set to rotate in texture packer.

However disabling that doesn't change the issue of the normals looking off still.

I noticed that my shader material settings didn't match (so will need to make sure they are copied properly in future automatically)

However the issue is that even after adjusting the material props to match in post... there still isn't parody

Here is an example with the diffuse removes and just the normals

left: original, middle is parts combine with texture repack, right is parts combine multi material

It looks like the g and b are like identical maps... and dont line up properly with how the r (horzontal) and the b( vertical) should be

The depth map (a in the repack) doesn't match up properly at all with how the depth map should be in the source

I will stress that these all share material settings that are exactly identical now. left and right share exact parody but the repack normals don't act the same.

I appoligize for letting this go on for so long, I just wanna know why its not resulting in the same results.

I also noticed that the channels in the normal map repack doesn't seem correct. the g and the b look like they are the same map...

Here take a look at the default base map

repack map (with no added textures, just a repack)

Harald

The depth map (a in the repack) doesn't match up properly at all with how the depth map should be in the source

The third channel in your input texture is not a depth map, it is the Z (where Z = direction of the vertex normal) component of the normal. This is redundant information and does not need to be stored. We do not manually copy channel content, it is left to Unity's implementation of Texture2D.PackTextures. I already answered why the repacked normalmap channel content looks like it does:

I would assume that the content is not normal xyz input data, but already the prepared reordered data, e.g. _y_x as is commonly used for e.g. DXT5_NM compression and other later compressed formats. Here the Y and A channels are filled with data, as they always receive the highest precision when packed using any texture compression format (since our eye is more sensitive to green than to blue). The third channel (Z, upward) is left out and deduced at runtime in the shader for greater precision.

It looks like the g and b are like identical maps... and dont line up properly with how the r (horzontal) and the b( vertical) should be

As described above, you should be looking at the g and a channels.
What do you mean by "dont line up properly"? I see no misalignment in terms of offset in horizontal or vertical direction.

What I assume is that the same project is interpreted differently on your hardware than it is on mine.
When I open your reproduction Unity project that you sent via email originally and hit play, the rest of the body has exactly the same lighting appearance after repacking as without repacking. Could you try if this is the case at your clean unmodified reproduction package as well? If it looks different (what I assume), which target platform are you currently on?

Anisoft

Your absolutely right about a new project having it look correct. I'm going to have to really look and see why the heck my main project is producing different results.

I am so sorry for wasting so much of your time on this.Thank you so much for your help.

Hey by chance... would it be something with the rendering settings color space?

In a empty test project, the results were exactly the same in gamma space, however switching it to linear the newly made normalmap packs were off color.

Gamma Space

Linear Space

In gamma they are all identical (which likely was how it was for in ur testing)

However in linear (the color space I'm working in) the colors of the repack (left image) are wrong

Easy solution would be to just use gamma however Im using Linear for specific post processing / lighting effects. Its not the same unless I use that color space.

Harald

That is interesting, I will have a look at your example project when switching it to Linear color space.

I just tested your reproduction package with color space set to Linear. Still I cannot reproduce your problem above of different results after repacking. I get the same lighting behaviour before and after repacking, as with Gamma color space.

Did you perhaps set any of the normal maps not to Texture Type Normal Map? Or did you accidentally set any diffuse texture to not be sRGB?

Anisoft

I'm really sorry for having this drag on so long and wasting your time.

linear:

gamma

This is the same exact project I gave you with the only thing changed being the normals being properly flipped, and the hair being resized correctly.

These are the settings applied to the base texture.

Settings for the hair piece

This is tested on a another computer as well. (my dev partners)

Last test with ambient lighting/reflections off;

Top: Gamma
Bottom: Linear

I tried testing this on a standard Sprite-Diffuse shader and found no difference between versions, however testing it with any shader at all. Its the use of those repacked normal maps this issue occurs.

I'm curious if this may have to do with the normal packing format and the colorspace?

I sent an updated demo that is set by default to linear if you have a moment to check it out. Just open the scene and press play.

FOUND THE ISSUE

I went into the AtlasUtilities and testing out forcefully setting the output texture to linear if the index was 1 (since I only have two textures the second texture index 1 would be the normal.

And I found that seemed to negate the issue!

Linear
left: repack right: none

Hack that I did
AtlasUtilities.cs

var newTexture = new Texture2D(maxAtlasSize, maxAtlasSize,
               (additionalTextureFormats != null && i - 1 < additionalTextureFormats.Length) ?
               additionalTextureFormats[i - 1] : textureFormat,
               mipmaps, i ==1);

I'm curious how this should translate into a fix? Setting the diffuse to linear isn't a good idea (since that messes up the color)

So maybe we need the ability to specifiy which textures are linear? and also specifically make normals be linear?

What do you suggest? (if I'm even correct in my analysis)

Harald

Thanks for digging in so deep and for your detailled report! Actually I read your email with the reproduction package first and immediately jumped into analysing and fixing the problem, just to come back to see that you did an analysis as well. At least we have come to the same conclusion. 🙂

I have just released the bugfix, you can download the updated packages here as usual:
Spine Unity Download
Please let me know if this resolved the problem on your sideas well.

The GetRepackedSkin() methods now provide an additional optional parameter:

bool[] additionalTextureIsLinear = null

Usage is as follows:

When additionalTexturePropertyIDsToCopy is non-null,
this array will be used to determine whether linear or sRGB color space is used at the
Texture at the respective property. When additionalTextureIsLinear is null, linear color space
is assumed at every additional Texture element.
When e.g. packing the main texture and normal maps, pass 'new bool[] { true }' at this parameter,
because normal maps use linear color space.

So the default, leaving it at null, shall lead to correct results out of the box.

Thanks for reporting and not giving up on the subject!

For the record:
Issue ticket was: https://github.com/EsotericSoftware/spine-runtimes/issues/1602

Anisoft

Thank you for not giving up on the problem! This seems to have worked wonders!

I assume the ability to force stuff to be linear or not is useful beyond just normals for if I have a linear texture I wanted copied over too (so maybe something like a emissive texture?, actually I'm not fully sure what its other uses could be)

Tbh I'm not really sure what the usage of checking stuff not to be srgb in the texture importer is for too, I know I was messing with it for my own normal map generator (and thats how i came across realizing I'd need to make sure my stuff converted to gamma/linear for it to be accurate to my reference)

Thanks again for being so quick to helping me with this issue. It really means a lot!

I'll be writing a system soon that allows for baking out colors from a material into the spine texture (for character creators skin/item colors).

Ill be sure to share when I finish it.

Thanks so much Harald