kgambill

I'm working in MonoGame (C#) and making a game which uses 2D billboards to render characters in a 3D world. Everything is currently working well using basic textures applied to billboarded quads. However, I am now working to try and get Spine wired into this flow as we are going to use it for the characters and animations.

However, I am currently running into an issue trying to get Spine rendering properly into my 3D world. I believe the major issues is that the SkeletonRenderer class always sets the Z value of its vertices to 0 in the following code snippet from the Draw function:
// submit to batch
MeshItem item = batcher.NextItem(verticesCount, indicesCount);
item.texture = texture;
for (int ii = 0, nn = indicesCount; ii < nn; ii++) {
item.triangles[ii] = indices[ii];
}
VertexPositionColorTextureColor[] itemVertices = item.vertices;
for (int ii = 0, v = 0, nn = verticesCount << 1; v < nn; ii++, v += 2) {
itemVertices[ii].Color = color;
itemVertices[ii].Color2 = darkColor;
itemVertices[ii].Position.X = vertices[v];
itemVertices[ii].Position.Y = vertices[v + 1];
itemVertices[ii].Position.Z = 0;
itemVertices[ii].TextureCoordinate.X = uvs[v];
itemVertices[ii].TextureCoordinate.Y = uvs[v + 1];
if (VertexEffect != null) VertexEffect.Transform(ref itemVertices[ii]);
}
I see three solutions to the problem but all have some issues:
1) Use the "World" matrix in the SpineEffect shader to achieve my 3D placement. While this works and is pretty easy to implement, it will be pretty slow because of the fact that I need to set this parameter for each character in the game level, which means I can no longer set the World / View / Proj once and then batch render everything.

2) Render all Spine characters to a render texture, and then use proper UV calculations to essentially cut that up into the textures used by the billboarded quads. This should be more performant than option 1, but the render texture may end up being fairly large if we have a level with a large number of characters, especially if we have multiple characters of drastically different shapes and sizes.

3) Since I'm building from source, just edit the SkeletonRenderer class to have a new z position variable. Then in the above code snippet when calculating the Position values, assign that z value to the verts instead of 0. My main concern here is mucking about in code I don't have a great understanding of and potentially causing issues down the line.

If anyone has had any success doing something similar, or if someone much more familiar with the Spine runtimes could weigh in, it would be very helpful. Thanks!
アバター
kgambill
  • 記事: 15

Harald

The intended use is number (1), however note that if by SpineEffect you are referring to the following line of the examples:
var spineEffect = Content.Load<Effect>("spine-xna-example-content\\SpineEffect");
then this is a special two color tint effect (as documented above). You can simply set the BasicEffect that is already assigned and set it's World matrix as follows:
skeletonRenderer.Effect.World = yourMatrixToPositionTheObject;
kgambill さんが書きました:I need to set this parameter for each character in the game level, which means I can no longer set the World / View / Proj once and then batch render everything.
Note that each SkeletonRenderer has its own BatchRenderer (to batch-render all attachments of a character that share the same texture), it is not shared across SkeletonRenderers. Also note that all coordinates are in local object-space, with Z=0 being consistent with all vertex coords being relative to the character's origin.
アバター
Harald

Harri
  • 記事: 3353

kgambill

Thanks for the information, Harald. I did not realize that each SkeletonRenderer uses its own BatchRenderer. This simplifies things greatly as I'm not going to really decrease performance with the World Matrix set per character. Looks like this is a classic case of me overthinking the problem!
アバター
kgambill
  • 記事: 15

Harald

No problem, here to help! :)
アバター
Harald

Harri
  • 記事: 3353

DapperDave

I'm trying to achieve the same thing in almost the exact same conditions. But I am prohibited from using a shader effect. I also cannot use XNA's BasicEffect.

Before using Spine, I was able to create a billboarding effect for 2D planes in my 3D game using an unusual method.
1) Rotate the four points that make up the quad horizontally about the center in order to match the camera's left/right orientation. currentHorizontalRotation = MathHelper.ToDegrees(-References._3Dcamera.Rotation.Y);

2) (This is the weird one) When changing the camera's X value, stretch the height of the plane so it does not seem compressed by change in the perspective. float rScale = 1f / (float)Math.Cos(References._3Dcamera.Rotation.X);


This makes the planes look the same at any angle. Also it keeps the planes from clipping with nearby 3D objects since the plane is not being tilted to match the camera's perspective.

See my video here:


This may seem like a strange way to achieve billboarding but there are some restrictions I must follow so please go with it:

Anyway, now how could I apply this to spines? I have added spineboy to my scene here:



Is there a simple way to rotate the entire spine animation about it's center? Then I could achieve the step 1 I achieved earlier.

Step 2, stretching the height of the entire thing, would be much more difficult since Spineboy isn't a single plane but many moving images.

I could potentially do this by rendering spineboy to a rendertexture as kgambill pondered but maybe these two steps can be done more easily in the skeletonrenderer somehow.
DapperDave
  • 記事: 15

Harald

DapperDave さんが書きました:But I am prohibited from using a shader effect. I also cannot use XNA's BasicEffect.
..
This may seem like a strange way to achieve billboarding but there are some restrictions I must follow so please go with it:
I'm not sure I understand your constraints completely. If your constraints are that you cannot use XNA's BasicEffect or any shader effect, this implies that you also cannot use the SkeletonRenderer component, as it always uses an effect. Or are your constraints only that your outcome needs to look the same as in the first video?

How are you currently applying the calculated rotation to the vertices? Are you manually rotating the mesh vertices? If so, then we would recommend against that. If you really need to do that, then it's better to calculate a complete World transformation matrix for your whole skeleton object first, and then apply it to all your mesh vertices.

Your steps 1 and 2 sound like you basically want to create a horizontal billboard effect, leaving the up-axis as-is. This could be achieved in an easier way by creating the object's World matrix once using the unmodified up axis and the camera's right and forward axes (right axis as-is, the forward axis needs to be projected onto the horizontal plane and normalized), instead of calculating rotation degrees and then rotating individual mesh vertices on the CPU. In any way, it's recommended to create a single matrix for the whole skeleton object once, you can also do this via rotation calculations and then combining rotation and translation matrices if you prefer that.

Then you can either transform the mesh vertices directly with this matrix (software transformation on the CPU) or set it as an effect parameter as effect.World = HorizontalBillboardObjectMatrix (hardware transformation on the GPU). If you really want to modify the mesh vertices in software, then you would need to either write your own MeshBatcher subclass and then assign it at the SkeletonRenderer, or modify the existing MeshBatcher code to perform an additional matrix multiplication of the mesh vertices in e.g. SkeletonRenderer.Draw(). Unfortunately there is currently no easy way to perform custom software transformation of mesh vertices, since it's not recommended.
アバター
Harald

Harri
  • 記事: 3353

DapperDave

Harald さんが書きました:
DapperDave さんが書きました:But I am prohibited from using a shader effect. I also cannot use XNA's BasicEffect.
..
This may seem like a strange way to achieve billboarding but there are some restrictions I must follow so please go with it:
I'm not sure I understand your constraints completely. If your constraints are that you cannot use XNA's BasicEffect or any shader effect, this implies that you also cannot use the SkeletonRenderer component, as it always uses an effect.
I'm using a custom shader that has some of the features of BasicEffect but not all of them. I can't modify this custom shader.
//var basicEffect = new BasicEffect(device);
//basicEffect.World = Matrix.Identity;
//basicEffect.View = DetentionScreen._3Dcamera.View; //Matrix.CreateLookAt(new Vector3(0.0f, 0.0f, 1.0f), Vector3.Zero, Vector3.Up);
//basicEffect.TextureEnabled = true;
//basicEffect.VertexColorEnabled = true;
//effect = basicEffect;

var textureEffect = References.textureEffect;
textureEffect.Parameters["MatrixTransform"].SetValue(References._3Dcamera.View * References._3Dcamera.Projection);
EffectParameter textureParameter = textureEffect.Parameters["Texture"]; //don't ask. not my design.
if (textureParameter == null) textureParameter = textureEffect.Parameters["TextureSampler"]; //don't ask. not my design.
effect = textureEffect;
How are you currently applying the calculated rotation to the vertices? Are you manually rotating the mesh vertices? If so, then we would recommend against that. If you really need to do that, then it's better to calculate a complete World transformation matrix for your whole skeleton object first, and then apply it to all your mesh vertices.
This is what I'm using in the draw method instead
if (skeletonRenderer.Effect is BasicEffect)
{
((BasicEffect)skeletonRenderer.Effect).Projection = References._3Dcamera.Projection;
((BasicEffect)skeletonRenderer.Effect).Projection = References._3Dcamera.View;
}
else
{
skeletonRenderer.Effect.Parameters["MatrixTransform"].SetValue(References._3Dcamera.View * References._3Dcamera.Projection);
}
Your steps 1 and 2 sound like you basically want to create a horizontal billboard effect, leaving the up-axis as-is. This could be achieved in an easier way by creating the object's World matrix once using the unmodified up axis and the camera's right and forward axes (right axis as-is, the forward axis needs to be projected onto the horizontal plane and normalized), instead of calculating rotation degrees and then rotating individual mesh vertices on the CPU. In any way, it's recommended to create a single matrix for the whole skeleton object once, you can also do this via rotation calculations and then combining rotation and translation matrices if you prefer that.

Then you can either transform the mesh vertices directly with this matrix (software transformation on the CPU) or set it as an effect parameter as effect.World = HorizontalBillboardObjectMatrix (hardware transformation on the GPU).
I'm still too ignorant in 3D coding to follow what you are saying here very well but I will look into this and report back. If you have any examples you can point me to, that would help immensely.

---

Harald さんが書きました:This could be achieved in an easier way by creating the object's World matrix once using the unmodified up axis and the camera's right and forward axes (right axis as-is, the forward axis needs to be projected onto the horizontal plane and normalized)

Then you can either transform the mesh vertices directly with this matrix (software transformation on the CPU)
This is my best stab at it. On a scale of 1 to 10 how close am I?
Vector3 forwardVector = References._3Dcamera.cameraLookAt;
forwardVector.Normalize();

Matrix SkeletonMatrix = Matrix.CreateWorld( //Creating a world matrix
References._3Dcamera.View.Left, //Camera's right/left axis
forwardVector, //camera's forward vector normalized
Vector3.Up); //up

//transforming mesh vertice with matrix
itemVertices[ii].Position = Vector3.Transform(itemVertices[ii].Position, SkeletonMatrix);
DapperDave
  • 記事: 15

Harald

DapperDave さんが書きました:This is what I'm using in the draw method instead
if (skeletonRenderer.Effect is BasicEffect)
{
((BasicEffect)skeletonRenderer.Effect).Projection = References._3Dcamera.Projection;
((BasicEffect)skeletonRenderer.Effect).Projection = References._3Dcamera.View; // I assume here you assign to View and not to Projection, right?
}
else
{
skeletonRenderer.Effect.Parameters["MatrixTransform"].SetValue(References._3Dcamera.View * References._3Dcamera.Projection);
}
You could basically add the Model matrix in line as follows:
skeletonRenderer.Effect.Parameters["MatrixTransform"].SetValue(modelMatrix * References._3Dcamera.View * References._3Dcamera.Projection);
DapperDave さんが書きました:This is my best stab at it. On a scale of 1 to 10 how close am I?
The matrix should be pretty close already! The forward vector is however a bit different (I just noticed that the cross product is easier than projecting it into the horizontal plane):
Vector3 forwardVector = Vector3.Cross(Vector3.Up, References._3Dcamera.View.Left);
The last line that manually transforms the vertices on the CPU is not necessary however, if you can set the "MatrixTransform" matrix. How are you setting the translation however? I did not see it in your code above.
アバター
Harald

Harri
  • 記事: 3353

DapperDave

Okay, with Harald and some outside help I have a solution.

First create this Matrix at the start of the SkeletonRenderer.Draw
ModelMatrix = Matrix.Identity;
ModelMatrix.Up = Vector3.Up;
ModelMatrix.Right = new Vector3(References._3Dcamera.View.Forward.Z, 0, References._3Dcamera.View.Forward.X);
ModelMatrix.Translation = new Vector3(skeleton.X, skeleton.Y, skeleton.Z);
Then multiply the Effect by this matrix at the end
var MVP_Matrix = ModelMatrix * References._3Dcamera.View * References._3Dcamera.Projection;
textureEffect.Parameters["MatrixTransform"].SetValue(MVP_Matrix);
This will rotate the Spine animation left/right along with the camera. It does not yet rotate the Up so that it is always oriented with the camera. I'm having second thoughts if I want it to do that. But if I do, I just need to change ModelMatrix.Up to something else that reflects the camera somehow. I'll post that if I figure it out.

Thanks for your help. It looks like Spine will be a good fit for this project and I will start creating my own animations soon.
DapperDave
  • 記事: 15

Harald

Glad to hear you've figured it out!

A remark about your code: We would recommend to still set the ModelMatrix.Forward vector. When you don't set the ModelMatrix.Forward, it should do no harm if you have your skeleton's MeshGenerator.zSpacing set to 0, but as soon as you need to offset your attachments in the models Z-direction, you will wonder why it behaves in a strange way.
アバター
Harald

Harri
  • 記事: 3353

DapperDave

I made a mistake that I should have caught. My camera view includes a zoom multiplier. I should have taken it out for this line:
ModelMatrix.Right = new Vector3(References._3Dcamera.View.Forward.Z, 0, References._3Dcamera.View.Forward.X);
Because it's being implemented twice with this line:
var MVP_Matrix = ModelMatrix * References._3Dcamera.View * References._3Dcamera.Projection;
DapperDave
  • 記事: 15

Harald

Glad to hear you've figured it out. :) Thanks for letting us know.
アバター
Harald

Harri
  • 記事: 3353


Return to Runtimes