• Showcase
  • Automating Layer Separation for Spine2D

Hello, I am very interested in segmentation models.
The reason I'm interested in segmentation model is that it can automate layer separation in Spine 2D.

I have created a project that utilizes the Dlib model to parse faces and create a layer format in GIMP.

https://github.com/jhj0517/Dlib-Face-Parsing-WebUI

The segmentation range recognized by Dlib model includes:

[Jaw line , Right eyebrow , Left eyebrow , Nose , Right eye , Left eye , Lips]

My project automatically parses these parts of the face from an input image, separates them into individual images, and combines them into a .tif file that can be opened in GIMP.

To make it easy to use, I made it using the Gradio package, which makes it easy to use on the Web-UI, and also made it easy to install and run with a batch file (bash for Linux).

Detailed installation and usage instructions can be found in my Github repository.

I believe Spine 2D is the most convenient program for creating animations.
However, separating layers from an existing image (instead of drawing them separately from the beginning) can be very challenging for people who are not professionals in drawing, just like me.

Therefore, I thought it would be great to have a project that automates this process.

However, Dlib may not be the ideal model for my project due to its limited range of segmentation types and lower-than-expected accuracy. Additionally, as it is trained on real human faces, the model does not perform well in recognizing the faces of animated characters, which is a significant drawback.
and most importantly, it is limited to facial recognition.

I am currently studying Controlnet to find a better model to achieve this cool feat. (I'm looking for an appropriate parsing model)
I am working on a project that can perform parsing properly as I want.

I hope my project can be helpful to Spine 2D users who have difficulty with layer separation like myself.
Thank you for reading.

  • Misaki が「いいね」しました。
Related Discussions
...

It looks cool, thanks for sharing! Though I can't really see what it does from the screenshot. I think it intelligently cuts up an image into layers, but the screenshot showing the layers just shows checkerboards, so I can't see what is on each layer.

  • jhj0517 がこの投稿に返信しました。

    Cool. The AI model is large at 95 Mb. How long does it take to segment the image?

    • jhj0517 がこの投稿に返信しました。

      Nate
      You are correct! I'm sorry for the lack of explanation. Using the dlib model, It parses multiple facial features (right eyebrow, left eyebrow, nose, right eye, left eye, lips, face, and background) in the photo and separated them into layered files.
      I am working on a project for parsing as shown in the picture below. (It's someone else's work, I just borrowed it as an example.)

      I am looking for a similar model that can do parsing like that on Hugging Face.

      SilverStraw
      My script includes rendering with cv2, saving as an image, and saving as a .tif file, so it's difficult for me to give an exact time (for me, running everything takes about 15-20 seconds), but since the model size is small, I am confident that the segmentation itself takes a very short amount of time (probably in single-digit seconds)

      That's a very cool idea, thanks for sharing your project!