So recently I have finally started working on 3D... as a 2D dev this has been a tough ride, but finally I achieved something that looks interesting. The major difficulty in this process was the fact that there hardly is any tutorials that tell you how to do the cloth simulation part, and most of the tools directly combine meshes so you must take a detour to isolate clothes meshes.
My goal is to create a character that is from a MMD model and apply animations from Mixamo. The character should also have cloth simulation (in the end I used Magica Cloth which is a paid asset but relatively affordable).
Unity version: 2022.1.0f1
Using MMD Model: https://3d.nicovideo.jp/works/td31564
**NOTE: This Unity Wisdom does NOT treat third person control. There are lots of good tutorials you can find elsewhere. **
I. Prepare the model
So you have downloaded models and you think that maybe you can drag the folder into Unity and Unity will do the trick... Sadly, Unity does not recognize MMD models, and you have to figure out a way to convert it to FBX.
MMD4Mechanim
There is a plugin for unity called MMD4Mechanim that you can potentially use: http://stereoarts.jp/ . However this plugin has several issues: (1) It might not be able to convert materials correctly and you need to fix them manually. (2) Meshes are combined, you will not be able to easily simulate clothes.
In case you don't care about clothes, you can use MMD4Mechanim and here are some of the common fixes that can be useful:
Purple Materials (HDRP)
This is because the material uses a non-HDRP shader. Keep a mind note of the MainTex used by the material, and in Shader dropdown, find and select HDRP/Lit. Your material will no longer be purple, but the texture is lost. You will have to find the Base Map field in Surface Inputs section and click on the little circle just before the field name. There, you should select the texture which should be the same as the MainTex used in the MMDLit shader.
There might still be a problem... that is your material may be Transparent (by default it is set to Opaque). You might want to change that for some materials, and you probably need to use your sense to tell which should be transparent and which shouldn't.
The final problem related to materials, is that whenever the project reloads, materials will be reset to non-HDRP materials. This can be fixed by checking the Locked field in the MMD4Mechanim asset.
Using Blender
So, you decided to do cloth simulation... As far as I know, there isn't a simple button you can press to isolate meshes with MMD4Mechanim. You will need to rely on Blender to do the trick...
Why blender? Well, it's free.
Before you drag anything into Blender, you will need to install an addon called Cats (<=link) since blender does not natively support MMD related files.
By the time of this article, there is a bug with Cats that either doesn't convert materials properly or loosing bones when fixing the model. So what I did is just exporting two FBX, the first separates the meshes, and the second converts the materials. And in Unity, I extracted the materials in the second FBX and rebind them to the first.
** Update: There is already a fix to the bug, but it is not released yet. If you are using Blender 3.0+ you can download the latest Development version and problem solved. ( Import => Fix => Separate by Material ) **
So here is how I did that:
1. The first FBX that separates the meshes: Import the model using Import Model, and click Separate by: Materials in Model Options. Export the model.
2. The second FBX that converts materials: Import the model again, and click Fix Model. Export the model.
Now, to extract the materials (for the second FBX), you only need to click Materials and Extract Materials. Store your materials in a dedicated folder.
And then, for the first FBX, rebind the materials using the Remapped Materials section. Your model should be now good for the next step.
II. Retargeting Animations
For animations, I used Mixamo.
There are tutorials on how to retarget animations, and I suggest you to look for external sources. Here I only address some pitfalls you may encounter during the process.
Floating Character
If you see your character hovering above the ground, chances are, you need to adjust the offset in your animation. But first, check whether its your collider that is too big, and fix it if it is the case. Now, if your character is still floating, go to the problematic animation and find Root Transform Position (Y) -> Offset, and give it a positive value. You can adjust this value in play mode to have a visual feedback.
**NOTE: MMD models, or even Blender models, have different hierarchy structure than those of Mixamo, so copy avatar may not work. Instead, create animation from the Mixamo avatar and it will still work. However it may result in minor defects, and you can try to solve it manually... Or, you can purchase the Better FBX addon in Blender and it will give a better result.**
III. Cloth Simulation
Now this part will largely depend on your method of choice. I used Magica Cloth and if you read the documentations carefully you should not encounter any problem. So I will leave this section blank...