Skin Data

Discussion regarding the decoding of the NetImmerse File Format and the Documentation System
No Active Developers

Skin Data

Postby elf » Tue Oct 25, 2011 8:55 pm

Hi

I was wondering if I could get some expert help, I had a quick search and couldn't see anything. I have also had a look at the NifSkope code and couldn't get the logic.

Ok so over to the question. I would like to understand from a psuedo code perspective how to render skeleton data. I'm using a morrowind example but would be good to know if there are any major difference with later versions of nif.

Test Nif : meshes/base_anim_female.nif (the male version doesn't work in NifSkope as it doesn't render the Triangle data)

What the Nif Contains :

It has a root node
It has 2 more root nodes, one bounding box and another called Bip01
Bip01 contains two NiTriShape and another NiNode which then has lots of child nodes representing a map of node and the leaf nodes are all NiTriData.

These NiNode's all look to represent the skeleton body (like bones in a body), each one has a rotation and translation set of data (and scale but always seems to be one).

The two NiTriShape's attached to Bip01 have a skin instance which then has skin data

What's the rule set here to determine if I should do anything special because theoretically I don't know it needs special processing till I have processed it. I don't know this nif has skin data till I get to the two NiTriShapes so I'm unsure of the general rendering rules here

So over to the skin data

The NiSkinInstance has a set of Bones, a Skeleton Root and SkinData (Skin Partition is null for my example, so not sure how this would apply to the processing logic). I have two of these with the same Skeleton Root, is this normal? The bones matches all the NiNodes which begin with Bip01, there are others like Groin, Tri Groin which aren't mentioned. I guess its because they make up the bits at the end of the bones.

The NiSkinData has a Skin Transform (always no transform in my example) and NiSkinInstance (again all blank for me) and a Bone List for every bone. Each bone list has a Skin Transform and Vertex Weights.

The skin transform looks obvious. Do these transforms override the translate and rotate in the actual bones? How do I use these?

The vertex weights have an index and a weight. What does the index refer to and what does the weight refer to?

At the end of bones in my nif there is a two objects, a NiNode and a NiTriShape. Do I replace these with the actual body parts when I work out what they are? Also the chest and hands get overwridden I guess you just replace the bones with the new bones.

Lots of questions, but it would be really nice to get some simple psuedo logic for how to process them. May something to add to the wiki after :)

Edit : I can get the basic skeleton to render now :D So removed the lines saying it was a mess (I had my glPushMatrix in the wrong place!!), but the question still remains as to what Skin data is and what it is meant to do
Last edited by elf on Wed Oct 26, 2011 7:10 am, edited 1 time in total.
elf
NifTools Developer
NifTools Developer
 
Posts: 74
Joined: Fri Mar 25, 2011 12:20 pm

Re: Skin Data

Postby neomonkeus » Tue Oct 25, 2011 10:13 pm

Hey Elf, you had disappeared before I had finished my install. I opened a copy of the ../meshes/base_anim_female.nif. I noticed something quite unusual. Under the Bip01 branch there is a NiTriShape called Tri QuadPatch 01. This has SkinInstance/NiNode pointers. I don't know why this is there but this may have some baring on your problem or might at least point someone in the right direction.

Equally ../meshes/base_anim.nif and ../meshes/anim_dancinggirl.nif have the same dataset so it appears that the data is correct but perhaps it is displaying correctly either in Nifskope?

I don't know if this was the male.nif you were referring to but it rendered for me fine and if it not I can check to see if the right one does render. I am using v1.0.22 rev8329d39. Again this could not be of any relevance whatsoever but who knows :D
- Please post bugs and feature requests on the forum.

Latest Blender Nif Plug-in News
Blender Nif Plug-in - Dev Version
Blender Nif Plug-in - Dev - Milestone/Issues

A diffuse map is a piece of paper which tells you where the bomb is so that you can diffuse it.
Please avoid using the .obj format, it usually ends in tears :D.
User avatar
neomonkeus
NifTools Developer
NifTools Developer
 
Posts: 1463
Joined: Wed Jul 14, 2010 12:14 pm
Location: whatever couch I can find, Ireland

Re: Skin Data

Postby elf » Wed Oct 26, 2011 4:34 am

Neomonkus

Out of the three meshes you mentioned base_anim.nif doesn't display the same thing in nifskope as the other 2. Also the Tri QuadPatch 01 and Tri Shadow don't have any baring on my problem. If you render Tri Shadow is just a purple outline of a body, the Tri QuadPatch 01 is just a wire frame of something else.

The shadow and the quad are a bit odd, when it renders you see the shadow object too as well as the skeleton and the quad is a little odd too.

The plan is to eventually take the nif representing the Head and replacing the NiTriShape data with NiTriShape data from that Nif (and the same for all the body parts). The only nif's which don't follow this pattern are Chest and Hand nif's where they have more skin data present (which I guess over rides the base)


Elf

p.s. The difference between the base nif's is the animation sequence
elf
NifTools Developer
NifTools Developer
 
Posts: 74
Joined: Fri Mar 25, 2011 12:20 pm

Re: Skin Data

Postby amorilia » Sat Oct 29, 2011 9:30 am

As another point of reference, pyffi has code for calculating the final positions of vertices, with skinning:

https://github.com/amorilia/pyffi/blob/ ... _.py#L4678

In essence, you need to multiply a lot of matrices together. Quite possibly, OpenGL has some functions to do this stuff "automatically".
User avatar
amorilia
NifTools Developer
NifTools Developer
 
Posts: 3807
Joined: Sat Oct 08, 2005 3:58 pm
Location: Desele's House of Earthly Delights.

Re: Skin Data

Postby elf » Sun Oct 30, 2011 12:23 pm

Amorilia

Thanks, at the moment I can render the base skeletons and I can partially render the skeleton with the body parts applied but I don't actually use any of the skeleton data yet. The only bits I can't render are the chest and hands as they seem to have skeleton data themselves (but should be able to fix this soon). I think the only things I am missing by not using the skeleton data are the weights.

The way I currently do it is to have a mesh which can have an array of sub meshes and so forth, as I go down the mesh list I only render the ones with no children themselves but apply the transforms of the predecessors. The base skeletons seem to link everything together fine. As well as not using the weights I don't need to use the transforms so I'm not sure what I am missing out there.

Elf
elf
NifTools Developer
NifTools Developer
 
Posts: 74
Joined: Fri Mar 25, 2011 12:20 pm

Re: Skin Data

Postby amorilia » Sun Oct 30, 2011 4:07 pm

Ah yes: in principle, you only need those weights and those transforms when you start animating the skeleton.

(Aside, in my experience, Morrowind is one of the more problematic games as far as skinning is concerned - in fact, none of the import plugins can handle all of Morrowind's corner cases. Oblivion's system is much easier with a single skeleton root being used consistently between all body parts.)
User avatar
amorilia
NifTools Developer
NifTools Developer
 
Posts: 3807
Joined: Sat Oct 08, 2005 3:58 pm
Location: Desele's House of Earthly Delights.

Re: Skin Data

Postby elf » Sun Oct 30, 2011 4:15 pm

Amorilia

Playing around a bit more its seems my method is quite as good as I first thought, the feet render as two right feet (not sure if this is normal), and the head is not in the right rotation.

I've had a look at the python code but its hard to get the underlying logic

Using the base_anim.nif from morrowind as an example

The root node has two children nodes
One is a bounding box and one is another node
This other node has two TriShapData objects and another node
This node continues with all the bone nodes and the two TriShapeData objects have Skin Data. One is a body shadow and one is random set of quad data, both have a flag saying don't draw.

Question : is having two skin objects ok? What order do you try and process them? Does one overide the other?

Question : The Skin data has transforms, do these over ride the transforms in the bones themselves (as those transforms seem to give you skeleton looking like a body) or are they used on top of?

Question : the vertex indexes in the skin data point to which vertex? The dummy ones in the base_anim.nif or the vertices I add to get the race specific look and feel?

Question : The weights just look like scale, do they just replace the scale in the bones /skin transform?
elf
NifTools Developer
NifTools Developer
 
Posts: 74
Joined: Fri Mar 25, 2011 12:20 pm

Re: Skin Data

Postby amorilia » Mon Oct 31, 2011 7:13 pm

The transform errors may be due to ignoring the skinning data (reason being that the engine will never render those objects without it). Usually you don't need it, but I've seen cases where one does (whence the many bind pose transforms being done by for instance the blender nif scripts, through pyffi).

Ignore the quad shape - it seems to be some kind of testing plane added by the artist and its skinning is screwed up.

elf wrote:Question : is having two skin objects ok? What order do you try and process them? Does one overide the other?

Every block can only have one skin instance, so I'm not sure what you mean...
elf wrote:Question : The Skin data has transforms, do these over ride the transforms in the bones themselves (as those transforms seem to give you skeleton looking like a body) or are they used on top of?

On top. The way this works is described in the comments that goes with the function I linked to before.
elf wrote:Question : the vertex indexes in the skin data point to which vertex? The dummy ones in the base_anim.nif or the vertices I add to get the race specific look and feel?

The indices always point to the underlying (local) NiTriShape.
elf wrote:Question : The weights just look like scale, do they just replace the scale in the bones /skin transform?

Not really. Again see the details described in that python file (weight[v][b] in particular).

I know, it appears all very confusing at first... but eventually it's just about multiplying the right matrices together.
User avatar
amorilia
NifTools Developer
NifTools Developer
 
Posts: 3807
Joined: Sat Oct 08, 2005 3:58 pm
Location: Desele's House of Earthly Delights.

Re: Skin Data

Postby elf » Tue Nov 01, 2011 7:46 am

Amorilia

In the male version of the skeleton there is a Tri Shadow and a Tri Quadpatch01 object but these are both hidden (The female version they aren't). When I said 2 Skin data's I meant one for Tri Shadow and one for Tri Quadpatch01. Do you have to apply both? Which one is the real one?

When you say the transform is done on top (the comments in the function weren't a 100% clear to me), did you mean that you use the rotation and translation in the NiNode object (representing the bone) the you add the translation, rotation of that bone in the skin transform after or before? ie do you rotate/translate the bone then the skin or skin then the bone? Also the NiTriShape like "Tri Groin" and NiNode like "Groin", are these important as they aren't bones? Is the rotation and translation of this important? or do I ignore it essential and when I get the groin NiF I just replace Groin with the data in the new Nif? (So it was never there essentially).

You say the indices point to the local NiTriShape, by local do you mean the one in he skeleton nif (base_anim.nif for example)? If so these vertices are never used right? Or do you mean the seperate nif representing the body component which you actually render?

When I said the weights look like scale I meant it by looking at

vertices[index] += weight * (self.data.vertices[index] * transform)

I know its per vertex rather than per bone but the concept is similar in my head.

I'm getting there slowly on the understanding, to me this is the most complicated bit of the NiF so far

Elf
elf
NifTools Developer
NifTools Developer
 
Posts: 74
Joined: Fri Mar 25, 2011 12:20 pm

Re: Skin Data

Postby amorilia » Wed Nov 02, 2011 9:54 pm

Shadow & Quadpatch: technically, they are "both" real, but only the shadow appears to be used by the engine.

The transformation is:

  • take vertex coordinate
  • apply "inverse bone matrix" (this is the bone transform matrix stored in the skin data block - it is already stored in inverted form)
  • apply bone NiNode transforms all the way down to the skeleton root
  • apply the "inverse skeleton root matrix" (this is the global transform matrix stored in the skin data block - it is already stored in inverted form)
  • apply the NiTriShape transform, and so on, until the root of the nif tree

Local: yes. Indeed, the skin data in the skeleton is never actually used (except for the shadow). Skin data always matches the geometry of the NiTriShape block it belongs to.

Weights: ok, looks good - transform is what I described above.
User avatar
amorilia
NifTools Developer
NifTools Developer
 
Posts: 3807
Joined: Sat Oct 08, 2005 3:58 pm
Location: Desele's House of Earthly Delights.

Re: Skin Data

Postby elf » Sat Nov 05, 2011 11:10 am

Just to be clear on a few points

take vertex coordinate

This is fine. This is the vertex coordinate of the NiTriShape (ie body part I want to render)

apply "inverse bone matrix" (this is the bone transform matrix stored in the skin data block - it is already stored in inverted form)

So in the base anim example, I have a NiNode with the name Bip01 R Calf. There is also a bone in the skin data with this name. You say to take the transform of the bone in the skin data block here.

apply bone NiNode transforms all the way down to the skeleton root

Do you mean all the ones not in the skin data? From where in the skeleton root do you start this process? If I take the Tri Right Knee, it has an NiNode Right Knee. Is this the point to start heading to the root node(so eventually I do another transform on Bip01 R Calf but from the skeleton data in this pass)? It sounds like most of the bones in the skin data are ignored as you only take the bones closet to where you want the body part, is this right?

Why do you start from the body part and head to the root and not the other way round? I kind of see the skeleton as something you start from point (0,0,0) and build up the skeleton from there.

apply the "inverse skeleton root matrix" (this is the global transform matrix stored in the skin data block - it is already stored in inverted form)

Ok, this is fine. They are all 0 in morrowind but understand the generic idea.

apply the NiTriShape transform, and so on, until the root of the nif tree

This is fine, I don't know of any nif so far where there are any transforms above the NiTriShape.
elf
NifTools Developer
NifTools Developer
 
Posts: 74
Joined: Fri Mar 25, 2011 12:20 pm

Re: Skin Data

Postby amorilia » Sat Nov 12, 2011 10:06 am

Sorry elf for the late response. All correct, except:
elf wrote:
amorilia wrote:apply bone NiNode transforms all the way down to the skeleton root

Do you mean all the ones not in the skin data? From where in the skeleton root do you start this process? If I take the Tri Right Knee, it has an NiNode Right Knee. Is this the point to start heading to the root node(so eventually I do another transform on Bip01 R Calf but from the skeleton data in this pass)?

You would for instance start from Bip01 R Calf (or whatever node is the bone influence), and then go down the hierarchy until you arrive at the skeleton root. Right-multiply with each NiNode transform as you go (except the skeleton root transform - this is dealt with later).
elf wrote:It sounds like most of the bones in the skin data are ignored as you only take the bones closet to where you want the body part, is this right?

I don't understand here what you mean exactly, but yes, for each bone influence, you only need a particular branch of bones to calculate its contribution.
elf wrote:Why do you start from the body part and head to the root and not the other way round? I kind of see the skeleton as something you start from point (0,0,0) and build up the skeleton from there.

Sure. Left-multiply and reverse the order of all operations. It's equivalent.
elf wrote:
amorilia wrote:apply the NiTriShape transform, and so on, until the root of the nif tree

This is fine, I don't know of any nif so far where there are any transforms above the NiTriShape.

Well, usually, this transform cancels out the penultimate transform (i.e. the global transform matrix in the skin data is exactly the inverse of the NiTriShape transform relative to the skeleton root). Sometimes it is not, and this covers these few odd cases (or so is the theory). I guess you could get away, for the most part, with ignoring the last two steps, and simply transform all the way back to the scene root. In fact I think that's what nifskope does, IIRC.
User avatar
amorilia
NifTools Developer
NifTools Developer
 
Posts: 3807
Joined: Sat Oct 08, 2005 3:58 pm
Location: Desele's House of Earthly Delights.


Return to Decoding the NIF File Format

Who is online

Users browsing this forum: No registered users and 1 guest