Sado Rabaudi
Reallusion allows you to create and animate production quality avatars using a wide range of tools such as the ability to use AI to create digital faces from one photo, animating in real time using your Iphone or webcam and accessing a vast digital library of resources from their online community as well as Daz3D. Combining this with Vizard allows you to utilize Vizard’s range of VR research tools, such as adding eye trackers, data gloves, displaying on VR projection systems, collecting data and much more. Here is a quick tutorial on how you can bring these high quality avatars into your Vizard application.
Scroll to the bottom of this post for a video version of this tutorial.
If you want to either modify an existing character or create one from scratch you would start with the Reallusion program “Character Creator”. One highly useful tool for creating custom avatars is the “Headshot plugin” that allows you to create a fully animatable avatar using a single photo.
Learn more at reallusion.com
Simply click on the “Headshot” tab then drag in your photo and click “Generate”. See this detailed tutorial from Reallusion on the process.
You can then add and adjust hair, clothes and other props to complete your avatar. The next step is to select “Send character to iClone to add or create animations.
In iClone you can choose to add poses, animations or create animations in real time using the Motion Live plugin. You can create facial animations using your Iphone using Live Face, or a webcam using “Faceware”. There are also plugins for various tracking systems such as Xsens, Optitrack and Perception Neuron to create full body animations.
The next step to bring your Reallusion avatar into Vizard is to select “Export FBX” and choose the preset “3Ds Max”. Select all frames to export all the frames of your animation and keep the other settings at their default.
Now open up the .fbx you exported into Vizard’s Inspector Tool (Tools- Inspector)
Make sure to toggle “Effects” on. This is necessary to show shader effects. Disabling the option shows the avatar in legacy mode.
First, check the size of your avatar. Vizard uses meters. A good general size is to have the ‘Y” value be around 1.8 meters.
Model Scale
Use the “Scale” tool to scale your avatar, making sure that the topmost node is selected in your scene graph. This should be the one that contains the avatar’s name.
Also, you will need to rotate avatars coming from Reallusion into Vizard, as they come in facing the opposite direction of the standard avatars. To do this, click on the root node and select the rotate tool, then rotate 180 degrees in Yaw.
Lighting
If the avatar looks flat, you can add a temporary light from your current view direction by pressing Ctrl+L on the keyboard. This preview light is not stored within your scene.
It's also possible to add permanent lights within Inspector, but in most cases you'll want to remove them before using the avatar so they don't have a light following them around. You can temporarily bring this same avatar into your environment in Inspector to adjust light in the correct context. To add a light in Inspector, go to Create > Light and choose which light type you want from Directional, Spot, or Point. You can further adjust the light in the Properties panel on the right. The same method can be used to add lights to an environment model, and they can also be added through code when necessary. For more on adding lights in Vizard, see the official documentation. You can see how lighting can make a big difference on the look of your avatar.
Transparency
Overlapping transparent objects can often have draw order artifacts. If the hair looks odd at certain angles, toggle on “Alpha to Coverage” and toggle off “Blend” and “Cull Face” for pieces of the model. Changes apply to all faces sharing the same material. If those options are not showing you can choose them by clicking on “Modes” and using the dropdown menu.
For some models you may also need to play around a little bit with the “shininess” value as well to get the hair to look how you want it.
Animations
Any animations that are included with your avatar will be shown in the bottom left panel under “Animations” . You can also add more animations by selecting the “root node” and then right clicking under “Animations” and selecting “import”. These animations are accessed in your main script using the “state” command.
female = vizfx.addAvatar('Avatar.osgb')
female.state(1)
See this page in the Vizard Documentation for some more information on using avatar animations.
Facial Morph Targets
Under the “Animations” on the right panel, you will see the facial morph targets. Click to see the full list and drag the slider to see the effect. In the Vizard code you will reference the morph ID if you want to utilize them in your scene. Use this code in Vizard to set a Morph (where “23” is the morph ID, 1 is the value and time is in seconds). See this page in the Vizard documentation for more.
morph = vizact.morphTo(23,1,time=2)
Setting up the lighting in your environment
Open up the environment model you wish to use in your application and use File-Add to bring in the avatar and adjust the lighting if necessary in the context of the scene (you will then remove the avatar from the environment model and add it back in code (by right clicking and choosing “delete”). Experiment with adding and adjusting lights, toggling on “Shadows”, adjusting the color of the lights and lastly, you can click on the “Scene” node to do things such as adjust the overall exposure and ambient lighting color (make sure to check “Save Scene settings”). Use the translate tools to place the avatar where you want it in the scene (you can then copy and paste this position into your code).
Note that for your lighting to look best, you should disable the default headlight in Vizard with the following code:
headlight= viz.MainView.getHeadLight()
headlight.disable()
Saving
Now save your avatar, making sure it is saved in the same folder as your Vizard project. If you haven’t already, create a new folder to save your project in. The avatar will be saved in Vizard’s default format “OSGB”.
Adding to Code in Vizard
Create a new Vizard file by going to File- “New Vizard File” and add the following code to bring in your avatar. Note that “vizfx” is needed if your avatar uses shader effects
Here is some more detailed code that also adds an environment, removes the default headlight, triggers an animation state, uses “setMultiSample” to increase the antialiasing, and animates the head to follow where the user is in the scene. Note that the coordinate system comes in a bit off from Reallusion, so when you may need to add this code to the neck bone to fix that:
neck.setEuler([180,-135,0],viz.REL_LOCAL)
The environment model I used for this was this one from Sketchfab (download as glTF). I added a few lights (one directional and two spot lights) in Inspector also and brought the exposure down a bit.
See Also
For more information on using avatars in Vizard see this page in the documentation. Additionally, if you wish to use a Reallusion avatar as a tracked avatar, see this tutorial on using a custom 3DSMax script to do that (requires 3DSMax).
Video Tutorial:
