Rather disappointing with Harmony's 3D render integration/capabilities

Just watched the Harmony 14 webinar and must admit, I’m rather disappointed by ToonBoom’s implementation of 3D render integration and capabilities.

I did a search for in the forums for customers asking for more renderers to be added and some of the requests went back 7 or 8 years.

With Harmony 14 just being released, it appears that Toon Boom’s road map for 3D renderers will not be changing soon and all the great improvements about 3D inclusion will be of no use to its customers who are not using Maya.

It’s great you can import 3D models and animate them; but, it’s all for naught as you won’t be able to render anything that you do; unless you have Maya. After reading through the manual, it appears that Maya is the go through app for even Renderman. And who knows how to setup Pixie or 3Delight, as there is nothing in the manuals about them. All that is there is to contact support. Well, I did that a few years ago and they only said ask them if I run into any issues. Real help.

In thinking about this issue further and reading more, it appears that data (translation, scale and rotation) of imported 3D models and camera data is sent to Maya and opens up the scene in Maya. Maya then renders the scene and transfers back .iff images to Harmony and then Harmony composites those images. If that is the core of the interaction between Harmony and Maya, then, why can’t TB open up that part of it and allow users to write scripts to get this data into their 3D software of choice and render out their files in .iff or any other image format that can be imported back into Harmony to use. And, most 3D renders can output 32-bit floating images for .exr images for deep compositing, as well.

So, the question to TB staff/developers, is why is there no documentation on this, why is it so closed and hidden and why don’t you make this transformation of 3D data alot easier for your paying customers to use?

The only options available (according to Harmony manual) are Maya, Pixie and 3Delight.

I looked at Pixie, but there is not alot of documentation available for that project to be a serious roadway to venture down.

3Delight offers more documentation, but, after looking at the manual for it, it is very technical and needs to be setup in a very specific way. And there is no UI/GUI for the interaction for setting up the environment. It appears to be all command line based. Also, one would need to setup shaders and trying to figure out how to setup a toon type shader would be trial and error at best.

So, that only leaves Maya as the only option at present. Which, in my opinion, is not a viable option for many customers/independent animators. It basically is a Maya tax to use the 3D capabilities of Harmony.

In looking at the bigger picture, Storyboard allows FBX export. So, why can’t Harmony export FBX?? Afterall, one can import FBX into Harmony (even Alembic) and animate FBX models and the camera and image planes with 2D animated drawings. So, why couldn’t this environment be FBX exportable?? That way one could export FBX into a 3D software package and texture and light the 3D models in it and have the 2D drawings as image planes and render out a final image from there.

Or, why not have a node in the compositor that can read .exr files, that way, one could render with depth map and composite in Harmony and have the depth map alignment in the image??

All this falls down to ToonBoom to offer other alternatives for using features currently in the software to its customers without having to pay additional money to use those features.

i have been disgruntled about being force-opted-in to the 3d “features” since first buying in to harmony. i’ve never wanted or cared about this, as i am sure many other customers agree. it is not in the same league of essential features like the node editor that are only available in the premium version. the 3d toolset should be an add-on cost to any tier, as it remains useless for folks like you and i that don’t use maya. strange, tenacious stance for a product “feature” that honestly doesn’t even produce great results (in what i’ve seen), reminds me of after effects with their bungled internal implementation of ray tracing 3d years back.

Yes, it should be a plugin for those who want it; because, using the 3D animation aspect doesn’t pay off because you won’t be able to render that out.

The odd thing is that the 3D model will render from the camera view. Albeit, it’s probably taking the flattened image from the OpenGL view. So, TB could add a shader node in the compositor to link to the 3D model and (even if it’s just a toon shader) have that show up in the final image instead.

There are possibilities that could be done; but, it’s all up to TB to implement and make it a priority to allow independent animators who don’t have Maya to utilize all the features in the software that they paid for.

Just say this tweet on the blog:
#harmony14 brings forward many innovative new features with existing functions catered to our studio’s needs.” @JamFilledStudio

I guess TB is only interested in bringing innovative new features to “existing” functions if you are a studio with needs. When will we see a tweet by a single person that says something like that.

In looking further, it appears a basic GLSL shader would be able to toon shade a 3d object and not being a coder, I was able to follow along in a tutorial understand what they were doing and the code. So, I can’t see that it would be extremely difficult to write a node for the compositor and pipe in a 3D object and be able to render out in the final image or even in the OpenGL render view. After all, GLSL is OpenGL shader code; so, OpenGL is already there in Harmony.

toonboom should launch some kind of campaign to align more developers to make little tools and utilities for this software, covering things such as this i/o with other products like the “official” maya bridge. the (free) toonz software that is quickly gaining some traction as a harmony competitor is expanding rapidly with the community support/development behind it, it is exciting to see.

the depth and variety of scripts and complete plugins available for after effects from the community are very impressive. AE is not a fair comparison as far as market penetration, but the same principles apply. toonboom has provided at least some “hooks” into the software, but nobody is really providing public results or training to spark movement on it. devs don’t know that it exists. maybe some rogue animation studio developers can leak out some cool customizations to see what is possible?

it is exciting that toonboom included OFX plugin format support. i’m already a huge fan of harmony’s node network compositor, and OFX allows for (the possibility of) some really cool plugins to be integrated to the comp. network, but there aren’t enough plugin vendors out there that have compatible OFX versions. this is another area that could be boosted by more animation/vfx community attention. i contacted both toonboom and red giant regarding getting a “handshake” going for the awesome red giant suites, for example, but don’t think anything will come of that at this rate.

I totally agree. There should be a big encouraged movement by TB to foster scripting. It appears quite capable and may provide some way of data exchange to and from Harmony and other software. I’ve been searching the net for scripting information and stumbled on one forum entry mentioning some studio created a script to bring camera data from Maya (ick) into Harmony to be able to match cameras. If that can be done, then a script should be able to be written that can export out Harmony camera data, even if it is text based.

I’m hoping that via scripting one might be able to export out position, translation, scale and rotational data for 3D objects in a Harmony scene.

i am still waiting to get my upgrade info, but i noticed this:

i’m seeing this here:
http://docs.toonboom.com/help/harmony-14/premium/2d-3d-integration/import-3d-object.html

“The textures are incorporated in the .fbx file created in the conversion. By doing so, they appear in OpenGL display”

maybe that is the ticket? is this still part of the maya chain, and doesn’t actually get written out of the standard write module? from the description it sounds like it allows harmony to internally composite the textures/object. maybe i’m looking at it too out-of-context…

even if this isn’t what i hope it is, maybe the new updates like the deep compositor make it easier to open up this 3d integration with future/community dev suport?

No question there should be FBX or Alembic export. That would solve a world of problems with the current 3D workflow.

Here are the issues I’ve run into:

a) the back and forth with Maya /TB composite is extremely slow. So slow, that I could never use that workflow on a real project. Much, much faster to render files out of Maya and TB separately and composite in your app of choice.
b) Limited to Maya.
c) limited to supported renderers in Maya.
d) Deep compositing is nice, but the files are huge and overkill for a lot of projects. Simply allowing elements to be tagged w/ ID mattes as in most 3D apps would work for 90% of compositing needs.

All of this could be solved by simply exporting an FBX with camera info out of Harmony like Storyboard Pro does.
The simple 3D object animation new in v14 is also exportable to FBX or Alembic.

Jeff

As far as OGL render, it may have changed in v14 (haven’t downloaded it yet) but the in v12 OGL render is not production quality. I was told this by TB tech and my tests confirmed it.
I’ll download v14 and give it try. It would be nice.r

Hi ernesttx,

You are totally right when you say that it must be possible to use scripting to have Harmony interact with 3rd party software. This is exactly how the rendering of 3D models using Maya is done, through a script!

Harmony is actually quite capable when it comes to interacting with other software. Interaction can can be done using the scripting engine of using plugins created using the Harmony SDK.

Now, to get back to the rendering part, there are multiple ways to render 3D models out of Harmony. If you don’t have any 3D rendering engine, Harmony will use the 3D model that is in the scene, whether it is an FBX, Alembic or Collada file, and render the 3D model using the GPU. In some cases, that is enough for basic 3D integration. The other way is to use a 3rd party rendering engine by using a ScriptNode in Harmony. The script node allows the execution of qt script (java like scripting) while rendering a scene. You can refer to our documentation to know more about this node:

Script Node:
http://docs.toonboom.com/help/harmony-14/premium/reference/node/miscellaneous/script-node.html

The script node has access to the following functions:
http://docs.toonboom.com/help/harmony-14/scripting/scriptmodule/

When we first introduced the 3D integration in Harmony, way back in version 7.5, three scripts were bundled in the installation to allow rendering using either Maya, Pixie and 3Delight through a ScriptNode. There was no real demand for the last two and the scripts were not maintained, but are still bundled with the software. The Maya rendering still relies on this script which can be modified to meet specific needs. Integration with other rendering engine is also possible, and these scripts can be used as reference.

All these scripts are available in the resources/scriptmodule folder of your Harmony installation. The script node can call functions from within those scripts, like the MayaBatchRender script node does, or the script can be written directly in the script node.

Harmony support iff, exr and dtex image formats, so as long as the rendering engine supports one of these, Harmony will be able to read rendered frames from it.

If you have any questions on scripting, we have a scripting specialist in the support team that can help answer any question regarding scripting.

For more information on creating normal scripts for Harmony, please look at this page:
http://docs.toonboom.com/help/harmony-14/premium/scripting/create-qt-script.html

You can also look at all the scripts that are bundled with Harmony from the script editor view, or in resources/scripts. Remember that there is a difference between a script that can be executed during rendering using the ScriptNode and normal scripts. The render engine does not have access to all the functions, thus the ScriptNode only supports a subset, that you can see in the link to the script node scripting resource above.

Thanks,

– Marc-Andre

excellent post marc-andre, thank you!

Marc, thanks for the information. I had just ran across the Script module while looking at the documentation. However, all this information would be wonderful…if I was a coder and not an single animator. It’s great that Harmony has this capability. However, it does me no good. Basically, I now have to spend time learning to code, then read the specifications for the modules, try to write code, test, debug, test, etc. Months later (without animating anything because I’m learning to be a coder) I still may not have anything I can use.

Contrast that with TB providing basic scripts that allow for 3D data exchange already. A few simple scripts written by your script specialist that allow 3D camera data export or a script that exports out translation, scale and rotation for 3D objects. Might take them a couple of days versus my months of learning.

Not everyone has a technical director or coder available to them.

Thank you and in the meantime, I’ll be learning what function() does.

Yes, I don’t think OpenGL render is production quality. When I went to render or Export, I choose the Image from Layout option (maybe different as I’m not in front of Harmony at the moment). This exported out an image file. My 3D object was in it. However, it did not have a texture. But, the 3D object did not appear jaggy or aliased. It looked smooth. I will be doing a few more tests tonight with texture. I’m pretty sure though we won’t be able to setup any lights or position and rotate them for OpenGL. So, we may only get a “front” light that provides ambient light but no control.

I have not tried the OpenGL export option in Export yet. I’ll give that a try too.

as a workaround you could bake the lighting out of your 3D app into the texture. Not ideal, but might work for some cases. I’ll download 14 as well. Last time I checked AA was the issue.

I will probably just use videocopilot Element 3D with After Effects, I will render in Harmony a character Animation as a PNG sequence and import into After Effects and composite with a 3D Object by importing it through Element 3D, don’t want to be forced to use Maya, I already have Cinema 4D to make 3D models if I want.

True, baking the lighting in is possible. But, one hopes the shot is locked down to not have to change the lighting during final draft.

This is a crazy work-around, but I’ve roughed everything out in Storyboard Pro w/ 3D elements etc. and then exported the camera to both TB Harmony and Maya (could be any 3D app that imports FBX) The Storyboard Pro file then becomes the Master as far as scene blocking and camera movement. A lot of hoops to jump through but it works.

compositing a scene in After Effects with Element 3D is actually easy and doesn’t take that long, as long as you can make a scene look exactly how you want it doesn’t matter how you do it.

Why no Blender integration? There are many pros who prefer if to Maya.