v4.5 Crashes during export of flash movie

V4.5 is much better than V4.0 with handling large projects and files. Kudos for those improvements. But now I have a project that crashes on export of a 2300 frame scene to flash. Essentially the program runs out of memory.

The project exports fine to AVI and Quicktime, only flash is a problem.
The export gets to 76% and crashes (it would be better to give an error message). If I export just 1200 frames, again the project crashes at about 86%. At 1000 frames it gets to 98% and crashes! (after about 1hr).

I’m now try to optimize the scene. Any suggestions welcome.
- Bulk of characters are vector drawings that I’ve individually tried to optimize or simplify (all less than 22kb, many at 10k)
- Few vectorized bitmaps (I’ve made them as small as possible, about 20k)
- There are shadow and color transitions (but I’d like to keep them)
- Tried OpenGL and other rendering options
- Tried not exporting flash preloader and textures
- Includes one audio track ‘cut’ to duration of scene

Does ToonBoom support /3Gb switch in boot.ini file?

Windows XP
Intel Core 2 Duo Quad Q6600
4 Gb RAM
8800GT 512Mb


I can see a couple of things you could work on. First if you had been using pencil lines in your projects try to convert as many as you can to Brush. Pencil take more memory to render out in the end so you could save there.

Concerning the vectorized BG the important thing there is that you need to have them in the lowest resolution you can allow. A bitmap of 5k weight but 10 000 pixel by 10 000 pixel takes probably hundreds of megs in the file directory. The reason behind this is that when you import and vectorize an images it uncompresses every pixel (since it is a vector the data can’t be a compressed bitmap) which by consequence creates huge files out of very tiny sized images that have high resolution.

As for the effects if you can try to avoid the shadow as much as possible for swf since that one is specifically heavy on render. You could work around that by cloning the element and process it through a color transformation module to make a shadow out of it. Then use a peg to create the shadow (may not work in all situation but still something to consider).

For the 3G we did not make any specific things to support it so I don’t think it would be taken in consideration though I guess it is worth the try.

Also if you have a chance try to make small scenes over long scenes (and maybe put them back together afterward) to allow the software to clear its cache between each renders.

Hopefully this will help you out.

Best regards,


Hey Thanks Ugo

I’m going through every option at the moment. Any option considered.
Doh, I thought that Pencil was better than brush. I used the Line Tool most of the time - I’m assuming that’s efficient? (In many cases only 2 vector points). I’ll have to try to work out where I’ve used the pencil tool…

I’ll check the PNG files I imported - I think I made sure the resolution had the minimum number of pixels (after issues I had with v4). I’ll double check the files again.

I think I’m going to switch off the shadows and see what happens. If it works like this then I’ll try to use a cloned element and test again. (Doesn’t look so good without the shadows though…)

If ToonBoom hasn’t been compiled with the correct header, it is likely to be unstable with the boot.ini 3Gb switch. (It could also make the whole PC unstable, so I’ll give this a go as a last resort)

Unfortunately I can’t break the scene down into separate parts…there’s also a background audio track. I don’t know how I could ‘glue’ the separate swf files together…maybe Flash would help.

I’m also taking a closer look at the texture files mapped to palettes…

Does ToonBoom support Vista 64-bit…(very last resort…!)


Concerning the pencil even though it seem to be a single line when you draw this is actually need to process 2 lines, the center one and the outlines where for brush what you see is what you get, only the outline.

As for Vista the software will run on the 64bits one but is not yet optimized for it so you won’t get any specific benefit from getting such OS.

Best regards,



Don’t suppose there a global “change all lines/pencil to brush” option is there?

When you say there is no optimization for Vista 64-bit are you saying it won’t be able to access the extra RAM? (That’s the main thing I was thinking about)



Unfortunately there is no tool to automatically do all the conversion. As for the Ram yet it will, what I meant is that Toon Boom Studio is not a 64 bits software (basically it still runs under compatibility mode, it should have access to all the resources the OS can provide).

Best regards,


Hi Ugo

After some research…

I believe the that if ToonBoom runs in compatibility mode on Vista 64 it will be running on what is essentially a 32-bit emulator (WoW64). This might actually make the app a little slower.

Without changes to the system, ToonBoom, like other apps, will only be able to access a maximum of 2Gb memory by default. This is regardless of the RAM available.

In Vista 64 bit only you can make it possible for an application to access up to 4Gb RAM. To do this you need to:
(a) Enable Memory Remapping in the BIOS (if you BIOS supports it)
(b) Change the Application preferences to use 4Gb.

However, a number of applications (Photoshop) will still only be able to access a maximum of 3Gb of RAM.

Ugo - Has anyone at ToonBoom tried Studio on a Vista 64-bit system? Any idea how much memory it has access to?



I am not sure any stress test were done to define the maximum amount of memory the application can use under 64 bits OS though I know we have such machine over here.

Though as you mentioned it may still block under the 4GB you have since the software still runs in 32bits even if it is under 64 bits OS. In the end you should be better off with a 32 bits OS for the moment.

Sorry for the confusion.



Hi Ugo - Help!!

I’ve now tried all the suggestions. After trying to optimize every vector point and pixel I can get Flash export to 100% (1.85Gb used)…at which point TB thinks about it a little while, and then post a message saying:
“Unable to generate movie: No enough memory”.

Any other ideas?
Can a developer at ToonBoom give me any ideas on how ToonBoom exports to SWF…etc.
(a) What consumes the most memory? I’m guessing its fast movement of texture maps? Color gradients don’t seem to affect the size of the vector image but do they affect the memory required for flash export?
(b) Does the export process stream all images (png snapshots of each frame?) into memory and then compile them into the final SWF? Or does it compile as it goes along? If it is doing the work as it goes along can’t it flush out the old frames/images from memory?
(c) How much free memory does TB need after export in order to create the final Flash file? Ballpark number is ok. (I want to know if I’m close).
(d) Is the export performed using the graphics library in Panda3D?
If so, Is there anyway to access the performance monitoring and tweak options via the command line. (libpanda.dll seems to be known as a memory hog)
If not, what do you use? Can I tweak it?
(e) Any other option (maybe Quicktime to Flash converter)?
(f) The CPU only hits a max of 25% utilization? The export process does seem to spawn 3, and at times 4, threads so each core has stuff to do. But why only 25%? Where’s the bottleneck? The code seems to be using the CPU very inefficiently (creating spikes and then releasing).
(g) Most of the time TB loaded the project and took up about 134Mb initially. It then increased memory use as time went on. On one occassion TB grabbed 1.4Gb, and only increased the memory after about 56%. Why the strange memory management?
(h) Are there any guideline numbers on the maximum project file size, together with max size of Textures/Drawings folder, that will prevent a Flash export? Might be a useful thing to know.

Here’s what I’ve done (and the results)
(1) Upgraded to 4Gb RAM, shut down all other processes.
Nonpaged kernel using 40Mb. TB has full 2Gb max for it to use.
(2) Reduced resolution of imported bitmaps to absolute minimum (to be honest some are actually a bit blury now). Also cropped texture files to minimum. The Texture folder was 1.61Mb. Now it’s 323Kb.
(3) Optimized every vector image (most are only a few Kb in size).
I tried converting from line to brush. Even after optimizing the vectore points on the line the images ended up about 20% larger. If I used smoothing the image detail became too low to be of use. The Drawings folder was 230Kb, now it’s 198Kb.
(4) Tried different rendering options OpenGL and Direct3D etc.
Not sure if these factor in to a Flash export at all, but I gave it a try. No obvious difference.
(5) Changed from 30 frames/second to 12.
Threw off the keyframes (thanfully didn’t change/delete anything from timeline). Seemed to help quite a bit.
(6) Looked into /3Gb switch - won’t work. OS unstable, TB not compiled with header in order to recognize extra RAM. Looked into extending memory map in BIOS (won’t help in 32-bit). Look at 64-bit OSs. By default still only 2GB RAM per app. Might get up to 4Gb by changing emulation preferences (not sure if TB have actually checked how much memory TB can access…Photoshop can only access max 3Gb, even on 8Gb system).

Thanks…for any ideas (I’m out of them at the moment)

Making the texture maps smaller resulted in less total memory being used (no surprise there) but also in smoother progression in memory increase (previous project had small but noticable stepped increases in memory). That seems to imply TB is loading the images frame by frame; big texture map, big increase in memory.

Increase in memory was not linear during export (other wise I’d be able to predict when the export would fail or not).

Sometimes when ToonBoom starts up after a few attempts (and failing) to export to Flash, it grabs up to 1.2Gb of RAM! No idea why it grabs so much virtual address space.


Here is what I know about this:

(a) For sure what takes the most memory would be texture. Then you would have gradient and flat colors afterward. This being said texture are quite more consuming then the other two.

(b) I am pretty sure everything is pressed and in memory before the file is compiled. Therefore if your final output file size exceeds the total amount of memory the software can be allowed to have it will crash for sure (which explains the 1.85 Gb used crash) In such a case you may want to export your project in multiple little chapters that you can patch together afterward. This being said if your project is 2 Gig wide I seriously doubt it will be possible to see it streamed on the web so you might want to go for alternate solution in such case.

(c) Since the file is compiled on the fly I would say as much memory as your output file size would be (most likely more if you have big images since you can only fit those in contiguous memory spot on Ram which may end up wasting some holes if no image fits in the holes left (memory allocation is not done in a continuous way, it is rather random)). Also, if you have lots of bitmap I would recommend you to go for a Quicktime/Avi output for the swf file will most likely be quite bigger then the bitmap format because bitmaps in swf have no compression at all.

(d) Export is using pretty much only physical memory of your machine. You can monitor it by checking your Ram and Virtual memory of your machine. The way it is done is that it will take ram up to the point it no longer have the right to do so (if it ever reach 2Gig(which is Windows XP limit for a single application)) then it will start swapping to virtual memory. You may want to increase the amount of Virtual Memory on your machine for it may help if you are close to getting the file completed.

(e) I don’t think you could fine a flat Quicktime to Flash for bitmap to vector is somehow complicated. This being said you probably should be able to find some Quicktime to FLV converter and use those flv’s within a swf afterward. You probably will get the best out of this if you movie contains lots of bitmaps.

(f) Processing speed is not a critical piece for the rendering. Having a more powerful processor will allow you to speed up the render but the Ram is where the file is stored so regardless of your processor if the thing can’t fit in Ram/Virtual memory the render may not be able to complete. The reason why it spikes and release is that processor will create the image (spike) and send it to ram for temporary storage (release) then take a new frame and so on.

(g) Is this when you proceed through render? If that is the case the answer to that is above. This being said if it is when you are actually working on the project I would need more information about it for I don’t think the software should get that much memory usage when proceeding through creation tasks.

(h) For SWF a conservative way to do thing would be to go for a maximum of 1000 frames per exported files. For sure if you are using lots of bitmap this may need to be reduced but for usual vector export that should be pretty much the limit. If every frames is 1-2k of vector a 1000 frame project is going to take at least 2 Gig of physical memory which will leave enough free space for Virtual memory to take over extra files that may not fit in the Ram itself.

As for what you have done so far every of those things you did are the proper thing to get things done. Hopefully the information above will help you figure out what is going on and complete the export of your project.

Best regards,


Thanks Ugo
Fast response and lots of details.
I’ve been doing more experiments and have more results…and, I think a workable solution…

(a) Understood
(b) I originally thought I could do all this in ToonBoom. Originally wasn’t sure how I could edit the swf files together after export from ToonBoom, at least without buying extra software or having jerky frames between sections (I don’t have Flash at the moment…otherwise I might even be using that for the some of the animation…). I think I just need to be a bit more creative. If you look at the results of my experiment below you’ll see that the resultant flash files are no where near 2Gb.
(c) Good to know. Thanks.
(d) In 32-bit OSs the max memory per app is 2Gb (memory address space). That includes RAM (physical) and any disk-swap memory (virtual memory, pagefile). With 4GB ToonBoom will use all 2GB in RAM - it won’t and can’t take any more memory. Asking around with people who have Photoshop on Vista 64-bit…and the maximum memory they can get it to use is 3Gb. Not much of a gain when you consider the other overheads of 64-bit OS.
Just and FYI - my disk-swap space is fairly large and on a separate HD. With 4Gb RAM - it really doesn’t get used much.
(e) See experiment below
(f) Actually this question was really about what the bottleneck was. TB has access to all the RAM it needs and 4 CPUs, but is only using the CPUs at 25%. Why doesn’t it run the CPUs up to 80-90%…what’s slowing it down. (See experiment below to see what I mean)
(h) Good to know a ballpark figure…but 1,000 frames is …kind of small. But I think I have a solution.

I took my original project (un-optimzed) and created a Quicktime movie (high quality, 30 frames/second). (There’s a reason why I want to have the option to have an output so large…and it’s nothing to do with the web). File is about 200Mb. ToonBoom does great job producing this. Looks fanstastic. (Again, CPUs weren’t being strained at all…maybe room to make it faster).

Downloaded trial of Sorenson Squeeze. Used it to convert the Quicktime movie file into a SWF+F4V file (again at the same, high quality settings and large resolution/frame rate). Infact, I had left ZoneAlarm, Creative Media player, WinTV all running…as I expected it to choke.

All 4 CPUs went to 80% load and it was done in less than 5 minutes. (In other words, it was efficiently using the CPU resources available rather than just running at 25%) I had planned a walk to get a coffee, but the darn thing finished before I had my coat on. The memory use (for complete system, OS and all) didn’t go above 700Mb. The final file looks the same as the Quicktime movie. Total file size (SWF+F4V) was 15Mb. On lower res they still look great but file drops to low Mb range.

I tried other Flash output formats with similar results.

I even tried an old 2.5Gb avi file I had lying around. Sorenson ripped through that even quicker, with a very small flash file (again, about 15Mb). Again memory for whole machine as below 800Mb.

The only problem is that Sorenson cost $200+. Nolan Scott gave a couple of really good suggestions on alternatives. One looks really good, but one is Mac only, and the other, I don’t think, outputs to Flash (but I’m going to download anyway - looks good for iPod).

So I kind of have a solution. In my other tests I found out that spending time trying to optimize every vector image and texture doesn’t have a real impact on the size of avi or mov file. I’m not sure it even speeds up the export. In other words, you don’t get much gain for the effort. You might as well just be lazy about it and save your time.

ToonBoom v4.5 is much more robust than v4.0 and keeps project sizes much smaller. That means I can create scenes that are relatively long without problems. That’s true even with fairly large textures (again, kudos to the ToonBoom designers). If I want to export, I can just use whatever is the fastest (avi or Quicktime), edit with a video editor (if I need to) and convert the avi or mov file to flash (if I need to) using a Sorenson equivalent.


Guess you figured out a solution. If you want any specific answer still let me know.



I still have to buy another piece of software…
And I’d still really, really prefer to do it all in ToonBoom… :wink: :wink: