Making Blender Faster with Crowdrender V0.4.1
Watch this short video to see whats new!
Prefer to read? Then scroll down to see the highlights of the new version :)
Whats new?
Firstly just in case you want the software now -> https://www.crowd-render.com/download. I was going to say something about, but patience is a virtue, read my article... meh.
A different organisation of code and compliance with the GPL
First, an apology
They say that assumptions make an "ass" out of you and an "ass" out of me. We're dealing with a particular developer (james) being an ass here. Well, maybe thats a bit harsh. The issue here is that for a long while, the subject of compliance with the GPL was largely ignored. This was a bad idea in the long run, but since we were being peppered with requests for everything but making the addon properly compliant, the GPL requirements for Blender addons was given little thought if ever, and usually the conclusion was that we didn't have to comply, or we did already, for various reasons that suited us since it meant not having to do much.
However, that all changed when someone actually pulled us up on it. At that point we had to pay attention to this and it became clear that there really are no exceptions when it comes to addon code that is run in Blender. Any python code that uses Blender's python API is required by the terms of the license to be either licensed under the GPL, or a similar license that is compatible with the GPL's own terms.
There will be a follow up article on this to give the full juicy details of all the analysis that we did to reach this conclusion, but for now, rest assured, Crowdrender is GPL licensed, you'll notice this next time you glance at the contents of the addon package, if you are curious enough to do that :)
As a result there are some pretty big changes to our addon package. The main ones being how code is organised now, we've split the addon up into a python package that is mainly to do with Blender, and a core component that deals with all the stuff we do. This is partly to make way for CRESS, which will have a new structure similar to this where the crowdrender bits are no longer in a single python package alongside the parts specific to Blender.
Though this likely doesn't change much for you as a user, its an important step for us to be able to prove the new way of doing things works just as well. So far it does, so yay for that.
You'll also find plain source code, that you can read, and if you feel adventurous, you can change it too. New additions include license text files that explain what licenses the code uses. These are required by the GPL license and since we also use python, it couldn't hurt to reproduce their license file as well, so that's in there too..
We now use rotating logs to avoid excessive disk space use
Logging is an important part of ensuring we can help with problems when you have them (note I said when not if there). However, up until now, we've had logs that are able to just keep on growing in size. This is obviously not a smart strategy, so in this new minor update, we've made all logs fixed in size, so they'll only ever occupy a finite amount of space on your hard drive.
Connections made more reliable by having less of them
Murphy's law dictates that anything that can happen, given enough opportunities, will. This includes connections failing. So having less connections to make, means less opportunities for them to fail. We refactored the code that transfers files to have just one network connection. Though I am not personally guaranteeing half the error rate :).
Whilst this was happening, the performance of the code was slightly improved as well, resulting in a small 4% increase in speed. Whilst not huge, this is no doubt a good thing.
Stuff that got fixed
There were some pretty important fixes included in this update.
UnBreaking what we broke
In V0.3.0 we made a pretty huge change to the interface of Crowdrender, though there was certainly some confusion caused about the render still and render animation button being removed. The actual issue here is that there was a bug introduced that could cause a crash in Blender. Most of the time Blender would be fine and the render would start when you pressed F12 or used the render menu options. But occasionally Blender would just disappear on you.
This issue forced us to reconsider the design we'd originally used in V0.3.0, and a much better one arose as a result. So far we've not ever seen Blender crash like it was prior to V0.4.1.
Autopack also causing a crash
Its possible to set auto pack on an external file, or even all external files. This means Blender will automatically save that file into the blend file. However, its possible, with a bit of fiddling to get remove the file from the disk when Blender's sort of not looking, and then when you try to save your file in Blender, there's still an internal list of files that should be packed automatically, but now one or more is missing.
This results in an error being raised that stops the render from happening and has even caused a crash or two. So we decided to always ensure that we check for this, and prevent problems in crowdrender where we save copies of your file for rendering or sending to other machines.
Missing or blank/black render tiles
This issue has been a tricky one to solve, but we finally nailed it. It would manifest itself most noticeably during an animation render, several frames of an animation could be affected and would be missing part of the image.
As you can see in the image above, missing frames were frequent and very annoying. In V0.4.1, however, we're pretty much 100% confident that this problem has been fixed.
Our confidence is based on the fact that we discovered the reason why this was happening. To understand all this, you need to know how we break up and image into tiles in the first place. We do this using Blender's own border render feature. You can set a border in Blender and only render what's inside it.
As a user, you can set the border with your mouse while in the camera view point, for us developers we use Blender's python API.
As you can see, there are four numbers required which specify the border's position as its left, right, top and bottom edges. Each of these numbers is a float number between 0.0 and 1.0 where zero refers to the left or bottom edge and one represents the right or top edge.
Here's the cause of the problem, we use python in our addon, python stores its floats using 53 bits for the significand (on most machines, this implies a 64 bit process architecture and an operating system that supports using all 64 bits as well). However, unbeknownst to us, Blender's C code actually stores the render borders's coordinates as a C float. This creates the problem, C floats have quite a different level of precision to python's floats in probably most cases where a 64 bit processor and operating system are in use. This is because C floats use only 23 bits for storing the significand, which is less than half of the python float.
So, when we give Blender the coordinates for the render tile, in some cases the numbers we assign for each of the variables you see in the image above, are changed slightly because of something akin to rounding the number since it won't 'fit' properly in the lesser precision of the C float.
In some cases this change was enough to cause the size of the tile in pixels to change by just one pixel in width (we don't yet divide the image in the y axis so this isn't a source of the issue). This is bad since now our code thinks the image is one pixel larger or smaller than what Blender actually rendered for us. This blows up in our faces when we try to load the image back into Blender for viewing and compositing since Blender does not allow you to load a render result that is a different size to that which you tell Blender.
And so this is what was causing the problem. Now, the fix was really simple. There is a quote attributed to Einstein (probably fake, many inspirational quotes attempt to latch themselves to someone revered for being intelligent) that goes...
If I were given an hour to solve a problem, I'd spend 55 minutes understanding the problem, and 5 minutes on the solution.
(Probably not?) Einstein
So with the understanding that this was a precision issue, we simply changed the precision in our code to match that of Blender's internal C float, and viola, the problem disappeared instantly. A test was run that would not be able to make it ten frames in a row without a missing tile, but after this one change, over 500 frames were rendered consecutively without a single missing tile.
Issues still at large
There are still some problems we noticed that we didn't have time to fix for this release. You might think it odd we don't fix absolutely everything before releasing? Well this version is no doubt a lot better than V0.3.2 which it supersedes. V0.3.2 also had the issues described below. In these cases we always release if we can make things better. So we did. We'll be looking to solve these problems in a future update.
Multiple Render layers get rendered multiple times
If you have more than one render layer, you'll find that currently, blender, by itself, will call one render for each layer. But when in background mode, just one call to render your file will cause all render layers to render. Since Crowdrender uses background renders to fulfil requests for stills and animations, each render layer is rendered more than once.
There is a workaround though, you can split your render layers into multiple scenes. By creating multiple copies of your scene, each scene can then have just one of the render layers. Using the compositor you can then setup multiple render layer nodes to get each render layer to reference the layers you want from each scene and now you're back in business :)
Yeah, not pleasant I know, but we're going to look into this. The problems we're facing are that Blender gives us little control over how our rendering system is invoked, so Crowdrender actually gets multiple requests to render the same scene with different layers, but then we have to find a way to turn off layers we don't want to repeatedly render.
We're working on this one, stay tuned for another update.
Interrupting an upload will cause Blender to hang
If you are uploading to a node, and then you eithe revert, or open the same file, or possiblly a different file, then Blender can hang.
Its best if you either let the upload complete, or use the "-" (minus) button next the "connect" button (bottom of the crowdrender panel's list of render nodes). Highlight any nodes that are uploading still, and remove them by clicking the "-" button. Then you can do what you will without Blender hanging.
V0.3.1 now free
In case you're not aware, we're using a model of crowdfunding the development of Crowdrender, we hope to eventually make the current addon free for everyone to use once we reach a rather modest target of USD2500 a month in funding.
Each time we pass a funding milestone, we get close to all the versions of Crowdrender being unlocked. Also each time we release a new version, we also update the publicly available versions too. Since we just released V0.4.1, we've made V0.3.1 free as well.
V0.3.1 was a pretty important update as it made our addon compatible with the latest LTS version of Blender, which is v2.93 at the time of release of this article.
If you'd like to contribute to the development of Crowdrender, you can donate, either as a one off donation, or subscribe to support us with a regular monthly donation of your choosing and help us make distributed rendering easier for everyone :)
Click this link to support development of the Crowdrender addon for Blender - > https://www.crowd-render.com/crowdfunding
What's coming next?
Depending on our progress over the next three months that are left of the year, we'll be aiming to release one more minor/feature update, V0.5.0.
We're going to convert the entire core component to use event driven code which will allow for far more efficient operations, less lag in the UI when using the addon on large scenes, and generally more responsive and less error prone code.
