Hi @Arnar Helgi Adalsteinsson and @Balduin Allrgg :) Ok, we've finally got to the bottom of this and think we have fixed it in the pre-release version of v017 which we want to release really soon. I would very much like to offer both of you the chance to test it now, however, so that we can verify that you can both use your GPUs.

The issue was that, we had an error in our code that sets what render devices to use. It was a subtle error that only showed up if you disabled the GPUs in the user preferences in blender on the slave nodes. If you had them enabled, then it would seem that CR would work.

Anyway, we fixed the error and now you can select your GPUs to render no matter what the user preferences setting.

So, a huge THANK YOU! To both of you for reporting and sticking with us despite having a frustrating problem, thanks to your persistence (esp @Arnar Helgi Adalsteinsson for the many posts!) we've found a very subtle bug that we didn't pick up on our systems, cause we always have the GPUs enabled in user preferences.

It would be awesome if you guys wanted to try v017, so please reply to this thread if you do and I'll find a way to get the new build to you. It works for both 2.78, 2.79b, 2.79 daily and 2.80 beta (tested with 2.80.53) versions of blender.

All the best

:D

Hey guys, nice thread you got going here - props to Arnar for the testing :D

I guess I have run into the same problem. CUDA problems. I am using a desktop machine with 1070 and a laptop with a MX150 I guess. Did some testing and found out the desktop machine just rendering via CPU - GPU was sitting at 0% load all the time (CPU 100%). Combined render took some 2:10min while single render on the desktop took just under 1min.

Tweaked the tile sizes to match the "ideal" tile size I tested earlier on every individual system. (256*256 on the 1070 and I guess 128 on the MX150). Still no improvement.

After that I was a bit frustrated and took a break, restarded everything and so on ... Eventually I got back into the settings and just unchecked the CUDA device - suprisingly my GPU starts working. Overall I got around 47-49 seconds combined render, both machines set to CUDA and both machines unchecked CUDA checkboxes and both machines got their individual ideal tile size set.

By the way my goal was to set everything up to use my desktop as renderer, to accalerate rendertimes while working remotely.

Hope this helps, might be a bug with the "inverted checkboxes"?

Cheers -b

Moved all cards to one computer to simplify , still the render does not show up , stuck on 100% render. Any idea why the render does not show up ? Chasing this is exhausting, opened another project and now it gets the rendered tile. Open a new project , defautl cube ,resync and it does not get the tile. No rhyme or reason to any of this. Where Can I find logs to send you guys ?

After reopening. Blender 1 Looks like its connected but will not sync, the other ones are just greyed out. Connect or resync does nothing. So far, even though all of this appears random behaviour to me , this might be the most consistant fault. Not showing the renders , or something not working and any attempt to fix it by saving , resyncing , reopening will result in it losing connection and after that , nothing I can do to get it connected again. I did close Blender , found two Blender processes in the task manager , killed them , reopened, same thing as you see in the image. Is this only happening on my side ? Does this work for you guys when using multiple computers or do any of these problems look familiar to you ? It would be so awesome to have this work , I now have 5 computers, 4 of them just sitting there.

Render finished , node renders do not show up. Question, is it normal that it saves as exr ? And is that path shown possibly the problem ? Also , as you can see, it still says "rendering" and keeps showing that until I hit cancel render, even though all rendering is finished.

Rendering , seems fine.

Been trying to use this , still with all kinds of seemingly random sync problems , but now I seem to have a persistent problem . When I get it to sync and everything looks like its going to work , none of the nodes show their render when done , only the master shows its render. Anything particular that comes to mind ?

Still doing some tests, hoping to find something common to the faults I am having, will post.

ok, tried my main project , an animation, basically the reason I am doing all this , Opened project , all nodes show up, upload project , works. Rendering, works ..I am super happy ...until the image is finished...no image, lol. I think I am going to have to give up on this for now. I have spent a few days on this and its just not working. Shame really as I cannot find anything else that will do this , eagerly awaiting the update.

Hi Arnar,

Would it work well for you to just use CR with the nodes not connected, then when you want to render, connect. There are no sync issues when you are not connected to any nodes. You can leave the addon enabled and running, it will not cause issues that way. Then when you connect, you can tweak the things I mentioned and render.

I use it this way and it works particularly well. Connecting to nodes really doesn't make sense unless you are intending to render using them. If you are only going to be editing, then there's no point connecting to your nodes anyway.

Hope that helps

I dont know if this tells you anything, but I was testing with an plane and a box, add materials , resync , works. Then I added a sphere , resync, resync failed and now its dead , nodes dont respond to anything. Saving, reverting ,reopening does nothing . If I reopen the project its the same , dead. Must be something wrong with the project itself at the node side. And I am going to sleep.

In conclusion from testing pretty much all day , when it works , its awesome ! some of the time its flawless, change something ,hit resync , bam , all good !

Most of the time it does not work...Why ? I dont know , it seems quite random. Sometimes there is just no activity on the nodes , connecting , resyncing either does nothing or fails, saving , reverting does nothing. Sometimes it even says render and nothing happens and it goes back to saying sync. Basically I have had all possible versions of not working. Having said that , node 3 now works ...I fully expect it to stop working :) Weirdly , sometimes, reopening the project does not help , its pretty much dead in the water and i need to use another save and try again. I dont know if I can be of any assistance , send a log or something , I would love to have this work properly. Or do you have any ideas to narrow down the problems I am having ?

Suggestion , its strange to set the manual load when it autocalculates the other values , especially when you are working with multiple nodes , you have to lower one value to raise the other and then maybe it raises the third, bit of a mess. Maybe it should allow one to set the values and then auto calculate numbers if they are off when you hit ok.

I see, that worked. Only working with a simple cube , rendering on 1080ti is 57 sec. Rendering on the main and 2 nodes x*970 and a 980ti gives me 22 sec. Now to see if any regular projects work . ;)

Remove one node as its not working... save , reopen , now I am back to Main computer on CPu and a fourth of the image not rendering. Can it not know that one of the nodes has been removed ?

I got it to render ..on two nodes , 3rd node is stuck at rendering 0% and main computer insists on rendering on CPU . 3rd computer is probably also rendering on CPU , cant operate it at the moment. EDIT: yes , nr 3 is rendering on CPU. Stopped the render , now node 1 -2 are ready and 3 is synced , not sure about the difference? Is there a procedure I should follow to make this more likely to work , like always saving and resyncing before trying to render ? Most of my resyncs fail so I have hesitated to use that . Also, any thoughts on why some nodes seem to refuse Cuda render ? EDIT: I got the main computer to render on CUDA by going into preferences , disabling CUDA, reanabling and doing the same for CRowdrender main computer. 3rd computer does not want to play ball. Not sure if it means anything but nr 3 says synced, while the other ones say Ready. So , its Semi working at the moment, god knows what happens if I open another project , seems very hit and miss :) As it stands , two main problems . 1. 3rd computer will not start CUDA render. 2 . Main computer will not CUDA until I disable/enable. PS...the speed is insane!

So far it seems impossible to make this work for me . At some point one computer got sync but that is a far as I could take this , most of the time its either unsynced or cannot connect.

Just now I got it to sync , tried rendering , image was obviously not synced , tried resync... Sync failed.

Today I will be testing 4 computer. 1. Main computer 1080ti 2 Second computer 980ti 3. Third computer 2x 970 4. Fourth Computer 1060 My testing last night ended with me kind of giving up on the inconsistencies of when It works. Some projects just stop wanting to sync, others will open with it saying 0% rendering. Others will fail resync. I will be trying to take this as far as I am able , lets see.

Just saw the image you posted, nice! I like it, but the purple part is usually due to a missing texture, or in some cases HDRI if you are using the background to light the scene. Each node needs those textures and so far we do not have an automated mechanism for collecting all the external assets and distributing them. So this means you will need to pack them into the blend file.

If you could try packing all your assets into the blend file and let me know if that fixes the purple image, that would be great!

Btw, Is there any benefit in rendering eevee with this setup ?

No problems , I am all in at this point . :) I did try shutting it down ...only on the master , then only on the slave , then both ...but. I did not look for any stray processes , will check that.

@Arnar Helgi Adalsteinsson

"In terms of load balancing , when I have the 1080ti set to 55 it finishes when the 980ti is at 67% If I then set the 1080ti to 56 in an attempt to sync them the 980ti finishes when the 1080ti is at 67%. So, maybe the load balancing needs a finer setting or something ?"

The load balancer swaps the part of the image that each node does for each frame. Which means each node will render a different part of the screen for each render, this can cause the times to vary, what you should do to test properly is to let it run for maybe 20 frames or so. This should give you a better sample and more realistic idea of how well the two cards are performing.

"Still having problems with some projects with the second computer not syncing from my main computer, looking at the second computer it always says synced in Blender even though its not working. Nothing I do makes it come online. Opening up an older save of this same project and it says uploading 1% and just stays there. So in conclusion , works very well, when it works , seems unstable in terms of when it works ?"

I think I know that this .... but would have to see the issue for myself. In any case, 017 will be released soon and from what you have told me I think it will help with this particular issue. The fix for this, at the moment, seems to be to shut down blender on your master/client machine, then also look in your task/activity/process monitor tool for any blender processes that are still running, kill them, then restart blender.

Apologies for such a terrible work around, this is what we're fixing in 0.1.7

Tomorrow I will hook up more computers , lets see.

Now running one computer 1080ti , second computer 980ti , with tweaked load settings to even things out. I get 1:42 BMW benchmark with both computers rendering and 2:28 with just the one 1080ti. I would love to try this with 1080ti in both computers but I dont have one ;) In terms of load balancing , when I have the 1080ti set to 55 it finishes when the 980ti is at 67% If I then set the 1080ti to 56 in an attempt to sync them the 980ti finishes when the 1080ti is at 67%. So, maybe the load balancing needs a finer setting or something ?

Then in other Projects it seems like the second computer is faster , even with a 50/50 load? Been working on this all day , bit confused at this point. Still having problems with some projects with the second computer not syncing from my main computer, looking at the second computer it always says synced in Blender even though its not working. Nothing I do makes it come online. Opening up an older save of this same project and it says uploading 1% and just stays there. So in conclusion , works very well, when it works , seems unstable in terms of when it works ?

Load balancing seems to be a little of scale when going from 100 samples to a 1000. I think it would be cool to be able to set up all the nodes , do a "load balancing render" and its set from there?

I will make this my log of getting this to work :) So...creating a project, simple cube , rendering on 2 computers , setting the load manually seems to work. Now...I open an existing project and it will not sync , restarted both Blenders ,Conn, does nothing, Resync does nothing. Tried save , revert...nothing. Opened another existing project , and now it works ? No idea as to what the problem is there. PS: simple cube works with a load distribution of 84/16 but when I apply that same setting for another project its way off and I am waiting for the second computer. So it seems like the load distribution calc needs some work as I would prefer it calculating it for me. I can imagine that its a lot of tweaking to do it manually for multiple computers. In fact if its not set perfectly you can end up with a longer render , especially since my second computer has a crap GPU. As it stands I get a faster render on one computer , probably due to the load balancing. Tweaking that to perfection gets me a faster render. Also , seeing the progressive render would be very nice a lot of the time that is where you spot problems , cancel fix, restart. But I guess that is not possible . Also, I am getting this render problem. Second computer renders purple ? Probably does not transfer the environment hdri. Tried "automatically pack into blend" resaved, resynced. and now it works perfectly.

I am using a 1080ti one my main computer , gtx560 on the second one, load set to 84-16 and the speed now syncs. 1920*1080 - 500 samples and I am sitting at 47 sec with a simple cube. Disabling Crowdrender alltogether and only using the 1080 and I get 57 sec.

Further testing on my simple box, seems to give me 10 sec , so that is quite stable. Still having problems with opening for example the BMW test , will test further. Also getting more computers later today for testing.

One more question ,now the load balancing seems way off , I would rather count on the auto load balancing , especially since I will be adding more computers. Can I make it recalibrate the load balancing by doing a few renders? It does not seem to do that , it hands way too much to the second lesser computer. Maybe there should be a setting where you zero the load balancing and do a fixed number of renders and then it should be set ...in terms of user experience ? At this point I dont know if its monitoring the load and calibrating it if that makes sense? As for setting the load manually , that seems reasonable for two computers but I would be unsure of what I am doing with multiple computers.

Hi Arnar, have you tried using the manual load balancer? This might help with the slow combined render. You can choose how much of the image's screen area each machine will do. Try to balance this with your best guess and see if the render time improves.

V016 (which I assume you are using) has a quirk that if you first render using just one machine, then add a second, the second machine will not render very much. It was supposed to be a fail safe for animation renders where if the machine that gets added second is slow, this technique would avoid giving that slow machine too much too render. But it seems this has just caused too many problems with users seeing slow renders anyway since the expectation is the rendering will be faster using our addon. We're going to change this.

As for the CUDA issue, that is a bit strange, our addon should ignore the user preferences settings and use the settings from the node menu. If you set the local/master machine to use CUDA in the little node menu (which has a gear/cog icon to the right of the node's name), then it should obey that setting. However, if there is an issue with finding the device, then it falls back to CPU.

It would be great to know a bit more about your system to see what's going wrong here. If you can, please e0mail us and we can get in touch?

info at crowdrender dot com dot au