Hi all,
I hope this is appropriate to post here - if not, mods feel free to delete.
In short though, I recently set up a CR node using Docker - and found the process a little bit challenging. So I wrote up a short guide/some things I learned on my website; maybe they will be useful for anyone else trying the same thing:
Part one is getting the Docker host ready with CUDA support:
Part two is running CrowdRender:
I hope it helps someone!
Cheers,
Tim
Hi Tim, if this helps others, its totally appropriate and jolly good stuff!
I am not an expert in docker though, I am going to refer to our resident docker expert @zocker1600 for some comments and perhaps we can work on a resource as a collaboration. If we can make things easier for people we should and I am completely behind anyone helping out with that.
Oh, and thanks so much for posting in a positive way mate, really glad you're putting something back into the community, its really cool.
Cheers and have a great day :D
Hi, thanks for the positive feedback; and many many thanks @zocker1600 for the Docker image :).
And sure, happy to offer anything I can do to help. Cheers!
No problem Tim :) Its pretty cool to have community input like this!
this is a great read @tim.walls , here are a few of my thoughts:
1. first I apologize for the very bad timing, but I just updated all my images to Blender 2.83 LTS after I finally could fix the stupid "GPU does not get used for some unknown reason" bug
2. the switch --runtime nvidia is deprecated and - while it still works - should not be used anymore. The replacement is --gpus all
3. if you want to see the logs of the start script, you can use
docker logs -f <name of container>
the name of container is what got specified with --name
4. the missing audio device bug is very weird. To me this happened only on Ubuntu based systems, on my Debian 10 render server that doesn't happen. It is weird, because when using -noaudio it should get ignored....
5. Eeevee does produce black images, because Blender does not support Eeevee on headless systems. They need an Xserver and an attached display for some OpenGL stuff, all workarounds I have tried so far produced completely broken images (not black ones though...)
Blender devs said, that they are working on a solution, albeit with a low priority.
6. why do you pass --rm to the run command? Normally there shouldn't be a reason to create a new container on each start
7. the start script does download the addon on the first start, but once downloaded it does reuse it. So you only connect to the CR server once.
Sadly as long as the CR addon is not openSource, there is no other way for me to do that.... :(
Cool, thanks for the feedback! I will make edits in the morning :-). A few responses:
Argh, but good of course... Will try and upgrade in due course :). In the meantime, I'm glad I wrote to use the :bl.2.82_cu_10 tag and not :nvidia ;-).
Noted. Actually, if anyone followed my instructions from part one they don't need either, so it's a bit belt-and-braces anyway. I'll amend when I update it.
Unfortunately, while this is true, the errors that had me tearing my hair out weren't in the container logs (i.e. weren't coming from the startup script.) After stratup, it's pretty much silent, hence having to dig into the container image to find the actual logs in /root/cr/logging. It would be nice if the runtime logs were also output to stdout for container logging.
To be honest, I'm just glad someone else has experienced this bug - it was driving me to question my sanity.
Uhuh, noted - I'll update the post to explain this as well.
Personal habit/policy - anything long-lived on this machine should be managed by Kubernetes (indeed, if I ever get round to a Part 3 it'll be the helm chart I write to do this). Running directly with "docker run" comes under the heading of quick hack (in the nicest sense of the word - I mean that includes most of my CI build jobs), so I like to clean up after as much as possible :-).
Good to know - although is there some CR server component in the matchmaking (as it were?) (And if not, then any ideas why I have two zombie instances for the same server in my connection list..?) I should say when I was writing the post, the CR site was down, meaning I was having to go entirely on my notes and couldn't actually double-check anything - so that comment was born out of tangible frustration :-D. I'll update to include that (and remove the -rm accordingly...)
Thanks very much - will post an updated version tomorrow, much appreciated.
@tim.walls
true this is actually good advice to use a stable tag, hence I offer them
--
I fully agree, I actually requested this feature multiple times to @James Crowther , do you have any updates on that? ;) It would be awesome to have errors in STDOUT when passing a certain CLI argument
--
You might link to the documentation: https://docs.blender.org/manual/en/dev/render/eevee/limitations.html#headless-rendering under "Headless Rendering" section at the bottom I am still trying to find a workable solution, this is a huge problem for my own renderfarm as well
fair enough, was just curious
AFAIK there were additional features planned, when you connect the client with your account, but as of now I am not aware of any, but I could be wrong, maybe @James Crowther can answer that question better.
Also I personally have to add, that I would like to be able to render completely offline too, so hopefully we can do that once the client gets open sourced.
I thank you for the good writeup, since now I can point ppl to it, when they are searching for a complete setup guide. 👌
@zocker1600
Ok, regarding 3.
Coming in the new CRESS based system which will have full logging output optionally sent to the console/terminal where you start CRESS. More on that as we build it.
Regarding 7. @tim.walls and @zocker1600 assuming you are referring to nodes appearing in the list inside the crowd render panel in blender? If not please can you screenshot me what you're seeing so I can educate myself :P
Thanks again for the update, I made most of the edits this morning - I'll update to 2.83 once I've done the upgrade myself this weekend (alas, the day job gets in the way) but in the meantime added a note that the newer version is available.
I should say thanks again for the image, and indeed to all the CR folks. Without this I pretty much wouldn't be able to use Blender for anything useful at all (my desktop machine is a lowly MacMini with integrated graphics) - so I really do appreciate.