Amazon EC2 Renderfarm

The Elastic Renderfarm - an inexpensive rendering solution

Amazon offers a great service through Amazon Web Services called “Elastic Computing.” Through EC2, you can build your own renderfarm with render core costs as low as $0.06 per core hour.

Through these tutorials, I demonstrate how to set up your own Backburner renderfarm for Autodesk’s 3DS Max and Maya software.

*note* the section on how to kick off a 3DS Max Backburner render starts at about 8 minutes in to section 4.2.  I will split that off into it’s own separate section soon.

 

49 thoughts on “Amazon EC2 Renderfarm”

    • Unfortunately I only have the resources to check the vanilla Backburner set up at the moment… no plugins or anything like that.

      If you have a license server for your plugins, you should be able to set up a VPN connection so your plug-in licenses can be checked. It’s also possible that the plug-ins don’t need to do the license check if they’re on a render-only node… depends on the license particulars.

      Reply ↓

  2. Hi
    thank you for your great tutorial.

    there is something i really dont understand with the amazon service.
    did it cost something to keep the machine where you installed 3dsmax ?

    as you can see, my english is very limited.
    what i understand is :
    i install 3dsmax on a server with 60gigas of hard drive.
    i install the manager on another server
    i choose x node with render slave
    i render my images from the manager.
    i send my image to my ftp/dropbox etc…

    but, what did it cost to keep my configuration with 3dsmax on the amazon server ?

    i hope you will understand me :)

    again, thank you !

    M.°

    Reply ↓

    • The AMI itself doesn’t appear to cost anything to keep. It’s the active instances and volumes that you’ll have to pay for your time on.

      The volumes themselves are very cheap to run. The instances are what will start to add up.

      When you have completed working with your instances, be sure to Terminate them in the EC2 console by right clicking on them and selecting ‘terminate.’ If you have a bunch, you can shift-click, select them all, and then terminate them all at once.

      After the termination is complete, you’ll want to go to the volumes tab and delete any remaining volumes.

      The saved 60 gig AMI doesn’t seem to cost anything to store when you’re not running any instances.

      Just for reference, here’s a sample of my bill for so far in September. All prices are in US dollars. This was for 20 instances rendering a 3DS Max project. Each instance had the 60 gig volume attached that’s associated with the AMI and there was also one single 40 gig volume attached to the render manager to be used as a network drive.

      The total render time was 45 hours of machine time spread over 20 machines which basically means I was able to do 45 hours of rendering in a little over 2 hours of actual time.

      Link to the September bill thus far: http://www.judpratt.com/wp-content/uploads/2011/09/SeptAWSBill.jpg

      Hope this helps!
      –Jud

      Reply ↓

    • Yep… should work on Cinema 4D as long as they have a way of setting up networked render nodes. (It’s been a long time since I played with C4D)

      Reply ↓

  5. Hi, You make a great tutorial. This tutorial make everything easier. Thanks you!

    I got a problem when I follow the video part 4.2.

    I spot instance 10 machine and try to use network drive to connect with my render manager machine. But windows said “network path was not found” . I checked my security groups setting again, make sure all setting are same as your video (part1) and tried turn off all firewall in instance machine already.

    Please help.

    Thanks a lot,

    Riuku

    Reply ↓

    • Off the top of my head… it could be the machines are not all in the same region.

      For example, in my case I have the options of us-east-1a, us-east-1b, and us-east-1c. You need to make sure that all of the machines are in the same region. So… I would assign all my instances to be, say, in us-east-1b. It could be any one of those, just make sure all of them are assigned to the same one.

      Hope this helps!

      Reply ↓

      • I make sure all of my machines are in same region already. Still got same problem. Maybe I retry from the first video one more time. :D

        Thanks!

        Reply ↓

  6. Very informative, thanks! Is there a way to store your runtime configuration so you don’t have to manually spawn all your networked instances each time you have a render job?

    Reply ↓

  7. Great Tutorial thanks a ton!

    It looks like you manually mapped the Z:\ drive on the Render Servers after you spawned the 10 instances which would seem to mean that you would have to go through the whole process on each server each time you launched the instances for each new job.

    Is there a way to map the Z drive and change the Max settings once and then save it into the Render Server AMI so you would be able to skip those time very consuming steps? This way it’s a fast process to launch as many Render Servers as needed, transfer the work files to the 60GB volume and you’re going.

    Reply ↓

    • That’s something I’m wondering too. It seems to me that if we setup a render machine with the mapped network drive and the server in autostartup mode and then create an ami of it, you should then be able to start 10 instance of it without setting up everything manually?

      Reply ↓

  8. Hey,

    You have blown my mind. I do have one question, though: Cinema 4D uses a browser-based interface for its net render that you can access from any computer on the network. Do you know how to make the render manager machine’s ip address publicly available, so that animators can access from any computer just as though it were a webpage? Or is that complex enough that I’d need to hire an IT guy?

    Thanks again for the awesome tutorial.
    Cameron

    Reply ↓

  9. Hi Jud

    Amazing post, great idea.
    I’ll going to do some tests next week, but I’m curious if you made some benchmark to evaluate the performance of your instances.

    Also in your bill you show that you used c1.xlarge instances.
    Amazon says the price is 1.16 per hour, and probably my math is wrong here, but I did the following math:
    20 instances X 45 hours =
    1.16 x 20 x 45 = 1044$

    What am I doing wrong?
    Thanks for your time!

    Reply ↓

    • Well, I would say, that the math is right, but the numbers are wrong ;)

      using spot price from the videos:

      0.5$ * 20 instances * 2 hours real time = 20$ which is about the bill :)

      Reply ↓

  11. Hi! thanks for sharing! Cloud are starting to look very interesting as a render farm. I would like to know how to install vray and get access to my USB dongle on my workstation to be able to render with vray on the cloud.

    Reply ↓

  12. Thanks for this great tutorial…. I couldn’t get the other node to see the shared folder, or to have the client (C4D) connect to the server :(

    [ And yeah, they are both in the same region ]

    Reply ↓

  13. Pingback: Очень подробная видеоинструкция как запускать рендер на Амазоне (на английском) « Рендер ферма 24 часа

  15. Hi,
    I can’t reproduce one of your steps.
    When I try to share a network drive across many instances. When I open 2 instances, they don’t see each other even if they are in the same zone… How to we allow them to see each other on the network?

    Reply ↓

  16. Hi,

    Nice tutorial!

    I have a question?
    You terminate the instance in you’re tut.
    If you stop you’re instances do you have to pay for them?

    if you terimate them you have to set them the next time again from zero, not?

    Regards,
    Gio

    Reply ↓

  17. Yes, you have to pay for the instances that is not terminated.

    Also you can create the image (AMI) . And on the next time create the instances from this one.

    Reply ↓

      • Difficult to answer.

        Maybe Firewall blocks the connection.
        - Allow all TCP, UDP, ICMP

        Maybe IP address of the server or manager is wrong.
        - Try to use this command to show the IP address and name of the computer: ipconfig /all).

        Maybe Backburner Manager/Server has wrong configuration.
        - Try to remove XML files from the directory “C:\Program Files (x86)\Autodesk\Backburner\Network”.
        Also remove all files from sub-directories “Jobs” “ServerJob” “Servers”
        Then start Manager/Server and configure them again.

        Reply ↓

  19. Hey Jud. Great tutorials. However, I have a question in regards to licensing. Do you need to have an extra license or licenses of Autodesk Maya or 3ds Max for if not all the instances, the manager instance?

    Reply ↓

    • Yep… Dave is right. I posted the following on another forum about this:

      On the machines you are going to use as render servers, you need to install 3DS Max or Maya but you do not need to activate those installs with a license number. Render-only nodes do not need a license (or should I say, their license is covered by the license you already own for your primary machine.)

      Backburner will fire up the render engines of those respective applications to complete the render, but they are not considered “full installs.”

      Here’s a bit from the 3DS Max (same is true for Maya) documentation where it specifically says:
      This topic contains basic procedures to follow when network rendering with 3ds Max.

      The following conditions are assumed:

      3ds Max has been installed on all machines to use as rendering servers. Note: You needn’t license 3ds Max on machines to be used only as rendering servers.
      …and here’s the link to the page in the 3DS Max 2012 documentation where it says this:

      http://download.auto…7FD2F-2315.htm

      Reply ↓

  20. Would this solution to redeploy 10 instances work ?
    Now amazon gives 750h free micro instances.

    So what I do I set up micro isntance to:
    35gb
    upgrade to 60 gb – do I pay for upgrade?
    Add extra 40 gb for share drive (I gotta rewatch the movies I got confused wrrr)

    Set up a master instance for manager – save image.
    Set up a slave image1 – save image.
    Set up a slave image2 – save image.
    Set up a slave image3 – save image.
    Set up a slave image4 – save image.
    Set up a slave image5 – save image.
    Set up a slave image6 – save image.
    Set up a slave image7 – save image.
    Images are free right?

    Then when I terminate- and re create Images I can just load image for each slave with all settings on n running?

    Will IP/machine names change etc etc? – what do u guys think?

    Reply ↓

  21. > upgrade to 60 gb – do I pay for upgrade?
    Yes.
    > Images are free right?
    No. You have to pay for the storage of each AMI.

    (Free for 30 GB/month of Amazon Elastic Block Storage (EBS))

    > Will IP/machine names change etc etc?
    That’s right. Also will be changed after stop/start instance.

    Reply ↓

  22. Thanks for info!

    I’ve been rendering on amazons for few days now. Quite wicked stuff and cheap to curtain point.

    In any case I tweaked slightly ur workflow. Instead of starting max, creating texture directories etc etc. All I do is to copy my ZIP BB job from my local machine to amazon machine and restart manager – job gets automatically updated and all start to render on its own – since I include textures with my BB file all goes sweet !

    Also I use distributed tile rendering. So I suspend second job and do it locally to save time and money ! :)

    All I have to find out now is how to update BB manager without the need to restart it arr !

    Reply ↓

  24. Eeee no I dont.

    When I save JOB to Backburner (press render other words) I click to Save maps with file. Then Max save ZIP with textures and MAX to JOBS inside BB>Network folder (I’m using 2008 newer BB sux…)

    Then all I do is to copy 2 folders (1 stripes + 1 stitching job) to render manager and restart manager. Then it pick up 2x jobs from which I suspend stitching since I can do it locally to save ££//$$

    All I’m trying to find out now is how to update Manager without the need of restarting it as well as how to set up my vray license server from home =D

    I wish there was a way to save Image + AMI to my own HDD so I dont have to pay for it while I dont use farm :(
    Thanks, bye.

    Reply ↓

  27. Dariusz, I understand what you are doing. Very foxy.

    >All I’m trying to find out now is how to update Manager without the need of restarting it…
    Will have to restart the manager that it saw a new jobs. I see no possibility to update list of jobs.

    >how to set up my vray license server from home =D
    Share your home computer for incoming connections.

    >I wish there was a way to save Image + AMI to my own HDD so I dont have to pay for it while I dont use farm
    Why do you need this? Amazon give you 30G per month for free. So you can create one image and pay nothing.

    Reply ↓

  28. I’m having a problem with the Servers trying to find the mapped D drive in the RenderManger instance. I did exactly what you did in the tutorial word by word but it still wont find the folder. It detects the network instance but not the volume.

    This is what I did.

    1) Created RenderManager Instance based on Previous AMI “RenderFarm”
    - attached Volume 30GiB to Instance RenderManager
    - Enabled Sharing on D Drive and Mapped to Network Z:

    2) Created 5 Instances based on Previous AMI “RenderFarm”
    - Attempting to access networked Z: drive : Resulting in failure.
    The New instances are not able to see the Shared D Drive. It keeps giving me an error. Please help!

    Reply ↓

  29. >. It detects the network instance but not the volume.
    On one of the server instance:
    If you are open this path “\\you-manager-instance-name-or-IP-address” what you are seeing?

    On the manager instance:
    Do the same. What you are seeing?

    Maybe you can send the screenshot.

    Reply ↓

  32. Hi Jud !!
    followed your videos to setup an instance ..got it up and running the issue i’m having is ,its taking way long time to transfer my setup ISO file from my pc to the instance i had just created .. aprrox 158hrs using teamviewer as iam unsuccessful in liking my pc to the amazon instance using Filezilla…some ip / port problem. I need to ask “how much time did it take you to copy your stuff to instance.” ? & is there a way i can do it better !!

    Reply ↓

  34. Thanks for your tutorial, nice work! But I have a question.

    I have to use \\\ instead of \\\ to get the network drive worked. The problem is, every time I restart my EC2 instance, the IP address changes, so I have to re-map the network drive for all of the render servers. Is there any way to work around this?

    Reply ↓

    • Sorry, angle brackets are filtered by the comment system. The third sentence should be corrected as:

      * I have to use \\(ip-address)\(drive-name) instead of \\(computer-name)\(drive-name) to get the network drive worked.

      Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>