Join our discord community

Please or Register to create posts and topics.

Submit renders from multiple shots (Houdini)

This is an issue I need to automate as it's taking way too long to go into each scene and submit renders with Prism. Before switching to Prism I used open pipeline, and since it versioned files by always writing out the latest version to the same file (and making backups for each save), I could easily make a .bat with a list of maya scenes to render from - as the generic file would always be the latest version. As prism goes around versioning and publishing with always iterating the scene names, I need to find another solution.

What would be the best way of going about this? Is there a way to query the path of the latest version and insert that into a .cmd file with a command to execute all active prism renderstates?

At the moment there is no function implemented, which would give you all the latest scenefile paths. You would have to write a python script, which searches all the shotfolders and queries all the files, which you need. Then the script would need to open each scene and execute a publish. I would execute that script directly in Houdini so there would be no advantage of having a .bat file.

I'll probably add some similar functionality in the future, when I'll have a look at Prism-TOPs integration.

Hi Richard!

I'm here with Magnus, I'm the local python wizard... or apprentice after looking at your code.

Been digging into Prism and how it is all setup and a script you describe sounds very doable.
But we're having a hard time finding the command that's executed when you press publish.

We can see a lot of 'Prism_*' modules are loaded and have found the publish() function in StateManager.py
Which Module can we reach that function from?

And then, is there an easy way to ignore the publish warnings about dependencies outside the project folder?
Is that the 'continuePublish' flag?

Prism is incredible from what I've seen so far!
We have started recommending it to all the students here at The Animation Workshop that's right next to us here.

 

-Bastian

Hey Bastian,

thanks for the recommendations to the Animation Workshop. Hopefully Prism helps them to create even better short films. I know the Pipeline prof from there and I showed him Prism a long time ago, before I released it officially. He didn't seem very impressed back then 😀

First of all, during the publish Prism queries some data from the StateManager user interface. That means it's not possible to use hython to execute a publish without an Houdini UI. I'll probably change that in the future, but it will be a lot of work.

Almost every python script in Prism relies on the PrismCore.py and you can access the StateManager, ProjectBrowser and other tools from the PrismCore module. You can get a PrismCore instance from the PrismInit module in Houdini, which is used to load Prism during the Houdini startup.

To execute a StateManager publish from python you can execute this:

import PrismInit

if not hasattr(PrismInit.pcore, "sm"):
PrismInit.pcore.stateManager()

PrismInit.pcore.sm.publish()

You cannot use the Houdini python shell to execute your code, because that executes python commands in a separate thread and all QT interfaces need to be in the Houdini main thread (you will crash Houdini if you try that). You can use the Python Source Editor in Houdini or a custom shelf tool to execute the code.

The "continuePublish" flag is only relevant in Blender and doesn't help you here.

Currently there is no flag to skip any warnings. But all the warnings are handled in the "publish" function and you can modify it easily. To skip all the warnings you can comment out this part:

action = warnDlg.exec_()

if action == 0:
return

I hope that helps. Let me know if you have any other questions. I completely understand that it's not easy to find things in the Prism code if you don't know where to search for it.

Hi Richard, thanks for the thourough answer. Bastian - feel free to fill in here 🙂

  1. I know the Pipeline prof from there and I showed him Prism a long time ago, before I released it officially. He didn't seem very impressed back then ?

Not sure who that might have been, but I can tell you I pitched prism to the head of the school and the coordinators for the CG Artist line, and they were super impressed. They will most likely switch from Open Pipeline to Prism very soon, and also include more Houdini in the curriculum. (after seeing how easy it is to transfer assets and animation with Prism)

2. First of all, during the publish Prism queries some data from the StateManager user interface. That means it's not possible to use hython to execute a publish without an Houdini UI.

Would it be possible to set the state settings in the the hrender cmd file? Just thinking if there would be any kind of work around... Alternively not using the publish, but maybe instead we can make a little script that mimics prisms naming and folder structure for rendering so that is still shows up in the project browser. Would that be possible?

Not sure who that might have been, but I can tell you I pitched prism to the head of the school and the coordinators for the CG Artist line, and they were super impressed. They will most likely switch from Open Pipeline to Prism very soon, and also include more Houdini in the curriculum. (after seeing how easy it is to transfer assets and animation with Prism)

I don't want to write names here, but that sounds really great, thanks Magnus.

Would it be possible to set the state settings in the the hrender cmd file? Just thinking if there would be any kind of work around... Alternively not using the publish, but maybe instead we can make a little script that mimics prisms naming and folder structure for rendering so that is still shows up in the project browser. Would that be possible?

Not all the state settings are supported in the hrender script as far as I can tell (for example the outputpath of the AOVs). You could modify the executeState function in the hou_ImageRender.py so that it not renders directly, but instead it sets all the settings on the ROP node and adds a line to the hrender file then. But this way still requires an Houdini UI to query all the settings from the State Manager.

If you want Prism to generate the outputpaths for you, you need the Houdini UI open. It would be possible write a script to mimic the Prism folderstructure and to set all the settings on the ROP without an UI, but then you would need to rewrite the getOutputName and executeState (and probably some more) functions of every statetype that you want to execute, which could be a lot of work. Without a very good reason I would try to avoid that.

I see, we probably won't go down that road, seems like a lot of work. I have another idea for a workaround that might work - even though it might be a bit too much for you too look at right away, but it is a feature that could have multiple benefits. If there was an option to publish assets by overwriting a genericly named publish file, like open pipeline does, it could actually solve a lot of things since the latest version would always have the same file. That way I could publish to a renderfarm once, and then just resubmit all the rendertasks from pandora or deadline and get all updates from the scenes. Only problem is that it would overwrite any existing renders, but either I could move the ones I want to you use for comp manually, or I could get the rendermanager to iterate the folder naming. I would also find this useful for models and rigs, as I prefer everything to be updated automatically. I would say that having the animators to manually update rigs have caused me a lot more headache than once in a while having to revert the asset version - so at least having to option to publish in this way would be awesome.

Is there a reason why you are looking for a workaround to not let the script loop through all the scenes with the Houdini UI? As long as there are no warning dialogs, which interrupt the automatic "open scene-publish" loop that should work fine.

As you probably know there is an option to export an asset as the previous version and overwrite the old version. But I agree that it would be more comfortable to have an option in the project settings for that if you prefer to work with the open pipeline/publish file workflow.

Does open pipeline make an automatic backup of the published file before it gets overwritten or is it the responsibility of the artist to add comments to the working scene files, which contain information about the changes of the asset?

It's just because when you have hundreds of shots I don't want to look over the computer opening houdini for each one. I like 'command line'-like solutions for these kind of tasks a lot better. But I might be misunderstanding you, so just to be clear, the render script would have to open the actual scene in houdini - wait for all nodes to cook etc etc and then submit a publish, right? Then close houdini, then open the next hip file etc.

I didn't know about that feature, but not sure how 'clean' it would be...

Yes, it makes a backup in a seperate folder each time you publish - so in the publish folder of any given asset there is only one file - for example publish/MyCharacterRig.ma - then there is a folder for the backups which are stored incrementally - for example publish/backups/MyCharacterRig_v025.ma. It has a ui where you can revert your publish to one of the backups - and all it does is basically to copy the selected backup and overwrite the publish file. (by the way when you publish it actually first add a new iteration to the backup folder, then overwrite the publish file with the same file - so it basically saves it twice - you need this in case you revert back, and then want to revert back to the newest again...)

Depending on how your Houdini scenes are set up not all nodes need to be cooked when opening a scenefile. You wouldn't need to close Houdini and open a new Houdini instance. The script could open all scenes after each other in the same Houdini session. But I completely agree, doing that over the commandline would be the better way to do this. However, this way would require a good amount of coding work.

If you rightclick on an export state and select "Execute as previous version", Prism will overwrite the previous version. The rigger would be responsible to always export as v0001. But that doesn't create any backup files, so it's definitely not an equal workflow to open pipeline.

I added it as [0096]. That will take some time to implement.

I also added support for commandline operations [0097] and support for TOPs [0098]