UK artist Pete Addington shares the MaxScript lesson he gave at the End User Event 2011, in which he explains how to create a LayerControl script using MaxScript and DotNet. More at LoneRobot.net.

آموزش مکس اسکریپت چگونگی کنترل لایه ها

MaxScript Lesson

One of the tools that I have tended to re-use over many productions is the LayerControl script. It allows animators to bypass the layer manager and hide layers according to object type. While there are a few ways of achieving this on a node level using AppData or UserProps, you can quickly setup a system like this with a methodical and consistent naming convention. When working, hiding and showing rig controls and meshes on masse become a simpler and quicker affair. This might not sound much, but over a long animation process tools like this save time.

Lets look at how to set this system up.

Firstly, at the rigging stage, you will want to make sure that all of your characters have consistent layer names.

It is the suffixes of these layers that the script will be using to control whether a layer is hidden or not.

Script Stage 1

-- Basic Layer Visibility Control
(
local str = "MESH"
local ishidden = true

for i= 1 to (LayerManager.count-1) where
(
(dotnetobject "system.string" (LayerManager.getLayer i).name).endswith str
)
do (LayerManager.getLayer i).ishidden = ishidden
)

This is a basic piece of code that will hide any layer with the suffix “MESH”. As it stands, its not really useful for anything except to illustrate the code of the script. Note that this has been formatted this way in order to make it clearer, you could add it all into one line. Using where in a loop is a useful trick. In this case, it allows us to perform the hiding of the layer without having to collect the Mesh layers into an array and iterating that. You’ll notice I use a dotnet string method in this. You could just as easily use :

matchpattern (LayerManager.getLayer i).name pattern:(“*”+str)

The dotnet method might be fractionally slower to execute, but since we’re talking a few milliseconds, it’s not going to make a whole lot of difference. I included it to illustrate the dotnet string obejct. While maxscript string methods are great, there are even more methods available to you via the dotnet methods should you need it. Use whichever one you like, it’s not going to make the final script any better or worse. If you’re just getting into scripting and programming, you’ll find a lot of the time you’ll want to make sure the core of the script is working before fiddling around with the UI.

Script Stage 2

The first script was just to establish how the method for hiding and unhiding will work.  The next version builds a basic UI and starts to add the functionality we want. It now works by calling a function that takes two arguments. For novice scripters, a function is something used widely in programming to represent and operation that you will want to re-use multiple times. As a form of shorthand, you just call the function name and pass the values it requires rather than typing the same code out repeatedly. In function calls the extra values are known as arguments. In this case, the suffix string that we want to hide/unhide is the first argument. The second argument is to decide whether the function will hide or unhide the layer. Since this is an either/or type, we use a boolean argument of true or false. So in this code we use a button click to hide the layer (passing the string and the value true) and the button’s right click handler to pass the same string and false to unhide it.

try(destroydialog HideRigs)catch()

rollout HideRigs “” width:73 height:84
(
fn LayerVisibiltyBySuffix str ishidden =
(
for i= 1 to (LayerManager.count-1) where ((dotnetobject “system.string” (LayerManager.getLayer i).name).endswith str) do ((LayerManager.getLayer i).ishidden = ishidden)
)

button btnRIG “Rig” pos:[2,56] width:67 height:23 border:false
button btnMESH “Mesh” pos:[3,30] width:67 height:23 border:false
button btnCTRLS “Controls” pos:[3,3] width:67 height:23 border:false

on btnRIG pressed do LayerVisibiltyBySuffix “RIG” true
on btnCTRLS pressed do LayerVisibiltyBySuffix “CTRLS” true
on btnMESH pressed do LayerVisibiltyBySuffix “MESH” true

on btnRIG rightclick do LayerVisibiltyBySuffix “RIG” false
on btnCTRLS rightclick do LayerVisibiltyBySuffix “CTRLS” false
on btnMESH rightclick do LayerVisibiltyBySuffix “MESH” false

)
createdialog HideRigs

Script Stage 3

Stage 3 has some improvements in the form of replacing the max controls with some dotnet controls. Have read of the code and I’ll discuss what s going on afterwards.
macroScript ShowHideLayers
category:”LoneRobot”
toolTip:”Show Hide Layers”
buttontext:”Layers”
(
try(destroydialog HideRigs)catch()

rollout HideRigs “” width:84 height:324
(
local DotNetColorMan = (dotnetclass “managedservices.cuiupdater”).getinstance()
local MlbSelection = #(“Angus”, “Big_Pig”, “Bo”, “Cow”, “Cowhand_One”, “Cowhand_Two”, “Crow”, “Farmer”, “FarmGirl”, “Hebaa”, “Leonard”, “Mini”, “Piggy”, “Shebaa”, “Trinny”, “Trotski”, “Unicorn”, “Winnie”)

fn LayerVisibiltyBySuffix str lbx ishidden =
(
if lbx.selection.isEmpty then
(
for i= 1 to (LayerManager.count-1) where ((dotnetobject “system.string” (LayerManager.getLayer i).name).endswith str) do ((LayerManager.getLayer i).ishidden = ishidden)
enableaccelerators = true
)
else
(
for each in lbx.selection do
(
local lay = LayerManager.getLayerfromname (lbx.items[each] +”_”+ str)
if lay !=undefined then lay.ishidden = ishidden
enableaccelerators = true
)
)
)

fn MouseButton args = dotNet.compareEnums args.button (dotnetclass “System.Windows.Forms.MouseButtons”).left

dotNetControl btnRIG “button” pos:[1,56] width:82 height:23
dotNetControl btnMESH “button” pos:[1,30] width:82 height:23
dotNetControl btnCTRLS “button” pos:[1,3] width:82 height:23
Multilistbox lbxCh “” pos:[1,82] width:82 height:18 items:MlbSelection

on HideRigs open do
(
btnRIG.flatstyle = btnMESH.flatstyle= btnCTRLS.flatstyle =(dotNetclass “System.Windows.Forms.FlatStyle”).Flat
btnRIG.backcolor = btnMESH.backcolor= btnCTRLS.backcolor = DotNetColorMan.GetControlColor()
btnRIG.forecolor = btnMESH.forecolor= btnCTRLS.forecolor = DotNetColorMan.GetTextColor()
btnRIG.text = “Rig”
btnMESH.text = “Mesh”
btnCTRLS.text = “Controls”
)

on btnRIG mouseDown sender args do LayerVisibiltyBySuffix “RIG” lbxCh (MouseButton args)
on btnMESH mouseDown sender args do LayerVisibiltyBySuffix “MESH” lbxCh (MouseButton args)
on btnCTRLS mouseDown sender args do LayerVisibiltyBySuffix “CTRLS” lbxCh (MouseButton args)

on lbxCh rightclick do lbxch.selection = #{}

)
createdialog HideRigs
)

You’ll see that there is only one handler for each button. So you may be wondering how it passes the required true/false argument with just one call. This is one reason why we use  dotentcontrols instead of max UI controls. When you click a dotnetcontrol you can get some additional provided about which mouse button you have used. This is contained within the args property. The mousedownevent calls a function before passing it’s final argument. This function returns the boolean argument we need. So before handling the mousedown event, it asks this function a question:

this translates in plain speech to “is the mouse button clicked the left button?”. We use dotnet.compareenums to return either true or false in answering a comparision of the mouse button used to click each button and the enumeration of the left button. Dotnet uses enumerations to simplify data types rather than expecting the user to identify an arbitrary integer code.

The other addition to this script is to add a listbox of character names. These names are hardcoded into an array at the start of the script. You could easily scan a set of character folders and retrieve these names dynamically. You just need to make sure you are consistent with naming or the system will break down.

Script Stage 4

From now on, I wont be posting the entire code for each example, but highlighting the important parts of the new code. Don’t worry though, full code samples will be provided at the end of the post so you’ll be able to see what I mean.  Stage 4 brings in base64 encoded strings to store button bitmaps. There’s no point me going into this as i’ve posted about this before here. In normal circumstances, the base64 struct would be added to the startupscripts as a separate entity so that it executes only once on maxstart.

We have also ramped up the modifier key functionality. I’m personally a fan of having multiple functions on a single button – purely for the fact that you keep the UI as small as possible. There’s nothing worse than cramming a UI with extra buttons that could be easily passed to a shift click variation of the same button. In the case of the layer control, its critical to keep the footprint as small as possible and just add a button with If somebody doesn’t want to use a shft click or ctrl click then they won’t. If they do, you’ve already got the functionality in there. You can’t lose really. I’ve always considered that you’ve memorised a whole load of keyboard shortcuts up to this point, it wont hurt to memorise a couple more. The key is to make them match design patterns that already exist in the software. So if you have shift click to select all the objects in a particular layer, then shift + ctrl click should add these nodes to the current selection, exactly how max appends a ctrl select. It’s just a way of keeping things consistent.

So this list of modifer key functions are as follows -

Left Click - Hide Layer

Right Click – Unhide Layer

Shift Click – Select Nodes on layer

Shift Ctrl Click – Add nodes to selection

Shift Double Click – Isolate layer node selection

Alt-Left Click – Freeze layer

Alt-Right Click – Unfreeze layer

Ctrl – Alt Click – Perform an inverse action – i.e. If one character name is selected, do the hide/unhide action on all other layers EXCEPT the selected one.

Too many? my logic is if they are overkill, people won’t use them. Bear in mind all of the permutations above were as a result of animators asking for them over the course of many productions!

Script Stage 5

Okay, final polish time here. The dotnet multilistbox has been changed to ownerdraw mode. This is more advanced dotnet stuff but it means that you can take control of the appearance of a dotnetcontrol in ways where the original appearance properties do not do what you require.

on dnlbx DrawItem sender args do
(
if (dotNet.compareEnums args.state (dotnetclass "DrawItemState").Selected) then
(
-- draw the selected state of the listbox
args.Graphics.FillRectangle selBrush args.bounds -- background colour
args.Graphics.DrawString dnlbx.Items.Item[args.index] args.font brushes.purple args.bounds.location.x args.bounds.location.y -- draw the text
)
else
(
args.DrawBackground() -- this is an inbuild call to draw the default background
args.Graphics.DrawString dnlbx.Items.Item[args.index] args.font uTextBrush args.bounds.location.x args.bounds.location.y -- draw the text string
)
-- args.DrawFocusRectangle() -- draw the focus rectangle last
)

The comments should explain what is going on.

Conclusion

So that’s it for my EUE talk roundup. I hope you’ve been able to get something out of it. If any part makes someone want to start coding useful tools to help their company productions then it’s all been worthwhile. For anyone reading this that attended my talk, thanks for turning up and not throwing anything. If you have any questions, feel free to contact me.

All versions of the script are available to download below.

If you want a PDF of the visual slide material that in the background, you can find it here