Marking Menus in Maya. Part 3: Customizing

Tutorial / 07 January 2018

Initially I planned on going through creating a regular marking menu from scratch and then move on to customizing the default marking menus, you know the semi-secret stuff not really documented by Autodesk. The first version of this blog post was waaay to long so I decided to just skip the first part and go straight to the sweet super secret tips and tricks 😉

So if you’re looking for a tutorial on how to use the default Marking Menu Editor, look no further than the official Maya help.

If you're new to my series, you can check Part 1 and Part 2 for more insight into Maya's marking menu system.

Disclaimer: This post is a bit more advanced than the previous parts so Mel experience is required as I do not plan on explaining Mel basics here. The specifics apply to Maya 2018 as previous versions have different ways of handling the customization.

Background information:

Around 2013 Autodesk started implemented contextual marking menus in Maya (see Part 2 for details). I really liked them so I decided to ditch my entire sweet custom marking menu system I had until that point and learn the defaults. I figured that this will allow me to mode easily adjust to new features added by each Maya version without having to redo my marking menus.

User Defined Context Marking Menus:

All in all, it was a good decision, yet I always felt some things were missing. While searching for how to customize these marking menus I found them all to reside here:

C:\Program Files\Autodesk\Maya2018\scripts\others

All contextual marking menus consist of two mel scripts, their names always start with “context”.

Most of them are commented properly so you can deduce a lot of their functionality by just reading them 😉


Let’s take for example contextPolyToolsDefaultMM.mel

There is also a companion script for each one which has a very similar name except the “.res”:  contextPolyToolsDefaultMM.res.mel. This script contains the display text for each command, they did this for localization purposes. So for example if you set up Maya’s interface to use something else other than English, it will load the .res files from a different location. The commands will remain untouched, only the display text will change.

This creates the default SHIFT+RMB marking menu when nothing is selected.

While exploring previous versions I found that most context* scripts have this at the end:

if (`exists contextPolyToolsDefaultUserMM`)
contextPolyToolsDefaultUserMM $parent;

This loads a user generated mel file (contextPolyToolsDefaultUserMM.mel) which can be used to modify the default one. This is very helpful since these script files change from version to version and having your customizations as separate files makes upgrading from one Maya version to another a lot simpler.

Now depending on version, this code might reside in different places:

  1. For Maya 2018 all the *UserMM are loaded by a single file: contextToolsMM.mel At this moment some of this functionality is broken (a bug probably). Scroll down to “If all all else fails:” category for a simple work-around.
  2. For Maya 2017 and older, each *UserMM file is loaded its corresponding default file. For our example this is: contextPolyToolsDefaultMM.mel

In simple terms, if you want to create your own “extension”, just create your own *UserMM file and copy the name of the default file and add “User” before “MM”. You can avoid loosing these files by updating to a new Maya version or un-installing by placing them in the user script directory, usually:

My Documents\maya\2018\scripts

All you have to do is to copy the original file, being careful to change the name of it and of the global procedure to match the mel file. If you do this and the procedure name matches the file name and the file is in the scripts folder, Maya will source it automatically.


Examples:

The first example inserts a new command (2) at the top of the south extension of the original marking menu (1):



contextPolyToolsDefaultUserMM.mel

global proc contextPolyToolsDefaultUserMM( string $parent )
{
            if (`popupMenu -query -exists $parent`){
            popupMenu -edit $parent;
                                    menuItem
                                                -label (uiRes("m_contextPolyToolsDefaultUserMM.k2DPanZoom"))
                                                -command "panZoomCtx -e -zoomMode PanZoomContext;setToolTo PanZoomContext;"
                                                -image "PanZoom.png" 
                                                -ia ""                                       
                                                ;
                                    setParent -menu $parent;
            }
}

The companion script looks like this:

contextPolyToolsDefaultUserMM.res.mel

displayString -replace -value "2D Pan/Zoom" m_contextPolyToolsDefaultUserMM.k2DPanZoom;

Each item in the marking menu item is defined with the menuItem command. If you read the documentation, you’ll notice this command returns an ID but the default implementation in Maya does not use this functionality and relies on creating and destroying new IDs on each call of the marking menu, sorting being done by creating each item one at a time.

This is worth mentioning since this seriously reduces the usage of the -ia (insert after) flag. There is no way of controlling where your custom button gets inserted since by default you don’t know which ID you want to “insert after”. Setting this to ” (NULL) inserts the command at the top of the marking menu.

The -label flag uses the uiRes command which reads the display text from the m_contextPolyToolsDefaultUserMM.k2DPanZoom variable. This is set up by the .res file mentioned previously, the structure should be easily recognizable from my example so I won’t go into the details since you can replace this whole label with a simple: 

-label “2D Pan/Zoom"

 and skip the whole localization file altogether.

The second example will overwrite a menuItem in the radial part of the face marking menu.

 contextPolyToolsFaceUserMM.mel

 global proc contextPolyToolsFaceUserMM( string $parent )
{
            if (`popupMenu -query -exists $parent`){
            popupMenu -edit $parent;
                                    menuItem
                                                -ia ""
                                                -label (uiRes("m_contextPolyToolsFaceUserMM.kDeleteFace"))
                                                -command "doDelete"
                                                -radialPosition "SW" 
                                                -enableCommandRepeat 1
                                                -image "polyDelFacet.png"
                                    setParent -menu $parent;
                                   
            }
}

Some of you will notice there are a few more changes in the image which are not reflected in the code above, I’ve chosen to only show one item for the sake of simplicity.

Usually we would use the -e (edit) flag to edit an already existing item but as I’ve mentioned before, there’s no easy way of obtaining the ID of the original without complicating matters too much so the trick I’ve found is using -ia “” flag which makes the item be recreated at the “beginning”, practically overwriting the item that was initially created in the original mel file.

Last but not least, the -radialPosition flag defines the position of the newly created item.

For more details, here’s the help for menuItem.

If all else fails:

As previously mentioned, Maya 2018 might be bugged so you might find that even if you named your *UserMM files properly, they don’t get automatically sourced and they don’t’ work. To go around this we can edit this file:

My Documents\maya\2018\scripts\userSetup.mel

This gets run at every Maya start so we can add a command to source the context files and make sure they work. Since I’m a bit of a neat freak (if that wasn’t clear from the whole .res thing above), I don’t actually load each context mel here, but instead use this line:

source userContextMenus;

..to source userContextMenus.mel which looks like this:

source contextPolyToolsDefaultUserMM.mel;
source contextPolyToolsDefaultUserMM.res.mel;
source contextPolyToolsEdgeUserMM.mel;
source contextPolyToolsEdgeUserMM.res.mel;
source contextPolyToolsFaceUserMM.mel;
source contextPolyToolsFaceUserMM.res.mel;
source contextPolyToolsObjectUserMM.mel;
source contextPolyToolsObjectUserMM.res.mel;
source contextPolyToolsVertexUserMM.mel;
source contextPolyToolsVertexUserMM.res.mel;
source contextUVToolsEdgeUserMM.mel;
source contextUVToolsEdgeUserMM.res.mel;
source contextUVToolsFaceUserMM.mel;
source contextUVToolsFaceUserMM.res.mel;
source contextUVToolsUVUserMM.mel;
source contextUVToolsUVUserMM.res.mel;

 You guessed it, it sources a file that sources some files! Why is that? Well, I guess Autodesk will fix the loading bug at some point so all I have to do is delete this file (userContextMenus.mel) and the line from userSetup.mel call it a day.

You could also use the code above to identify the context marking menus that can be customized and search their corresponding defaults in the Maya installation directory.