The short version and an important notice!

This script builds on the idea of having weighted vertex normals and allows the user to "snap" normals to faces to create better looking smoothing for low poly hard surface objects. It does not lock unnecessary vertex normals and does not break hard edges which differentiates it from all other similar scripts out there!
I've made this script some time ago while I still had my Maya license. Due to various reasons (detailed here and here), I have ditched Maya for Blender. At the time of this blog post I do not own a Maya license and since it's illegal to use the Student Version for any commercial work like putting scripts on Gumroad or recording free tutorials on Youtube I cannot take screenshots or do a tutorial on how to use my script. I hope this will change at some point as Maya Indie becomes available worldwide and I buy a subscription.

A bit of technical background

(Feel free to skip this section if you're familiar to how weighted normals work)

A couple of years ago Autodesk introduced weighted vertex normals to Maya, paving the way to better vertex normal shading. 

For each vertex in a mesh, the new algorithm would calculate the normal as a weighted average of the surrounding faces' normals. The weighting is done by surface area so the resulting normal will align more with the larger faces.

The advantages are significant for low poly meshes since it reduces the amount of gradients in the shading. These lesser gradients also get reflected in less gradients in baked normal maps which make these normal maps easier to compress and reduce those bad shading artifacts that are caused by the compression algorithm.

The script will align (and lock) the vertex normals of the current selection to the normals of the selected faces. The effect is similar to Maya's weighted normals setting but the resulting normals will be "snapped" rather than just weighted.

This allows for smoother shading on low poly hard surface materials that use bevels. This will also result in cleaner normal map bakes with less gradients which in turn will cause less normal map compression artifacts.

You can download my script here:

https://gum.co/vrnNormalifique

Installation:

1. Copy the files into the script directory, usually:

My Documents/maya/<version>/scripts

2. Restart Maya or execute "rehash" in the Script Editor.

Usage:

Call the function with either commands:

vrnNormalifique(0);

-will set the perimeter normals so that they snap to the current

selected faces

vrnNormalifique(1);

-will set all the normals in the current selection to the average normal

of the entire selection

Bonus commands:

vrnSoftEdge;

-softens edges while keeping custom normals

vrnCopyNormal;

-copies the normal of the current selection

vrnPasteNormal;

-pastes the normal to the current selection

Requirements:

Maya 2017 or newer (might work in previous versions although not tested)

Version History:

v 0.1 alpha

-initial version

-limitation: internal hard edges relative to initial selection are

not preserved

v 0.2 beta

-added vrnSoftEdge procedure which soften edges while preserving custom normals

-rewrote from scratch with new logic based on vrnSoftEdge

v 1

-initial public release

-fixed some bugs

v 1.1

-added bonus vrnCopyNormal and vrnPasteNormal

If you've ever tried loading a shiny normal mapped low poly object to Sketchfab, you might have noticed the horrendous compression artifacts you get not in the editor but in the viewer. Worst thing about it is that the editor uses uncompressed textures which are also visible in the viewer after saving and exiting. That's because the compression is not done locally on the user's machine but on Sketchfab's servers. It takes a couple of hours before the textures get compressed, at which point the model transforms and looks pretty bad (1). This means most users don't bother to recheck the textures after compression and are not aware of the quality degradation that occurs.

Here's an example:

To reduce this, you could try forcing the viewer to download the uncompressed textures by appending this to your link (2):

?image_compression=0

So the link for the Chevy Pickup will look like this:

https://sketchfab.com/models/1bcf5ad3f59645ca9693cd76d6c56d0c?image_compression=0

As you can see below, the textures look a lot better. The great news is that you can also append this to the short links and use them when embedding in a Artstation page or blog post!

There's another layer of confusion if you're working with 16 bit images hoping to get around the compression somehow. It's the fact that Sketchfab does not dither the 16 bit image when taking it down to 8 bit before compression. At the time of this article, Sketchfab does not support 16bit normal maps, although they are working on it. So importing 16bit pngs are a no go, there's no quality benefit from uploading these.

Last layer of confusion is for users of Substance Painter. If you work in 16bit and you use Painter to upload directly to Sketchfab, you might have noticed that when exporting with the Sketchfab preset, you get locked out of controlling the texture file format and bit depth:

Although the format reads JPEG, the normal maps are saved in PNG format without compression to avoid dual compression when uploading to Sketchfab. The dithering issue is still there though, Substance Painter does not dither the file when reducing the bitdepth so you get banding artifacts. Here's a comparison between the PNG exported from Painter with no dither and one exported in 16bit and then manually reduced to 8bit with dithering in Photoshop by using Image->Mode->8 Bits/Channel.

So there are two problems here: Sketchfab's normal map compression and the 16bit to 8bit compression issues. Depending on how you work, I would suggest the following depending on your initial normal map bakes:

8 Bit / Channel bakes:

I would advise against working with 8bit bakes but if by any chance you HAVE too or you already have a bunch of assets that were created from start to finish in 8 bit and don't want to rebake, using the ?image_compression=0 trick above will help improve final quality.

16 Bit / Channel bakes:

I advise using 16 bit normal maps for as much of the pipeline as possible and only reduce the textures to 8 bit / channel when the final normal map is completed. This ensures you get the maximum amount of information in your normal maps through-out the texturing process. 

a. Bake the normal map in 16 bit.

I use Marmoset Toolbag 3 for my bakes but most bakers such as Substance Painter and Designer as well as xNormal support 16 bit normal map baking.

b. Texture in 16 bit.

If you're using Painter make sure you're still working in 16 bit. It should be on by default. This will make the generators and filters that use the normal map take advantage of the increased bit depth to create smooth effects.

PS: If you're using Photoshop or any other texturing tool, also make sure to be working with 16 bit normal maps through-out the texturing process.

c. Export the final normal map in a format that supports 16bit.

Here's an example for Substance Painter:

d. Open the normal map in Photoshop and reduce the bit depth. Dithering will be applied automatically:

If you want to get the maximum quality from Sketchfab, you can export using the Substance Painter plugin but then in the Sketchfab editor replace your normal maps with the ones you saved at c. and d. above. Idea taken from this thread (3).

Finally, you want to use the ?image_compression=0 links to share your work!

References and further reading:

(1) https://help.sketchfab.com/hc/en-us/articles/202600873-Materials-and-Textures#textures-processing

(2) https://forum.sketchfab.com/t/image-compression-issues/2839/59

(3) https://forum.sketchfab.com/t/substance-painter-to-sketchfab-a-help-guide-to-normal-maps-and-triangulation/12074/2

Initially I planned on going through creating a regular marking menu from scratch and then move on to customizing the default marking menus, you know the semi-secret stuff not really documented by Autodesk. The first version of this blog post was waaay to long so I decided to just skip the first part and go straight to the sweet super secret tips and tricks 😉

So if you’re looking for a tutorial on how to use the default Marking Menu Editor, look no further than the official Maya help.

If you're new to my series, you can check Part 1 and Part 2 for more insight into Maya's marking menu system.

Disclaimer: This post is a bit more advanced than the previous parts so Mel experience is required as I do not plan on explaining Mel basics here. The specifics apply to Maya 2018 as previous versions have different ways of handling the customization.

Background information:

Around 2013 Autodesk started implemented contextual marking menus in Maya (see Part 2 for details). I really liked them so I decided to ditch my entire sweet custom marking menu system I had until that point and learn the defaults. I figured that this will allow me to mode easily adjust to new features added by each Maya version without having to redo my marking menus.

User Defined Context Marking Menus:

All in all, it was a good decision, yet I always felt some things were missing. While searching for how to customize these marking menus I found them all to reside here:

C:\Program Files\Autodesk\Maya2018\scripts\others

All contextual marking menus consist of two mel scripts, their names always start with “context”.

Most of them are commented properly so you can deduce a lot of their functionality by just reading them 😉

Let’s take for example contextPolyToolsDefaultMM.mel

There is also a companion script for each one which has a very similar name except the “.res”:  contextPolyToolsDefaultMM.res.mel. This script contains the display text for each command, they did this for localization purposes. So for example if you set up Maya’s interface to use something else other than English, it will load the .res files from a different location. The commands will remain untouched, only the display text will change.

This creates the default SHIFT+RMB marking menu when nothing is selected.

While exploring previous versions I found that most context* scripts have this at the end:

if (`exists contextPolyToolsDefaultUserMM`)
contextPolyToolsDefaultUserMM $parent;

This loads a user generated mel file (contextPolyToolsDefaultUserMM.mel) which can be used to modify the default one. This is very helpful since these script files change from version to version and having your customizations as separate files makes upgrading from one Maya version to another a lot simpler.

Now depending on version, this code might reside in different places:

1. For Maya 2018 all the *UserMM are loaded by a single file: contextToolsMM.mel At this moment some of this functionality is broken (a bug probably). Scroll down to “If all all else fails:” category for a simple work-around.
2. For Maya 2017 and older, each *UserMM file is loaded its corresponding default file. For our example this is: contextPolyToolsDefaultMM.mel

In simple terms, if you want to create your own “extension”, just create your own *UserMM file and copy the name of the default file and add “User” before “MM”. You can avoid loosing these files by updating to a new Maya version or un-installing by placing them in the user script directory, usually:

My Documents\maya\2018\scripts

All you have to do is to copy the original file, being careful to change the name of it and of the global procedure to match the mel file. If you do this and the procedure name matches the file name and the file is in the scripts folder, Maya will source it automatically.

Examples:

The first example inserts a new command (2) at the top of the south extension of the original marking menu (1):

contextPolyToolsDefaultUserMM.mel

global proc contextPolyToolsDefaultUserMM( string $parent )
{
            if (`popupMenu -query -exists $parent`){
            popupMenu -edit $parent;
                                    menuItem
                                                -label (uiRes("m_contextPolyToolsDefaultUserMM.k2DPanZoom"))
                                                -command "panZoomCtx -e -zoomMode PanZoomContext;setToolTo PanZoomContext;"
                                                -image "PanZoom.png" 
                                                -ia ""                                       
                                                ;
                                    setParent -menu $parent;
            }
}

The companion script looks like this:

contextPolyToolsDefaultUserMM.res.mel

displayString -replace -value "2D Pan/Zoom" m_contextPolyToolsDefaultUserMM.k2DPanZoom;

Each item in the marking menu item is defined with the menuItem command. If you read the documentation, you’ll notice this command returns an ID but the default implementation in Maya does not use this functionality and relies on creating and destroying new IDs on each call of the marking menu, sorting being done by creating each item one at a time.

This is worth mentioning since this seriously reduces the usage of the -ia (insert after) flag. There is no way of controlling where your custom button gets inserted since by default you don’t know which ID you want to “insert after”. Setting this to “” (NULL) inserts the command at the top of the marking menu.

The -label flag uses the uiRes command which reads the display text from the m_contextPolyToolsDefaultUserMM.k2DPanZoom variable. This is set up by the .res file mentioned previously, the structure should be easily recognizable from my example so I won’t go into the details since you can replace this whole label with a simple: 

-label “2D Pan/Zoom"

 and skip the whole localization file altogether.

The second example will overwrite a menuItem in the radial part of the face marking menu.

 contextPolyToolsFaceUserMM.mel

 global proc contextPolyToolsFaceUserMM( string $parent )
{
            if (`popupMenu -query -exists $parent`){
            popupMenu -edit $parent;
                                    menuItem
                                                -ia ""
                                                -label (uiRes("m_contextPolyToolsFaceUserMM.kDeleteFace"))
                                                -command "doDelete"
                                                -radialPosition "SW" 
                                                -enableCommandRepeat 1
                                                -image "polyDelFacet.png"
                                    setParent -menu $parent;
                                   
            }
}

Some of you will notice there are a few more changes in the image which are not reflected in the code above, I’ve chosen to only show one item for the sake of simplicity.

Usually we would use the -e (edit) flag to edit an already existing item but as I’ve mentioned before, there’s no easy way of obtaining the ID of the original without complicating matters too much so the trick I’ve found is using -ia “” flag which makes the item be recreated at the “beginning”, practically overwriting the item that was initially created in the original mel file.

Last but not least, the -radialPosition flag defines the position of the newly created item.

For more details, here’s the help for menuItem.

If all else fails:

As previously mentioned, Maya 2018 might be bugged so you might find that even if you named your *UserMM files properly, they don’t get automatically sourced and they don’t’ work. To go around this we can edit this file:

My Documents\maya\2018\scripts\userSetup.mel

This gets run at every Maya start so we can add a command to source the context files and make sure they work. Since I’m a bit of a neat freak (if that wasn’t clear from the whole .res thing above), I don’t actually load each context mel here, but instead use this line:

source userContextMenus;

..to source userContextMenus.mel which looks like this:

source contextPolyToolsDefaultUserMM.mel;
source contextPolyToolsDefaultUserMM.res.mel;
source contextPolyToolsEdgeUserMM.mel;
source contextPolyToolsEdgeUserMM.res.mel;
source contextPolyToolsFaceUserMM.mel;
source contextPolyToolsFaceUserMM.res.mel;
source contextPolyToolsObjectUserMM.mel;
source contextPolyToolsObjectUserMM.res.mel;
source contextPolyToolsVertexUserMM.mel;
source contextPolyToolsVertexUserMM.res.mel;
source contextUVToolsEdgeUserMM.mel;
source contextUVToolsEdgeUserMM.res.mel;
source contextUVToolsFaceUserMM.mel;
source contextUVToolsFaceUserMM.res.mel;
source contextUVToolsUVUserMM.mel;
source contextUVToolsUVUserMM.res.mel;

 You guessed it, it sources a file that sources some files! Why is that? Well, I guess Autodesk will fix the loading bug at some point so all I have to do is delete this file (userContextMenus.mel) and the line from userSetup.mel call it a day.

You could also use the code above to identify the context marking menus that can be customized and search their corresponding defaults in the Maya installation directory.

In Part 1 we covered the basics of how marking menus work. In this part we'll cover how they integrate in Maya's UI and UX. There are some interesting things here so even if you've used them before, I would recommend giving this a good read as it might help you better understand the philosophy behind marking menus.

Here’s a quick run though of their main characteristics:

Marking Menus are cursor-centered.

Marking Menus allow access to more commands with less shortcuts.

Marking Menus are muscle-memory based.

This will come in handy when we move to Part 3 and customize them. The default marking menus I will be referencing all revolve around polygon modeling but as you will see, they also apply to other areas of Maya.

To make it easier to list shortcuts, they will be marked in bold italic text I will use the following notation for buttons.

SHIFT - the shift key

CTRL - the control key

ALT - the alt key

LMB - left mouse button

MMB - middle mouse button

RMB - right mouse button

(N), (S), (SE), etc - the direction of the command where applicable.

All marking menus are visible for as long as the mouse button in the combo is pressed. It is therefore implied that the mouse button has to be held pressed for the marking menu to be visible.

Shortcut consistency across the interface.

We'll start off with a bit of a tangent about the mouse buttons in Maya.

By default, LMB is a selection button, MMB is an action button and RMB is a marking menu button. All the default marking menus use the RMB but as you will see in Part 3, that doesn't have to be the case for all custom marking menus. This default mouse behavior carries across the entire interface which means that for example text selection is done with LMB but drag and dropping is done with MMB. This might look a bit heavy-handed for first time users but it's just the result of keeping things consistent. As a side note, the LMB can be used as an action button but when clicking on (i.e. selecting) a control, such as one of the axis of the move manipulator tool. This does not contradict the philosophy, it's merely a shortcut.

Still, even some more advanced users have trouble understanding why the separation between these two.

 Here's an example of using the manipulator tool to move an object that is just off-screen, so the user cannot click and drag on the manipulator itself. But you can select an axis then reposition the camera and then click and drag MMB to move the object along the selected axis.

The lesson here is that Maya's UI/UX quirks are a result of careful design and years of evolution. As you will see going forward, this design process and the amount of thought put into the UI/UX will become more evident.

Default Marking Menus are contextual.

That’s how they're referred to in the help: contextual marking menus.

I will demonstrate the most basic and universal marking menus which are almost all you need to remember if you plan on do 3d modeling in Maya. I am not exaggerating this one bit.

Let's start with the most basic marking menu:

• RMB

That’s easy! Just click and hold the RMB. Doing this on elements of the interface will pop up a regular context menu but holding RMB in the viewport will create this marking menu:

It doesn't look like much since there's nothing under the mouse cursor. It is worth mentioning that there are two very useful commands here: Select All and Complete Tool which are very helpful. So, every time the user needs to select all objects in a scene, this can be accomplished with a simple RMB flick in the S direction with the mouse over an empty space in the 3d scene. Same thing about completing a tool such as Multi-Cut.

The contextual part kicks in when the user holds RMB over an object, in which case the component marking menu pops up.  This menu differs based on the selection or the type of object the mouse cursor is over. Here are two examples, for meshes and NURBS surfaces respectively:

Marking Menus are ergonomic.

Of course, you could use the F8 to F12 keys to switch between object mode and each component mode. But there’s a catch, doing so means you have to lift either the left or the right hand from their regular resting places: the left of the keyboard and the mouse, respectively.  This decreases speed and increases fatigue.

Maya’s default interaction system is very ergonomic, the most important keyboard shortcuts are all concentrated on the left side of the keyboard, so the user won’t have to move the left hand too much to reach them. Similarly, marking menus allow the user to bring menus to the cursor, minimizing the movement of the right hand.

Next up, working with selections: selecting loops, rings, converting selections, all can be done from this marking menu:

• CTRL+RMB

This one is pretty much self-explanatory as far as content goes, yet it’s very important to note the position of each type of component conversion, like the positions in the component marking menu (RMB): (W) is for vertices, (N) for edges, (E) for UVs, (S) for faces.

As mentioned in Part 1, marking menus are muscle memory based, so having similar commands in similar directions of different marking menus makes learning them easier.

Next up, accessing tools and commands, they can be accessed from this marking menu:

• SHIFT+RMB

This marking menu pulls out all the tricks to accommodate almost all tools and commands related to polygonal modeling.

First off, this menu is different depending on the object the mouse is over or it’s component selection mode.

If there’s nothing under the mouse cursor, the menu contains commands and tools to create geometry from scratch: primitives, the Create Polygon Tool, etc. See the image above.

For polygon meshes in object selection mode, it contains:

And for component mode, here are the default menus for each component:

You can also observe the same consistency in direction, for example the Extrude command is always in the (S) direction for vertices, edges and faces.

• CTRL+SHIFT+RMB

This one is a bit more obscure but it’s a universal marking menu that houses the settings for the current tool. For example, if you’re using the Move Tool or Multi-Cut or Extrude you can access its settings without having to go searching for buttons or open new windows.

The reason I mention these four marking menus is that you don’t need a cheat sheet to remember them, all you need is the RMB and the good old modifier keys CTRL and SHIFT.

The analogy I use is working in Maya is like playing the violin while working in a program like 3d Max is like playing the piano or the drums if you get angry at the keyboard.

As a recap, here’s an easy table of these marking menus and what they do:

In Part 3 we’ll discuss creating custom marking menus from scratch as well as modifying the default marking menus.

One of Maya's cool features is the ability to assign different commands for pressing and releasing a key or key combo.

This feature is used by all marking menus: pressing the key will generate the menu, releasing it will destroy it.

We can take advantage of this feature and create some cool scripts. For example we can create a selection filter using the inbuilt polySelectConstraint command. This is usually available through the interface in the top menus under Select->Select Constraints... Unfortunately the design of the window that houses the commands is a bit confusing as it changes based on current selection. In needs a lot of clicks even for a simple operation such as selecting hard edges.

I've found myself using the hard edge filter a lot. I've also used Select Constraints to select planar or near-planar faces so I decided to just go ahead and create a hotkey that would activate these filters based on current component selection. Really, why should contextual behavior be limited to marking menus?

Create a new hotkey and after setting you desired shortcut for it, make sure it's set to "On Press" by clicking the small triangle at the right of the text box:

Use the following code for the shortcut:

//While hotkey is pressed, check selection mode:
//If edge mode -> filter by hard edge
if (`selectType -q -pe` == 1){
polySelectConstraint -m 1 -t 0x8000 -sm 1;
}
//If face mode -> filter by face angle. Adjust -at attribute as desired.
else if (`selectType -q -pf` == 1){
polySelectConstraint -ap 1 -at 5;
}
//End

Similarly, create a new hotkey, use the same key but set it to "On Release". For the command, use this code:

//When hotkey is released, check selection mode:
//If edge mode -> revert edge constraing mode
if (`selectType -q -pe` == 1){
polySelectConstraint -sm 0;
}
//If face mode -> revert face constraing mode
else if (`selectType -q -pf` == 1){
polySelectConstraint -ap 0;
}
//End

PS: I found that using resetting all constraints is buggy sometimes so for the release part it's more reliable to revert each constraint as needed.

If you know Maya, you've probably used marking menus before. If not, you've probably heard Maya users mentioning them every time they want to prove you Maya's superiority (Maya Master Race FTW). Other programs might have something similar, but Maya still has the edge due to usability and customization options. This is a three-part series that will take you through the marking menu system in Maya.

Part 1 (this one) will cover the basics. If you're either a Maya beginner trying to make sense of them or you don't use Maya but want to know what the fuss is about, this is for you!

Part 2 will cover the default marking menus and how to use them properly. If you want to master the marking menus and never use the regular interface again, this is the article to read.

Part 3 will cover creating marking menus from scratch or editing the default marking menus using some features that are not properly documented in Maya's help. I will share my mel files with everyone, so you can edit your own marking menus.

Disclaimer: All through-out these articles I will be referencing Maya 2018 marking menus. Most will apply to previous versions as well but expect bigger differences the older your preferred Maya version is. In the future I will write an article about Maya versions and why it’s probably better stay up-to-date.

Marking Menus are cursor-centered.

Marking menus are radial menus that exist in the interface only for the duration of hotkey combination being pressed. This hotkey combination usually consists of a keyboard key and a mouse button pressed. They're always created where the mouse cursor is so they effectively bring the interface to the mouse cursor, the focus point of every 3d modeling app. They're very similar to mouse gestures first introduced in the Opera browser a few years back.

In the old school windows-style graphics UI, the user has to remember button positions in the interface and navigate them like a librarian searching for a book. Marking menus transforms this by bringing all the buttons to your fingertips, look at it like some sort of visual Google for the UI.

Marking Menus allow access to more commands with less shortcuts.

The only apparent downside for beginners is the steep learning curve due to their unusual interaction model. They look pretty daunting at first with all of those buttons crammed in that unusual radial pattern. From my experience with Maya both as an artist and as a teacher, I can promise that the steep learning curve pays off very fast, as marking menus takes out the hassle of remembering a lot of keyboard shortcuts. We're talking fractions of seconds here but in time they add up to a lot. Being able to quickly access commands means you can do more in less time, make more mistakes and learn faster!

Marking Menus are muscle-memory based.

Each marking menu allows the user to access 8 commands by flicking the mouse in their direction. These are marked by their geographical direction: N for north, NE for north-east and so on. There is no need to click these areas, releasing the mouse button will automatically activate the button in that direction. (See (1) in the image above)

There are also option boxes for some commands as well as an extension in the south direction which allows access to several commands which all have to be precisely activated. (2) and (3) respectively in the image above.

Beyond they cold hard efficiency, marking menus evolve with the artist. A beginner can take their time, activate a marking menu and visually choose the button they want to press. Once muscle memory starts building up, the marking menu items can be activated and with only a flick in the right direction, making them blistering fast, so fast that they’re barely visible on the screen. This makes working with Maya's interface like playing a musical instrument where hitting all the right notes becomes second nature after a while without having to concentrate on the movements or visually aiming for each string/drum/key.

These would be the basics, next up we'll go through how marking menus are integrated into the Maya UI/UX. Click here for Part 2.