A machine learning-based model is configured to make inferences about computer game actions to execute based on dynamic, varying player gestures and to translate those game actions into input sequence macros. In some instances, the button sequence mapping for the macros can even dynamically change based on game state so that different macros for the same computer game action might be inferred by the model depending on game state.