Abstract: Engineering design often involves qualitative and quantitative design variables, which requires systematic methods for the exploration of these mixed-variable design spaces. Existing machine learning (ML) models that can handle mixed variables as inputs require a large amount of data but do not provide uncertainty quantification that is crucial for sequential (adaptive) design of experiments. We have developed a novel Latent Variable Gaussian Process (LVGP) based ML approach that involves a latent variable (LV) representation of qualitative inputs, and automatically discovers a categorical-to-numerical nonlinear map that transforms the underlying high dimensional physical attributes into the LV space. The nonlinear mapping also provides an inherent ordering and structure for the levels of the qualitative factor(s), which leads to substantial insight and interpretable ML. In addition, LVGP provides uncertainty quantification of prediction which is critical for adaptive sampling to sequentially choose samples based on current observations and the method also offers easy integration with Bayesian optimization (BO) or other reinforcement learning strategies for the purpose of design optimization. We will demonstrate the benefits of the LVGP approach using designs of emerging microstructural and metamaterials systems as examples. Furthermore, the LVGP approach has been extended to non-hierarchical multi-fidelity modeling and adaptive sampling to benefit a wide range of engineering problems that involve multi-fidelity or multi-modal data fusion.