We present and discuss gradient-based optimization methods for solving minimization problems arising in machine learning applications. The analysed methods employ first-order models for the objective function and stochastic gradient approximations. We will focus on the stochastic gradient and some of its widely used variants. The application of such methods to the training of classifiers and neural networks will be considered and numerical results will be also shown.