NickName:PRCube Ask DateTime:2016-11-18T16:33:31 Behavioral difference between Gradient Desent and Hill Climbing I'm trying to understand the difference between these two algorithms and how they differ in solving a problem. I have looked at the algorithms and the internals of them. It would be good to hear from others who already experienced with them. Specially, I would like to know how they would behave differently on the same problem. Thank you. Copyright Notice：Content Author:「PRCube」，Reproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.Link to original article：https://stackoverflow.com/questions/40672502/behavioral-difference-between-gradient-desent-and-hill-climbing Answers radbrawler 2016-11-18T09:09:13 Difference\n\nThe main difference between the two is the direction in which they move to reach the local minima (or maxima).\n\n\nIn Hill Climbing we move only one element of the vector space, we then calculate the value of function and replace it if the value improves. we keep on changing one element of the vector till we can't move in a direction such that position improves. In 3D sapce the move can be visualised as moving in any one of the axial direction along x,y or z axis.\nIn Gradient Descent we take steps in the direction of negative gradient of current point to reach the point of minima (positive in case of maxima). For eg, in 3D Space the direction need not to be an axial direction.\n",