Cognitive function is considered one of the most important factors for the quality of life in the elderly, as it contributes to the successful performance of activities of every day living. The general slowing associated with aging, has a negative effect on cognitive function. Physical activity has a positive effect on cognitive function, particularly in tasks with high attentional demands, as choice reaction time and digit symbol substitution. It might also positively affect motor task learning. Exercise programs contribute to improvements of simple reaction time, only when they last over one year, unless the participants’ level of physical activity is very low. Exercise programs of shorter duration result to an immediate effect on some cognitive tasks, as improvements in working memory and increased complexity tasks are reported after four months of exercise. The combination of physical and cognitive activity might also result to immediate positive effects in such tasks. Those findings indicate the need for physical activity in as long age as it is possible. The limitations that elderly persons face, not only due to their age but also due to their level of physical activity, should be taken under consideration when designing intervention programs. It is also apparent that there is a need for more research on the effects of physical activity on cognitive function.