This thesis consists of three parts. In the first part an introduction to three special functions (gamma, betta and the Mittag-Leffler functions) is given. These functions play the most important role in the theory of fractional calculus and fractional differential equations. Next, some approaches to generalizations of the notion of differentiation and integration are considered. In each case, we start with integer-order derivative and integral and show how these notions are generalized using some selected approaches. We consider the Grunwald-Letnikov, the Riemann-Liouville and the Caputo fractional derivatives and integrals. Also properties of the considered fractional derivatives and integrals and the links between these approaches are introduced. In the second part compact finite difference methods at interior and boundary grid points are derived and an analysis of computational efficiency is given. Compact finite difference methods are high order implicit methods for numerical solutions of initial and/or boundary value problems modeled by differential equations. These methods generally require smaller stencils than the traditional explicit finite difference counterpart methods. Finally, in the third part, two of the most famous fractional partial differential equations, fractional wave-diffusion equation and fractional diffusion equation, are discussed. In the fractional wave-diffusion equation in order to approximate the second order derivative with respect to the space, the forth order compact finite difference method is applied and to approximate the Caputo fractional derivative with respect to time, the L1 approximation is utilized. Furthermore, stability and convergence of the method is proved using the energy method. For the fractional diffusion equation, first using the relation between Grunwald-Letnikov fractional derivative and Riemann-Liouville fractional derivative, the Riemann-Liouville fractional derivative is approximated. Moreover, to approximate the second order derivative with respect to the space, the forth order compact finite difference method is applied and the stability and convergence is prove using Fourier analysis. In the sequence, to increase the accuracy with respect to time, the fractional diffusion equation is converted to another equivalent equation where the Caputo fractional derivative appears. In this case also the L1 approximation is applied to approximate the Caputo fractional derivative and to approximate the second order derivative with respect to the space, the forth order compact finite difference method is once again applied and the stability and convergence is prove using the energy method. Finally, using several examples the efficiency of compact finite difference method in solving differential equations is demonstrated.