First we present a unified theory of connections on bundles necessary for the next studies. Then as an example we introduce the bundle of infinite jets. Then for a vector bundle (E,p,M), we show that there are two splitting theorems for TE at the presence of a connection. These splitting theorem may be seen as natural generalizations of Tensorial and Dombrowski's splitting theorems for the case of the tangent bundle. These splitting theorems have applications in the study of the geometry of bundle of accelerations. In spite of natural difficulties with non-Banach modelled vector bundles, we generalize these theorems for a wide which can be considered as projective limits of Banach vector bundles. For a given manifold M, modelled on a Banach space B, second order ordinary differential equation rovides an alternative way to study geometric structures on M. In this thesis for every connection D on M we associate a second order differential equation S in a way that the D-geodesics are geodesics with respect to S. In a further step despite of natural difficulties with non-Banach modelled manifolds, and even spaces, we generalize these results to a wide Frechet manifolds. More precisely we show that for a Frechet manifold M, which can be considered as projective limit of Banach manifolds, for a given initial value there exists a unique geodesic. In the sequel, for a vector bundle (p,E,M), we show that (s,TE,M) admits a vector bundle structure if and only if (p,E,M) is endowed with a linear connection. Moreover we clarify the relation between vector bundle structures and also the induced bundle morphisms which will be used for vector bundle structures. Afterward the concept of second order connections on a manifold M is introduced which leads us to the interesting geometric tools on the {bundle of accelerations}. In fact by using the v.b. structure for (s,TTM,M), we will study the geometric tools on the second order tangent bundle. The concepts of second order covariant derivative, first and second order auto-parallel curve, the appropriate exponential mapping and second order Lie derivative are introduced.