Collaborative Filtering (CF) methods make recommendations just by using the rating matrix. However, in practice there is extensive side information about users and items that can solve the coldstart problem. To study the impact of side information on the recommender systems, two novel models based on the probabilistic graphical models are proposed. The first model, called Collaborative Poisson Factorization with Side-information (CPFS), yields an effective way of harnessing not only textual information, but also any kinds of categorical or real-valued side information about users and items. First, CPFS encodes all categorical or real-valued information to binary vectors, and then replaces the shape parameter of all prior distributions with a log-linear combination of the binarised features. We develop a Gi sampler and also a Variational method with closed-form updates for the inference of CPFS. The second model is a Bayesian graphical model, called Content-Aware Listwise Collaborative Filtering (CALCF), which incorporates text information into the listwise CF using Plackett-Luce and topic modeling. We propose a Gi sampler with closed-form samples using data augmentation techniques to infer the latent variables. The results demonstrate that in most cases CPFS and CALCF achieve better Normalized Discounted Cumulative Gain (NDCG) and Recall compared with the previous models.