Increasing demand for oil as the main source of energy supply around the world has urged different countries to maximize oil recovery from their oil reservoirs. Relative permeability and capillary pressure are of the rock and fluid properties and are among the key parameters in oil reservoirs. Hence, during the years of research and study, in order to estimate these two parameters, different kinds of analytical and experimental methods have been developed both of which have advantages and disadvantages. In the first part of this research, relative permeability and capillry pressure in two phase systems of oil-water and gas-oil were estimated using the ant colony optimization in an innovative way. First of all, fundamentals of ant colony optimization iired by real ants were developed in a creative way. After then, by connecting the developed model with coreflood simulator, relative permeability and capillary pressure values were successfully estimated. Comparing the obtained results from developed model and those of the analytical JBN method implies the higher accuracy of proposed model in prediction of relative permeability and capillary pressure. The main advantage of proposed model is that it considers capillary effects in simulation process which in turn, facilitates prediction of capillary pressure values as well as the relative permeability ones. In the most of analytical methods, however, capillary effects are considered to be negligible. Using the developed model in this research, multi-dimentional flows as well as three phase systems could be covered. It is worth noting that analytical methods of prediction of relative permeability (like JBN) have serious limitations in this regard. As one of the most important methods on enhancing recovery, in the second part of this project, water alternating gas injection method was studied. At first, principles and mechanisms of WAG process was discussed. Considering the establishment of three phase region in the WAG injection processes, fundamentals and different methods of calculation of three phase relative permeability was surveyed. Finally, fluid flow equations for immiscible water alternating gas injection were presented and simulation was carried out using the coreflood simulator. Then, some of the most effective parameters on WAG performance like water gas ratio, relative permeability, hysteresis effects etc. were discussed in a sensitivity analysis and optimum values were presented for each case. The results of simulation revealed that the WAG injection should be optimized before implementation and using this method could inflict significant operational costs as well as having the efficiency less than the gas injection or water-flooding alone. K e y Words Relative Permeability, Capillary Pressure, Optimization, Enhanced Oil Recovery, Oil Reservoir, Ant Clony