Machining is usually one of the most common and may be the most complicated cutting process. In order to produce high-quality surfaces in batch production with great amount of chipping, and to create high-quality surfaces at high speed a phenomenon called "chatter" encounter. Studying the chatter theory in drilling operation and analyzing the parameters influencing the system stability, a model can be offered to control the chatter vibrations during drilling operations. In drilling and other machining processes, we want to recognize the areas or circumstances in which the tool vibration becomes unstable, and to avoid trapping in these areas and situations which lead to the tool vibration instability. For this purpose, stability diagram method is one of the most credible methods and assessment criteria that show the border of stable and unstable regions in machining. In the current research, the chatter vibrations in drilling process are simulated using MATLAB software and with regards to the modeling performed, stability lobes are diagrammed. In order to determine stability areas, several methods can be used; one of them is simulating the vibrations in the time domain. By simulating the tool vibration, we can determine its stability or instability, and even specify drilled surface quality and dimensional accuracy of the created hole. Hence, by using simulation in the time domain, prior to drilling, we can assess the vibration stability and the hole tolerances and determine whenever the vibration is not stable and hole shape or size is not acceptable. This can be done by choosing appropriate cutting circumstances and therefore, without wasting money or spoiling the tool or workpiece, favorable quality can be obtained. In order to simulate in the time domain, estimation of shear forces is critical; increasing the force model accuracy resulted in more accurate results. For this purpose, a variety of models for estimation of shear forces in the drilling operation has been presented that each of them had some limitations.in addition to the rake angle, oblique angle, cutting width and helix angle of the drill along with the cutter lip have also effects in the shear pressure and their values are not constant. Therefore, in this thesis, a force model was proposed that considers all of these effects resulting greater application compared with the other models. In current model, variations of geometrical parameters along the cutter lip have been considered.In drilling process, determination of the vibration model needs more carefulness, because the tool impact with the workpiece is intensive and based on the material and various factors of the drill and the workpiece, the vibration model would be changed. Key Words Drilling operation, Chatter vibration, Stability diagrams, Time domain simulation, Force model